Description
Background
As identified in PR #354, there's a need to refactor and unify the Makefile test targets to create a consistent entry point for CI/CD pipelines. The current implementation has multiple overlapping targets with similar functionality but different execution paths.
Current Implementation
The Makefile currently has several test-related targets:
test
- Runsjust-test
and combines coverage reportstest-pytest
- Marked as deprecated, with a message to useTESTS_USE_PYTEST=1 make test
insteadjust-test
- Runs tests using either pytest or unittest discover based onTESTS_USE_PYTEST
environment variabletest-mod
- Recently added target that runstest-mat
and combines coverage reports (similar structure totest
)test-mat
- Runs multiple specific test categories in a defined order
Goals
- Unify the test entry points to provide a consistent interface for both local development and CI/CD pipelines
- Ensure both pytest and unittest-based testing frameworks are supported seamlessly
- Reduce duplication in how coverage reports are generated and combined
- Maintain backward compatibility for existing workflows
- Improve the organization of test targets for better maintainability
Specific Targets for Refactoring
- Merge the functionality of
test-mod
andjust-test
to create a unified approach - Ensure
test
remains the primary entry point that works with both test frameworks - Consider deprecating redundant targets after the refactoring is complete
- Standardize how coverage reports are generated and combined
Insights and Proposed Approach
Looking at the current implementation, the main issues are:
- Duplication in how coverage is handled across targets
- Parallel execution paths for pytest vs. unittest frameworks
- Inconsistent naming conventions for test categories and groups
A potential approach would be to:
- Refactor
just-test
to handle both the current behavior and whattest-mod
does - Ensure proper environment variable checks to determine the test framework
- Create a unified coverage reporting mechanism
- Update documentation and deprecation notices
Related Work
This refactoring is related to the changes implemented in #199 and was identified during review of PR #354.
Note
The current test-mod
implementation is useful for local testing but is described as "hack-ish and incomplete" - it will serve as the starting point for this refactoring work.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
In progress