Skip to content

v2.1.x Chore - Refactor and unify Makefile test targets for consistent CI/CD entry points #356

@coderabbitai

Description

@coderabbitai

Background

As identified in PR #354, there's a need to refactor and unify the Makefile test targets to create a consistent entry point for CI/CD pipelines. The current implementation has multiple overlapping targets with similar functionality but different execution paths.

Current Implementation

The Makefile currently has several test-related targets:

  • test - Runs just-test and combines coverage reports
  • test-pytest - Marked as deprecated, with a message to use TESTS_USE_PYTEST=1 make test instead
  • just-test - Runs tests using either pytest or unittest discover based on TESTS_USE_PYTEST environment variable
  • test-mod - Recently added target that runs test-mat and combines coverage reports (similar structure to test)
  • test-mat - Runs multiple specific test categories in a defined order

Goals

  1. Unify the test entry points to provide a consistent interface for both local development and CI/CD pipelines
  2. Ensure both pytest and unittest-based testing frameworks are supported seamlessly
  3. Reduce duplication in how coverage reports are generated and combined
  4. Maintain backward compatibility for existing workflows
  5. Improve the organization of test targets for better maintainability

Specific Targets for Refactoring

  • Merge the functionality of test-mod and just-test to create a unified approach
  • Ensure test remains the primary entry point that works with both test frameworks
  • Consider deprecating redundant targets after the refactoring is complete
  • Standardize how coverage reports are generated and combined

Insights and Proposed Approach

Looking at the current implementation, the main issues are:

  1. Duplication in how coverage is handled across targets
  2. Parallel execution paths for pytest vs. unittest frameworks
  3. Inconsistent naming conventions for test categories and groups

A potential approach would be to:

  1. Refactor just-test to handle both the current behavior and what test-mod does
  2. Ensure proper environment variable checks to determine the test framework
  3. Create a unified coverage reporting mechanism
  4. Update documentation and deprecation notices

Related Work

This refactoring is related to the changes implemented in #199 and was identified during review of PR #354.

Note

The current test-mod implementation is useful for local testing but is described as "hack-ish and incomplete" - it will serve as the starting point for this refactoring work.

Metadata

Metadata

Labels

ChoreMiscellaneous chores to maintain the projectMake LangChanges to makefilesMulticastAny main project file changesTestingSomething can be verified

Type

Projects

Status

In progress

Relationships

None yet

Development

No branches or pull requests

Issue actions