Skip to content

fix(fill): fix reported fixture source urls for the case of auto-generated tests #1488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 24, 2025

Conversation

danceratopz
Copy link
Member

🗒️ Description

Fixes the fixture_source_url field in the fixture _info section for the case of a test function who's body is simply pass. Here's an example of such a test function, whose test logic is applied via a decorate from an EEST framework generator:

@pytest.mark.parametrize(
"system_contract", [Address(Spec_EIP7251.CONSOLIDATION_REQUEST_PREDEPLOY_ADDRESS)]
)
@generate_system_contract_error_test(
max_gas_limit=Spec_EIP7251.SYSTEM_CALL_GAS_LIMIT,
)
def test_system_contract_errors():
"""
Test system contract raising different errors when called by the system account at the
end of the block execution.
To see the list of generated tests, please refer to the
`generate_system_contract_error_test` decorator definition.
"""
pass

In v4.3.0, consume test reports report an invalid URL (with a non-existent line number), e.g. https://hive.ethpandaops.io/#/test/pectra/1745242958-27275e8c3ce031d481359ddd1eb46b12?page=1&testnumber=4776

In these special cases, pytest reports the generator function as the test function, instead of the original test function. For this case, where the test module name doesn't match the module name reported by pytest, both files are written to the url string, e.g.,

https://github.com/ethereum/execution-spec-tests/blob/e7cc42c660c466fa955cd9983906c50a3e64aab3/src/ethereum_test_tools/utility/generators.py#L280 called via `test_system_contract_errors()` in https://github.com/ethereum/execution-spec-tests/blob/e7cc42c660c466fa955cd9983906c50a3e64aab3/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py

@pytest.mark.parametrize("test_type", [v.param() for v in SystemContractTestType])
called via test_system_contract_errors() in https://github.com/ethereum/execution-spec-tests/blob/e7cc42c660c466fa955cd9983906c50a3e64aab3/tests/prague/eip7251_consolidations/test_modified_consolidation_contract.py

This isn't perfect, I didn't find an easy way to get the test function's line number in this case.

🔗 Related Issues

None

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.

@danceratopz danceratopz added type:bug Something isn't working scope:consume Scope: Consume command suite scope:fill Scope: fill command labels Apr 23, 2025
@danceratopz danceratopz marked this pull request as ready for review April 23, 2025 12:35
@danceratopz danceratopz requested a review from felix314159 April 23, 2025 12:35
Copy link
Collaborator

@felix314159 felix314159 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, ty!

@danceratopz danceratopz merged commit 70a1183 into main Apr 24, 2025
21 checks passed
@danceratopz danceratopz deleted the fix/fill-reported-test-src-line-numer branch April 24, 2025 12:38
pacrob pushed a commit to pacrob/execution-spec-tests that referenced this pull request May 5, 2025
felix314159 pushed a commit to felix314159/execution-spec-tests that referenced this pull request May 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:consume Scope: Consume command suite scope:fill Scope: fill command type:bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants