[email protected]

First iteration of the EEST Fusaka Devnet-1 release. More test cases will be added for each EIP within the following releases.

Summary

Additional test cases are added to EIP-7883 & EIP-7823 following the devnet-0 release, including re-filled EIP-189 tests for Osaka. Both new EIPs: EIP-7918 & EIP-7825 are included. Some EIP-4844 tests are re-parameterized by block base fee per gas to include additional coverage for EIP-7918 regarding an active reserve price mechanism.

The next releaes will include additional edge cases for EIP-7918 & EIP-7825.

Included EIPs

The list below links the specific commit versions of the EIPs included in devnet-1 and in this release:

• EIP-7823: Set upper bounds for MODEXP
• EIP-7883: ModExp Gas Cost Increase
• EIP-7918: Blob base fee bounded by execution cost 🆕
• EIP-7825: Transaction Gas Limit Cap 🆕

[email protected]

This is an auxiliary release of the now DFI'd EOF EIP-7692 to include all the changes made to reflect the eof-devnet-1 spec and also the new pseudo-fork EOFv1. There are no further releases of EOF tests planned

What's Changed (Only EOF-relevant changes listed)

• new(tests): Add EOF validation tests for stack underflow by @chfast in #1331
• fix(tests): Change EOFCREATE new address and related gas calculations by @gumb0 in #1265
• fix(tests): EOF - Align EOFCREATE args with EXT*CALL by @pdobacz in #1274
• new(tests): EOF validation CALLF stack overflow by @hugo-dc in #1329
• fix(tests): assign ids to parametrized EOF code sections by @chfast in #1427
• fix(tests): EOF - Change kind_data from 0x04 to 0xff by @pdobacz in #1339
• fix(tests): EOF - Change container section size to 4 bytes by @gumb0 in #1403
• new(tests): EOF validation non constant stack by @hugo-dc in #1417
• new(tests): EIP-7873 TXCREATE execution tests - 1st batch by @pdobacz in #1413
• feat(fw,tests): change EOF to stack_height_increase by @chfast in #1393
• feat(fw): add max_stack_increase to EOF Section.Code by @chfast in #1446
• refactor(tests): change RJUMP validation tests to max_stack_increase by @chfast in #1448
• refactor(tests): Rename stack height constants by @chfast in #1447
• refactor(tests): change CALLF tests to max_stack_increase by @chfast in #1455
• new(tests): add EOF execution at maximum stack height by @chfast in #1443
• new(tests): EOF validation stack_range maximally broad by @hugo-dc in #1457
• fix(tests,fw): Remove EIP-7698 EOF creation txs by @pdobacz in #1451
• fix(tests): Create transaction sarting with 0xef is a valid tx, but should fail execution by @shemnon in #1480
• fix(tests): EIP-7620 - make CREATE/CREATE2 restrictions specific by @pdobacz in #1475
• new(tests): EIP-7873 expand legacy creation tx tests by @pdobacz in #1489
• new(tests): EIP-7620/7873 - ensure no account gets created on failure by @pdobacz in #1486
• new(tests): extend RJUMP* to self tests with variadic stack by @chfast in #1462
• fix(tests): Update legacy creation tx rules by @pdobacz in #1492
• new(tests): TXCREATE executes validation by @pdobacz in #1476
• chore(all): move eof from osaka to unscheduled by @spencer-tb in #1507

Full Changelog: https://github.com/ethereum/execution-spec-tests/compare/[email protected]@v3.0.0

[email protected]

First EEST release for Fusaka Devnet-0.

Execution Layer EIP List for fusaka-devnet-0

The list below links the specific commit versions of the EIPs included in devnet-0 and in this release:

• EIP-7823: Set upper bounds for MODEXP
• EIP-7883: ModExp Gas Cost Increase

[email protected]

This release introduces a suite of tests specifically designed for benchmarking and stress-testing zkVMs. These tests provide high coverage of opcodes and precompiles, focusing on worst-case and challenging scenarios for block proving, which can also help discover mispriced opcodes for zkVMs.

Highlight of tests added in this release:

• Arithmetic Opcodes: specifically crafted for efficient gas computational load.
• JUMP/JUMPDEST scenarios: tests various jump and potential JUMPDEST analysis triggers.
• Stateful opcode accesses: Covers warm, cold, and (non-)absent account and storage access scenarios.
• KECCAK opcode flooding
• Precompile Coverage: Includes all existing precompiles active in Pectra (further enhancements are still possible).
• Bytecode witness attacks: worst-case related to non-chunkified code.

All tests are generic to the provided gas limit (configurable via --block-gas-limit). This allows for easy generation of future test scenarios with arbitrary gas limits.

To run these tests effectively in your zkVM, we highly recommend using the zkevm-benchmark-workload tool (currently experimental). This tool streamlines the process by:

• Running tests in their standard fixture format.
• Automatically generating the necessary witness data.
• Executing blocks statelessly across multiple zkVM implementations.

This suite aims to provide robust tools for evaluating and hardening zkVM performance and security. It already contains a script to download and run the latest test release.

v4.5.0 (Hradčany)

🇨🇿 Summary

EESTs Hradčany is our final release for Prague following a successful Pectra!

💥 Breaking Change

EOF removed from Osaka

Following "Interop Testing Call 34" and the procedural EIPs PR the decision to remove EOF from Osaka was made.

To accommodate EOF testing for the interim within EEST, its tests have migrated to a new tests/unscheduled folder. This folder will now contain tests for features that are not yet CFI'd in any fork. When EOF is CFI'd for a fork in the future, all tests will be moved from unscheduled to the respective future fork folder.

A new fork EOFv1 has additionally been created to fill and consume EOF related fixtures. Client tests fillers such as evmone (and client consumers) will now need to use this fork name.

🛠️ Framework

• ✨ Add an empty account function for usage within fill and execute (#1482).

📋 Misc

• ✨ Engine API updates for Osaka, add get_blobs rpc method (#1510).
• ✨ The EIP Version checker has been moved from fill and execute to it's own command-line tool check_eip_versions that gets ran daily as a Github Action (#1537).
• 🔀 Add new tests/unscheduled folder, move EOF from Osaka to unscheduled, add EOFv1 fork name for EOF tests (#1507).
• ✨ CI features now contain an optional field to skip them from EEST full releases, zkevm and EOF features are now feature only (#1596).
• 🐞 Don't attempt to install solc via solc-select on ARM (official Linux ARM builds of solc are not available at the time of writing, cf ethereum/solidity#11351) and add a version sanity check (#1556).

🧪 Test Cases

• 🔀 Automatically apply the zkevm marker to all tests under ./tests/zkevm/ and ./tests/prague/eip2537_bls_12_381_precompiles/ via conftest configuration (#1534).
• ✨ Port calldataload and calldatasize tests (#1236).

New Contributors

• @kevaundray made their first contribution in #1514
• @ArunRawat404 made their first contribution in #1482
• @flcl42 made their first contribution in #1542

Full Changelog: v4.4.0...v4.5.0

[email protected]

What's Changed

• zkevm: add bytecode worst case by @jsign in #1456
• zkevm: keccak worst-case by @jsign in #1497
• zkevm: fix max contract size limitation in bytecode attack by @jsign in #1521

Full Changelog: https://github.com/ethereum/execution-spec-tests/compare/[email protected]@v0.0.2

[email protected]

What's Changed

• feat(ci): add zkevm as a release fixture feature by @marioevz in #1503
• chore(tests): add zkevm marker for bls g1 add tests by @kevaundray in #1514

New Contributors

• @kevaundray made their first contribution in #1514

Full Changelog: https://github.com/ethereum/execution-spec-tests/compare/[email protected]

v4.4.0 (Stromovka)

🇨🇿 Summary

EESTs Stromovka release adds re-filled static tests from ethereum/tests (state tests), improved Prague coverage including many additional cases for EIP-2537, and additional fixes for the exeption mapper now used within consume engine.

Please read further for more information!

💥 Breaking Change

fixtures_static

A new fixture tarball has been included in this release: fixtures_static.tar.gz.

This tarball contains all tests inside of ./tests/static, which at this point only contains all tests copied from GeneralStateTests in ethereum/tests@7dc757ec132e372b6178a016b91f4c639f366c02.

The tests have been filled using the new static test filler introduced in #1336, and enhanced in #1362 and #1439.

Users can expect that all tests currently living in ethereum/tests should eventually make its way into ./tests/static and can rely that these tests, filled for new forks even, will be included in fixtures_static.tar.gz.

🛠️ Framework

fill

• 🐞 Fix the reported fixture source URLs for the case of auto-generated tests (#1488).

consume

• 🐞 Fix the Hive commands used to reproduce test executions that are displayed in test descriptions in the Hive UI (#1494).
• 🐞 Fix consume direct fails for geth blockchain tests (#1502).

📋 Misc

🧪 Test Cases

• ✨ EIP-7702: Test that DELEGATECALL to a 7702 target works as intended (#1485).
• ✨ EIP-2573: Includes a BLS12 point generator, alongside additional coverage many of the precompiles (#1350), (#1505).
• ✨ Add all GeneralStateTests from ethereum/tests to execution-spec-tests located now at tests/static/state_tests (#1442).
• ✨ EIP-2929: Test that precompile addresses are cold/warm depending on the fork they are activated (#1495).

Full Changelog: v4.3.0...v4.4.0

Vltava (v4.3.0)

Summary

This release introduces more coverage for Prague EIPs, along with many new features described below.

💥 Breaking Change

EIP Changes

• EIP-6110: Spec parse_deposit_data
• EIP-7002: Add system call gas aspects and directive for handling failures
• EIP-7251: Add system call properties and directive for failure cases

Consume engine strict exception checking

consume engine now checks exceptions returned by the execution clients in their Engine API responses, specifically in the validationErrorfield of the engine_newPayloadVX method.

While not strictly a breaking change since tests will continue to run normally, failures are expected if a client modifies their exception messages.

This feature can be disabled by using --disable-strict-exception-matching for specific clients or forks.

🛠️ Framework

fill

• ✨ The static_filler plug-in now has support for static state tests (from GeneralStateTests) (#1362).
• ✨ Introduce pytest.mark.exception_test to mark tests that contain an invalid transaction or block (#1436).
• 🐞 Fix DeprecationWarning: Pickle, copy, and deepcopy support will be removed from itertools in Python 3.14. by avoiding use itertools object in the spec BaseTest pydantic model (#1414).

consume

• 🐞 Fix fixture tarball downloading with regular, non-Github release URLS and with numerical versions in regular release specs, e.g., [email protected] (#1437).
• ✨ consume engine now has strict exception mapping enabled by default (#1416).

Tools

• 🔀 generate_system_contract_deploy_test test generator has been updated to handle system contracts that are not allowed to be absent when the fork happens (#1394).
• ✨ Add generate_system_contract_error_test to generate tests on system contracts that invalidate a block in case of error (#1394).

Exceptions

• ✨ BlockException.SYSTEM_CONTRACT_EMPTY: Raised when a required system contract was not found in the state by the time it was due to execution with a system transaction call (#1394).
• ✨ BlockException.SYSTEM_CONTRACT_CALL_FAILED: Raised when a system contract call made by a system transaction fails (#1394).
• ✨ BlockException.INVALID_BLOCK_HASH: Raised when the calculated block hash does not match the expectation (Currently only during Engine API calls) (#1416).
• ✨ BlockException.INVALID_VERSIONED_HASHES: Raised when a discrepancy is found between versioned hashes in the payload and the ones found in the transactions (#1416).

🧪 Test Cases

• ✨ EIP-7702: Test precompile case in same transaction as delegation without extra gas in case of precompile code execution; parametrize all call opcodes in existing precompile test (#1431).
• ✨ EIP-7702: Add invalid nonce authorizations tests for the case of multiple signers when the sender's nonce gets increased (#1441).
• ✨ EIP-7702: Add a test that verifies that set code transactions are correctly rejected before Prague activation (#1463).
• ✨ EIP-7623: Additionally parametrize transaction validity tests with the to set to an EOA account (previously only contracts) (#1422).
• ✨ EIP-7251: Add EIP-7251 test cases for modified consolidations contract that allows more consolidations (#1465).
• ✨ EIP-6110: Add extra deposit request edge cases, sending eth to the deposit contract while sending a deposit request (#1467).
• ✨ EIP-6110: Add cases for deposit log layout and other topics (ERC-20) transfer (#1371).
• ✨ EIP-7251: Remove pytest skips for consolidation request cases (#1449).
• ✨ EIP-7002, EIP-7251: Add cases to verify behavior of contracts missing at fork (#1394).
• ✨ EIP-7002, EIP-7251: Add cases to verify behavior of system contract errors invalidating a block (#1394).
• 🔀 Remove EIP-7698: EIP has been removed and the tests related to it have also been removed, while preserving a subset of the tests to verify that functionality is removed in clients (#1451).

📋 Misc

• 🐞 Configure markdownlint to expect an indent of 4 with unordered lists (otherwise HTML documentation is rendered incorrectly, #1460).
• 🔀 Update eels_resolutions.json to point to temporary commit bb0eb750d643ced0ebf5dec732cdd23558d0b7f2, which is based on forks/prague branch, commit d9a7ee24db359aacecd636349b4f3ac95a4a6e71, with PRs ethereum/execution-specs#1182, ethereum/execution-specs#1183 and ethereum/execution-specs#1191 merged (#1394).

✨ New Contributors

• @GarmashAlex made their first contribution in #1370

Full Changelog: v4.2.0...v4.3.0

Vyšehrad (v4.2.0)

Summary

This release introduces more coverage for Prague EIPs, along with many new features described below.

💥 Breaking Change

Note: Although not a breaking change, consume users should delete the cache directory (typically located at ~/.cache/ethereum-execution-spec-tests) used to store downloaded fixture release tarballs. This release adds support for ethereum/tests and ethereum/legacytests fixture release downloads and the structure of the cache directory has been updated to accommodate this change.

To try this feature:

consume direct --input=https://github.com/ethereum/tests/releases/download/v17.0/fixtures_blockchain_tests.tgz

To determine the cache directory location, see the --cache-folder entry from the command:

consume cache --help

🛠️ Framework

consume

• ✨ Add support for ethereum/tests and ethereum/legacytests release tarball download via URL to the --input flag of consume commands (#1306).
• ✨ Add support for Nethermind's nethtest command to consume direct (#1250).
• ✨ Allow filtering of test cases by fork via pytest marks (e.g., -m "Cancun or Prague") (#1304, #1318).
• ✨ Allow filtering of test cases by fixture format via pytest marks (e.g., -m blockchain_test) (#1314).
• ✨ Add top-level entries forks and fixture_formats to the index file that list all the forks and fixture formats used in the indexed fixtures (#1318).
• ✨ Enable logging from consume commands (#1361).
• ✨ Propagate stdout and stderr (including logs) captured during test execution to the Hive test result (#1361).
• 🐞 Don't parametrize tests for unsupported fixture formats; improve consume test collection (#1315).
• 🐞 Fix the the hive command printed in test reports to reproduce tests in isolation by prefixing the --sim.limit flag value with id: (#1333).
• 🐞 Improve index generation of ethereum/tests fixtures: Allow generation at any directory level and include generatedTestHash in the index file for the fixture_hash (#1303).
• 🐞 Fix loading of ethereum/tests and ethereum/legacytests fixtures for the case of mixed 0x0 and 0x1 transaction types in multi-index (data, gas, value) state test fixtures (#1330).
• ✨ Add Osaka to the hive ruleset, includes a small ruleset refactor (#1355).

fill

• 🐞 Fix --fork/from/until for transition forks when using fill #1311.
• 🐞 Fix the node id for state tests marked by transition forks (#1313).
• ✨ Add static_filler plug-in which allows to fill static YAML and JSON tests (from ethereum/tests) by adding flag --fill-static-tests to uv run fill (#1336).

execute

• 🔀 Test IDs have changed to include the name of the test spec where the test came from (e.g. state_test, blockchain_test, etc) (#1367).
• ✨ Markers can now be used to execute only tests from a specific test spec type (e.g. -m state_test, -m blockchain_test, etc) (#1367).

📋 Misc

• 🔀 Bump the version of execution-specs used by the framework to the package ethereum-execution==1.17.0rc6.dev1; bump the version used for test fixture generation for forks < Prague to current execution-specs master, fa847a0 (#1310).
• 🐞 Init TransitionTool in GethTransitionTool (#1276).
• 🔀 Refactored RLP encoding of test objects to allow automatic generation of tests (#1359).
• ✨ Document how to manage execution-spec-tests package dependencies (#1388).

Packaging

• 🐞 Fix eest make test when ethereum-execution-spec-tests is installed as a package (#1342).
• 🔀 Pin setuptools and wheel in [build-system], bump trie>=3.1 and remove setuptools from package dependencies (#1345, #1351).

🧪 Test Cases

• ✨ Add additional test coverage for EIP-152 Blake2 precompiles (#1244). Refactor to add variables for spec constants and common fixture code. (#1395), (#1405).
• ✨ Add EIP-7702 incorrect-rlp-encoding tests (#1347).
• ✨ Add EIP-2935 tests for all call opcodes (#1379).
• ✨ Add more tests for EIP-7702: max-fee-per-gas verification, delegation-designation as initcode tests (#1372).

New Contributors

• @lightsing made their first contribution in #1276
• @sonhv0212 made their first contribution in #1349
• @hugo-dc made their first contribution in #1341
• @Olexandr88 made their first contribution in #1401

Full Changelog: v4.1.0...v4.2.0