Skip to content

Disable assertion rewriting external modules #13421

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
231cb63
13403: Disable assertion rewriting for external modules
Tusenka May 11, 2025
bdc096c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 13, 2025
0701fcb
13403: Disable assertion rewriting for external modules - move root p…
Tusenka May 13, 2025
0d0551e
13403: Disable assertion rewriting for external modules
Tusenka May 11, 2025
3ec4da8
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 15, 2025
f03f24e
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 16, 2025
b6db388
13403: Disable assertion rewriting for external modules - refactor
Tusenka May 16, 2025
1346500
13403: Disable assertion rewriting for external modules - add tests
Tusenka May 18, 2025
023dfdb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 16, 2025
1fee11d
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 19, 2025
ba41963
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 22, 2025
253497e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 22, 2025
67e7a69
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 16, 2025
ee836ad
13403: Disable assertion rewriting for external modules - add test fo…
Tusenka May 24, 2025
e7a98bb
13403: Disable assertion rewriting for external modules - eliminate o…
Tusenka May 25, 2025
c947f9a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 27, 2025
abd69ac
13403: Disable assertion rewriting for external modules - fix ruff
Tusenka Jun 1, 2025
fd59d87
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 1, 2025
3da821d
Update testing/test_assertrewrite.py
Tusenka Jun 1, 2025
48cfbc3
13403: Disable assertion rewriting for external modules - add tests
Tusenka Jun 3, 2025
eceae2e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 12, 2025
8c95bcd
13403: Disable assertion rewriting for external modules - add tests
Tusenka Jun 12, 2025
1d5a612
13403: Disable assertion rewriting for external modules - add tests
Tusenka Jun 12, 2025
c1f3772
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/13403.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Disable assertion for modules outside current working dir(cwd)
6 changes: 6 additions & 0 deletions src/_pytest/assertion/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,14 @@ class AssertionState:
def __init__(self, config: Config, mode) -> None:
self.mode = mode
self.trace = config.trace.root.get("assertion")
self.config = config
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's perhaps just pass in the invocation details

Id like to avoid full config in more objects

self.hook: rewrite.AssertionRewritingHook | None = None

@property
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this must mot use getwd instead use the invoction params

Copy link
Author

@Tusenka Tusenka May 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then we cannot change it on runtime as far as invocation param for pytester changes rootpath

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for the type - this code shoud use the rootdir or the invocationdir from the invocation params of the config
see https://docs.pytest.org/en/stable/reference/reference.html#pytest.Config.invocation_params as well as the config rootdir

Copy link
Author

@Tusenka Tusenka May 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My error, I mean as far as Invocation Param is frozen - it can't be changed on runtime. Pytester starts after the config has been loaded. So to change the rootpath for pytester I need to rewrite the rootpath in someway. I could mock all invocation params or I could use getcwd alongside with config rootdir for the testing purpose.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The complete object can be replaced with a changed one

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. would be done)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But is config.invocation_params.dir the correct thing to use? If I execute the tests using pytest src/tests/foo.py, I expect pytest to continue to rewrite the same files as if I execute just pytest.

I feel we should use config.rootpath here? What do you think @RonnyPfannschmidt ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed - the rootpath - else we miss imports to rewrite

def invocation_path(self):
"""Get current root path (current working dir)"""
return str(self.config.invocation_params.dir)


def install_importhook(config: Config) -> rewrite.AssertionRewritingHook:
"""Try to install the rewrite hook, raise SystemError if it fails."""
Expand Down
4 changes: 3 additions & 1 deletion src/_pytest/assertion/rewrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,9 @@ def _should_rewrite(self, name: str, fn: str, state: AssertionState) -> bool:
# rewritten if they match the naming convention for test files
fn_path = PurePath(fn)
for pat in self.fnpats:
if fnmatch_ex(pat, fn_path):
if fnmatch_ex(pat, fn_path) and fn_path.is_relative_to(
state.invocation_path
):
state.trace(f"matched test file {fn!r}")
return True

Expand Down
8 changes: 8 additions & 0 deletions src/_pytest/pytester.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
from collections.abc import Iterable
from collections.abc import Sequence
import contextlib
import dataclasses
from fnmatch import fnmatch
import gc
import importlib
Expand Down Expand Up @@ -749,6 +750,13 @@ def chdir(self) -> None:
This is done automatically upon instantiation.
"""
self._monkeypatch.chdir(self.path)
self._monkeypatch.setattr(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do i gather correct that this bit is to alter the assertion rewtiter state

we should alter only that state, not the full pytest config of the surrounding pytest - this looks like a assertion rewriter shortcoming that needs a followup after this

but we certainly need to alter the assertion hook in sys metapath instead of a pytest config

self._request.config,
"invocation_params",
dataclasses.replace(
self._request.config.invocation_params, dir=Path(self._path)
),
)

def _makefile(
self,
Expand Down
2 changes: 2 additions & 0 deletions testing/example_scripts/rewrite/pytest.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[pytest]
python_files = *.py
11 changes: 11 additions & 0 deletions testing/example_scripts/rewrite/some_plugin/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from __future__ import annotations

import pytest


@pytest.fixture
def special_asserter():
def special_assert(x, y):
assert {"x": x} == {"x": y}

return special_assert
Empty file.
9 changes: 9 additions & 0 deletions testing/example_scripts/rewrite/src/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""Stub file for testing"""

from __future__ import annotations


def func(x: int, y: int) -> int:
"""Stub function"""
assert (x) > 0
return 0 if x == y else 1 if x > y else -1
Empty file.
16 changes: 16 additions & 0 deletions testing/example_scripts/rewrite/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from __future__ import annotations

from _pytest.fixtures import fixture


pytest_plugins = ["pytester", "some_plugin"]


@fixture
def b():
return 1


@fixture
def a():
return 2
13 changes: 13 additions & 0 deletions testing/example_scripts/rewrite/tests/test_main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from __future__ import annotations

from typing import Callable

from testing.example_scripts.rewrite.src.main import func


def test_plugin(a: int, b: int, special_asserter: Callable[[int, int], bool]):
special_asserter(a, b)


def test_func(a: int, b: int, special_asserter: Callable[[int, int], bool]):
assert {"res": func(a, b)} == {"res": 0}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
def some_check(a: int):
assert abs(a)<2
return a in set(0, 1, -1)
61 changes: 61 additions & 0 deletions testing/test_assertrewrite.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import inspect
import marshal
import os
from os import mkdir
from pathlib import Path
import py_compile
import re
Expand All @@ -35,6 +36,7 @@
from _pytest.assertion.rewrite import rewrite_asserts
from _pytest.config import Config
from _pytest.config import ExitCode
from _pytest.monkeypatch import MonkeyPatch
from _pytest.pathlib import make_numbered_dir
from _pytest.pytester import Pytester
import pytest
Expand Down Expand Up @@ -370,6 +372,7 @@ def test_rewrites_plugin_as_a_package(self, pytester: Pytester) -> None:
pytester.makeconftest('pytest_plugins = ["plugin"]')
pytester.makepyfile("def test(special_asserter): special_asserter(1, 2)\n")
result = pytester.runpytest()

result.stdout.fnmatch_lines(["*assert 1 == 2*"])

def test_honors_pep_235(self, pytester: Pytester, monkeypatch) -> None:
Expand Down Expand Up @@ -1294,6 +1297,34 @@ def test_meta_path():
)
assert pytester.runpytest().ret == 0

def test_invocation_dir(self, pytester: Pytester, monkeypatch: MonkeyPatch) -> None:
"""Test get invocation param afrom AssertionState"""
from _pytest.assertion import AssertionState

config = pytester.parseconfig()
state = AssertionState(config, "rewrite")
assert state.invocation_path == str(config.invocation_params.dir)
new_rootpath = str(pytester.path / "test")
if not os.path.exists(new_rootpath):
os.mkdir(new_rootpath)
monkeypatch.setattr(
config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=Path(new_rootpath),
),
)
state = AssertionState(config, "rewrite")
assert state.invocation_path == new_rootpath

@pytest.mark.skipif(
sys.platform.startswith("win32"), reason="cannot remove cwd on Windows"
)
@pytest.mark.skipif(
sys.platform.startswith("sunos5"), reason="cannot remove cwd on Solaris"
)
Comment on lines +1322 to +1327
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these skipif marks still necessary?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is sunos still a supported python platform

As far as I'm aware sun is down

def test_write_pyc(self, pytester: Pytester, tmp_path) -> None:
from _pytest.assertion import AssertionState
from _pytest.assertion.rewrite import _write_pyc
Expand Down Expand Up @@ -1971,6 +2002,36 @@ def test_simple_failure():
assert hook.find_spec("file") is not None
assert self.find_spec_calls == ["file"]

def test_assert_rewrites_only_rootpath(
self, pytester: Pytester, hook: AssertionRewritingHook, monkeypatch
) -> None:
"""Do not rewrite assertions in tests outside `AssertState.rootpath` (#13403)."""
pytester.makepyfile(
Copy link
Member

@nicoddemus nicoddemus Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to avoid mocking here, instead using a real scenario to ensure Python files outside of the rootpath do not have their assertions rewritten.

Please create a simple but real-world project using pytester, something like:

myproject/
   /venv
     /lib
        /external_lib.py
     pyvenv.cfg  

   /src
      main.py
   /tests
      conftest.py
      test_main.py
   pytest.ini
outside.py

pytest.ini should have these contents:

[pytest]
python_files = *.py

Then execute pytest.runpytest(), which will execute from the root of the above tree.

If I understand the objective of the issue correctly:

  • myproject/venv/lib/external_lib.py: should not have assertions rewritten -- the file is inside a virtual environment and is not part of a pytest plugin. The pyvenv.cfg file is how pytest detects virtual environments.
  • myproject/src/main.py: rewritten -- inside rootpath and matches python_files.
  • myproject/tests/conftest.py: rewritten -- conftest.py files are always rewritten.
  • myproject/tests/test_main.py: rewritten -- test files are always rewritten.
  • outside.py: should not have assertions rewritten -- outside the rootpath.

I think the scenario above should cover what needs to be tested.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to include an external plugin here too, to test the same scenario as in test_assert_rewrite_correct_for_plugins.

Copy link
Author

@Tusenka Tusenka Jun 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to create the structure along with a some conftest plugin. Even witout changes it doen't rewrite conftest plugins assertions
image

Example:
https://github.com/Tusenka/pytest/tree/disable_assertion_rewriting_external_modules_draft/testing/example_scripts/rewrite
image

However it sees only top-level plugins:
['anyio', '_hypothesis_ftz_detector', '_hypothesis_globals', '_hypothesis_pytestplugin', 'hypothesis', 'pytest_twisted', 'typeguard']
Logs for rewriting

Does it an expected behavior? How does pytest provide plugins which are needed to be rewritten at the import stage? May be I do something wrong

**{
"file.py": """\
def test_simple_failure():
assert 1 + 1 == 3
"""
}
)
with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("file") is not None

rootpath = f"{os.getcwd()}/tests"
if not os.path.exists(rootpath):
mkdir(rootpath)
monkeypatch.setattr(
pytester._request.config,
"invocation_params",
Config.InvocationParams(
args=(),
plugins=(),
dir=Path(rootpath),
),
)
with mock.patch.object(hook, "fnpats", ["*.py"]):
assert hook.find_spec("file") is None

@pytest.mark.skipif(
sys.platform.startswith("win32"), reason="cannot remove cwd on Windows"
)
Expand Down