SteelmemoSign in

Steelmemo · Sample memo

Regression in 5.2.3: pytest tries to collect random __init__.py files

Oct 20, 2026 · 4 models from 4 labs · 13 file reads · 6 min 3 s

Verdict

Delete the single line self._mount_obj_if_needed() at pytest/src/_pytest/python.py:642, the first statement of Package.collect, and leave the + ["__init__.py"] allowance at line 181 alone. That line imports every package's __init__.py during collection, before the code checks whether the file matches python_files. So a package with no tests, like your Windows-only one, still gets imported.

First step this week: Run git log -L 641,648:src/_pytest/python.py to confirm when and why line 642 was added.

Confidence

Medium. Three of four labs agree and back it with the source code, but Gemini held a different fix, nobody ran the reproducer or the test suite, and nobody checked against the git history whether 5.2.3 actually introduced this line.

Agreement

2 of 4 agree; 1 partly agrees (Grok); 1 disagrees (Gemini).

What changed under review

  • Claude first said to remove + ["__init__.py"] at python.py:181. In its later review it moved to deleting line 642, because main.py:511-519 and 534-542 show that package discovery runs __init__.py through that allowance, and creating a Package (python.py:560-568) imports nothing.

Why

  1. Package.collect calls self._mount_obj_if_needed() as its very first statement. That call runs _getobj(), which imports the package's __init__.py, so every package found during the walk gets imported, even one with no tests. (pytest/src/_pytest/python.py, line 642 and lines 261-268)
  2. The check of __init__.py against python_files only happens after that import. The import comes before the filter, and that is the regression. (pytest/src/_pytest/python.py, lines 645-647)
  3. Line 181 has to stay. Session._collect builds and registers Package nodes by passing each __init__.py through _collectfile. Removing the allowance would hide the error but break package discovery, package setup_module/teardown_module and package-scoped fixtures. (pytest/src/_pytest/main.py, lines 511-519 and 534-542)

Where they disagreed

  • Which line to change. Claude (after revising), GPT and Grok: delete python.py:642. Gemini: remove + ["__init__.py"] at python.py:181. Not settled. To settle it: Apply the line-181 change and run a test where a package with a real test also has a package-level setup_module and a package-scoped fixture. Per Claude and Grok, both stop running when line 181 is removed.
  • Delete line 642 outright, or make the import conditional. Claude, GPT and Grok all accept deleting it, though Grok flags the cost. Grok, as challenger, argues a conditional or later import would fix the bug without losing collection-time pytestmark handling (python.py:267-268). Not settled. To settle it: Run git log -L 641,648:src/_pytest/python.py to see why line 642 was added, and check whether your code relies on a skip or mark in a package __init__.py.

The case against

Grok argued that what changed between 5.2.2 and 5.2.3 is assumed, not shown: this tree is only the broken commit, so the regression might also involve line 181 or main.py. Deleting the import is a blunter fix than the bug needs. Importing only when the package actually has matching tests would fix the repro and keep collection-time marks, which may be the reason the call exists. If the Windows-only tree contains tests or a conftest.py, deleting 642 may not fix it, because normal imports and setup still touch __init__.py. Test (a) also passes under the rejected line-181 fix, and nothing guards mark handling.

Not verified

  • Claude recalled from memory that line 642 was added in 5.2.3 for skip-in-__init__.py (#5830) and reverted in 5.2.4. Nothing in the attached tree confirms this.
  • The claim that 5.2.2 never ran this import is an inference. Nobody compared against 5.2.2.
  • No lab ran the reproducer or the test suite. The fix is reasoned from reading the source.
  • Nobody checked whether your real Windows-only package contains tests or a conftest.py. If it does, it may still be imported after the fix.

What would change the answer

  • Git history shows line 642 was not new in 5.2.3, or that 5.2.3 also changed python.py:181 or the package walk in main.py:511-542.
  • Your project relies on a skip or pytestmark in a package __init__.py to deselect child tests. If so, use a narrower conditional import instead of deleting the line.
  • Your Windows-only package contains test files or a conftest.py, which means it would still be imported after deleting 642.
  • Removing line 181 still lets package setup_module and package-scoped fixtures run. That would undercut the main reason for rejecting Gemini's fix.

Next steps

  1. This week: run git log -L 641,648:src/_pytest/python.py to confirm when and why line 642 was added.
  2. Delete self._mount_obj_if_needed() at pytest/src/_pytest/python.py:642. Leave this_path = self.fspath.dirpath() and everything after it unchanged, and don't touch line 181.
  3. Add regression test (a): foobar/__init__.py containing assert False, with a passing test_foo.py beside it (not inside it). Default collection must report 1 passed with no collection error.
  4. Add regression test (b): a package with a real test plus a package-level setup_module and a package-scoped fixture. Assert that both still run.
  5. Following Grok's challenge, add a case with a skip or mark in a package __init__.py so you can see the behavior you are giving up.
  6. Run the full existing suite, paying attention to package, fixture and mark-collection tests, before merging.
Each model's position (4)

Claude (Anthropic): First proposed removing + ["__init__.py"] at python.py:181. In its later review it dropped that idea: Session._collect in main.py needs the allowance to build Package nodes for packages it finds while walking directories. It now says to delete line 642 only. It also says that losing skip-in-__init__.py handling is an acceptable cost.

GPT (OpenAI): Delete line 642 only and keep the rest of collect() and line 181 as they are. Creating a Package node imports nothing. The import comes from the mount at 642, which runs before the python_files check at 645-647. Package setup still reads self.obj when it needs to. GPT ran no tests and did not check the release history.

Gemini (Google): Says the root cause is + ["__init__.py"] in pytest_collect_file (python.py:176-184) and recommends removing it so collection follows python_files again. Gemini kept this view in its second review and did not respond to the main.py evidence that package discovery depends on that allowance.

Grok (xAI): Agrees line 642 is where the import happens during collection, and that removing line 181 would break package discovery, package setup_module and package-scoped fixtures. Warns that deleting 642 also stops marks and skips in a package __init__.py from applying while tests are collected. As challenger, it argued you should confirm the 5.2.2 diff first, and consider moving the import later or making it conditional instead of deleting it.

Written by AI models. They can be wrong: check anything important before you act on it. Not legal, financial or professional advice.

Ask your own question