test(harness): make the stub scoping independent of file order (t44)

A subset of test files in an order the suite was not written for failed 3
test_init.py::TestImportStep tests with ModuleNotFoundError: voluptuous, while the
full suite stayed green. Two order-only mechanisms, both now fixed at the source:

1. entity_stublib.install() guarded on ONE sentinel
   (`homeassistant.components.media_player`). A file whose teardown popped
   `voluptuous` but left the sentinel turned the next file's install() into a no-op,
   so importing config_flow failed. The guard now checks every face a file scopes on
   (`_STUB_FACES`, derived from ENTITIES_STUB_NAMES so the two cannot drift) and
   rebuilds the set when anything is missing. install() also registers the
   `homeassistant.helpers.selector` face itself, which the per-file installers had
   been adding separately — without it the derived list would never be complete.

2. Teardowns restored only the names their file had listed, so a module the file
   loaded (`nsh.diagnostics`, `nsh.media_player`) survived into the next file still
   holding stub objects — a stub exception class, the redact helper — that had since
   been replaced; comparisons against the new objects then failed (5 tests in
   test_command_routing on an unlucky order). Every file now ends through
   entity_stublib.snapshot_modules()/restore_modules(), which drops EVERYTHING the
   file created under the harness namespaces (homeassistant/voluptuous/nsh) and puts
   back everything it replaced. Hand-written lists were the bug, so there is none to
   keep in sync.

The t7 no-leak discipline is unchanged: a file's stubs still never reach another
file, and a file's loaded modules still do not survive it. A session-scoped baseline
install was tried first and rejected: files whose own installer guards on "the stubs
are already there" skipped installing their richer faces, which produced exactly the
inconsistent state a baseline is meant to prevent (10 test_standalone_setup failures).

tests/test_harness_scoping.py: the sentinel poisoning, the wholesale-rebuild
invariant, a teardown that removes what the file loaded, the stale-object case
(diagnostics vs the redact helper), and — because order is the thing under test —
two subprocess runs of the reported orderings.

Verified: the reported order (test_command_routing, test_protocol_transport,
test_standalone_setup, test_init) 152 passed / 1 skipped; the minimal pair both ways;
the previously failing 6-test order now 167 passed; every test file in REVERSE
alphabetical order 369 passed / 2 skipped; two scrambled thirds green; full suite
369 / 2 and 370 / 1 with androidtvremote2; ruff clean.
This commit is contained in:
James Mackay committed 2026-10-11 06:41:56 +01:00
1 parent fbc72dfd9e
commit 2e1d162ef4
9 files changed
+233 -63

No files matched your search

+34 -20
View File
@@ -178,26 +178,11 @@ class RemoteEntity:
#: behind while another name it popped — e.g. `voluptuous` — is gone. A sentinel
#: guard then made install() a no-op in the next file, and whichever module needed
#: the missing face failed with ModuleNotFoundError depending on file order (t44).
_STUB_FACES = (
"voluptuous",
"homeassistant",
"homeassistant.config_entries",
"homeassistant.core",
"homeassistant.const",
"homeassistant.exceptions",
"homeassistant.components",
"homeassistant.components.media_player",
"homeassistant.components.remote",
"homeassistant.components.button",
"homeassistant.components.sensor",
"homeassistant.helpers",
"homeassistant.helpers.event",
"homeassistant.helpers.entity",
"homeassistant.helpers.entity_platform",
"homeassistant.helpers.device_registry",
"homeassistant.helpers.entity_registry",
"homeassistant.helpers.redact",
"homeassistant.helpers.selector",
#: The faces install() provides, derived from the list every test file scopes on.
#: The ``nsh.*`` entries there are the integration modules the harness loads, not
#: stub faces, so they are filtered out here.
_STUB_FACES = tuple(
name for name in ENTITIES_STUB_NAMES if not name.startswith("nsh.")
)
@@ -508,6 +493,35 @@ def install() -> None:
sys.modules["homeassistant.helpers"] = helpers
#: Namespaces the stub harness owns. Anything under these prefixes that a test
#: file creates is harness state, so a file's teardown must remove it: a module
#: left behind keeps the stub objects it captured (an exception class, the redact
#: helper the diagnostics module holds) after the next file rebuilt them, which is
#: how a file-order-dependent failure looked (t44).
_TRACKED_PREFIXES = ("homeassistant", "voluptuous", "nsh")
def _tracked(name: str) -> bool:
return name.split(".", 1)[0] in _TRACKED_PREFIXES
def snapshot_modules() -> dict:
"""Every stub/module name live right now — for a leak-free teardown."""
return {name: module for name, module in sys.modules.items() if _tracked(name)}
def restore_modules(snapshot: dict) -> None:
"""Undo everything the current test file did to those namespaces.
Two halves, and the first is what a hand-written list of names kept getting
wrong: drop every tracked name the file created (so the next file loads it
fresh, against ITS stubs), then put back every object it replaced.
"""
for name in [name for name in sys.modules if _tracked(name) and name not in snapshot]:
del sys.modules[name]
sys.modules.update(snapshot)
def load_entity_modules() -> dict:
"""Load the integration's HA-free-ish entity modules under nsh."""
install()