engineer
181a51af95
tests/test_config_flow_real_ha.py skips itself at import time when homeassistant is absent, and a skipped file is indistinguishable from a passing one: the stub jobs end in "N passed, 1 skipped" and exit 0, and the drift sweep never reads it. That is how three real-HA-only facts stayed hidden while five gates were green. New real-ha-config-flow job installs the real stack on Home Assistant's interpreter floor (reusing HOMEASSISTANT_PYTHON_FLOOR from tests/ha_api_drift.py), runs the file with -rs, and then checks the pytest summary line itself: it must report passed tests and no skipped/failed/error line, because some pytest versions exit 0 for a file that skipped itself. The header comment now names both jobs that install Home Assistant.
220 lines
9.8 KiB
YAML
220 lines
9.8 KiB
YAML
# Quality gate: lint + unit tests + the HA API-drift sweep.
|
|
#
|
|
# The test suite is stdlib-only (parsers, intents, running-apps) and runs the
|
|
# vendored ha_adb_common helpers without installing Home Assistant — see
|
|
# tests/conftest.py. androidtvremote2 is installed so the real-library
|
|
# volume/TYPED-dict tests execute (tests/test_protocol_transport.py). Real
|
|
# firmware captures live in tests/fixtures
|
|
# (how to capture more: docs/RESEARCH_SHIELD.md §8).
|
|
#
|
|
# Home Assistant is installed in exactly two jobs: `ha-api-drift` (below) and
|
|
# `real-ha-config-flow`. Both exist because every other job validates our code
|
|
# against hand-written stubs, which stay green even when a call site no longer
|
|
# matches the real API. That is how three drifts reached master (t36 D1/D2/D3) —
|
|
# see tests/ha_api_drift.py.
|
|
|
|
name: Test
|
|
|
|
on:
|
|
push:
|
|
branches: [master]
|
|
pull_request:
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Install test tooling
|
|
run: |
|
|
set -euo pipefail
|
|
# Try the plain install first: it succeeds on images whose Python is
|
|
# not externally managed (and on pip < 23, which has no PEP 668
|
|
# enforcement at all). When the runner image DOES mark its Python as
|
|
# externally managed — Debian/Ubuntu based Gitea act images do — pip
|
|
# refuses with externally-managed-environment and we retry with the
|
|
# sanctioned override. That override is safe here specifically
|
|
# because the job runs in a disposable container, and because the
|
|
# flag only exists on pip >= 23.0.1, the same version that raises
|
|
# the refusal in the first place.
|
|
#
|
|
# Tools are pinned to the versions the suite was validated against so
|
|
# a newer ruff rule set cannot redden the gate on unrelated commits;
|
|
# androidtvremote2 tracks the manifest floor so CI exercises the same
|
|
# dependency users install.
|
|
PKGS="pytest~=9.1 pytest-asyncio~=1.4 ruff~=0.17.0 'androidtvremote2>=0.2.0'"
|
|
if ! eval python3 -m pip install --quiet $PKGS; then
|
|
echo "system Python is externally managed; retrying with --break-system-packages"
|
|
eval python3 -m pip install --quiet --break-system-packages $PKGS
|
|
fi
|
|
|
|
- name: Lint
|
|
run: ruff check custom_components tests
|
|
|
|
- name: Unit tests
|
|
run: python3 -m pytest tests -q
|
|
|
|
# Resolve every Home-Assistant API use in our production modules against a
|
|
# real install: imports, keyword arguments and await/sync mismatches. The
|
|
# sweep is repo-owned (tests/ha_api_drift.py) and skips cleanly when
|
|
# Home Assistant is absent, so it is inert in the job above and live here.
|
|
ha-api-drift:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Provision the interpreter Home Assistant needs
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
# Home Assistant 2026.10.1 declares Requires-Python >= 3.14.2 (PyPI
|
|
# metadata; tests/ha_api_drift.py holds our copy of that number and the
|
|
# drift suite pins it against the installed distribution), so the
|
|
# runner's own python3 cannot install the version this gate must run
|
|
# against. This resolves the newest 3.14.x patch; the guard below
|
|
# refuses anything still short of the floor rather than letting the
|
|
# install fail halfway through.
|
|
python-version: "3.14"
|
|
|
|
- name: Check the interpreter satisfies Home Assistant's floor
|
|
run: |
|
|
set -euo pipefail
|
|
# The floor comes from tests/ha_api_drift.py — one source of truth for
|
|
# this guard and the metadata pin in tests/test_ha_api_drift.py.
|
|
python3 - <<'PY'
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
sys.path.insert(0, str(Path("tests").resolve()))
|
|
from ha_api_drift import HOMEASSISTANT_PYTHON_FLOOR as FLOOR # noqa: E402
|
|
|
|
needed = ".".join(str(part) for part in FLOOR)
|
|
if sys.version_info < FLOOR:
|
|
print(
|
|
f"Home Assistant needs Python >= {needed}, but this job runs "
|
|
f"{sys.version.split()[0]}."
|
|
)
|
|
print(f"Fix: raise actions/setup-python in this job to a >= {needed} build.")
|
|
raise SystemExit(1)
|
|
print(f"interpreter ok: Python {sys.version.split()[0]} satisfies >= {needed}")
|
|
PY
|
|
|
|
- name: Install Home Assistant + test tooling
|
|
run: |
|
|
set -euo pipefail
|
|
# Same PEP 668 dance as the test job (see the note there). The
|
|
# version floor tracks what we validate against; a resolver failure
|
|
# here is a real signal (Python too old), not something to swallow.
|
|
PKGS="pytest~=9.1 pytest-asyncio~=1.4 'homeassistant>=2026.10'"
|
|
if ! eval python3 -m pip install --quiet $PKGS; then
|
|
echo "system Python is externally managed; retrying with --break-system-packages"
|
|
eval python3 -m pip install --quiet --break-system-packages $PKGS
|
|
fi
|
|
|
|
- name: HA API-drift sweep
|
|
run: python3 -m pytest tests/test_ha_api_drift.py -q
|
|
|
|
# The real-HA config-flow tests (tests/test_config_flow_real_ha.py) are the
|
|
# ONLY place the config flow meets the real homeassistant.data_entry_flow
|
|
# machinery — and that file SKIPS ITSELF at import time when homeassistant is
|
|
# absent (pytest.importorskip at module scope). A skipped file and a passing
|
|
# one are indistinguishable in the jobs above: `python3 -m pytest tests -q`
|
|
# ends in "N passed, 1 skipped" and still exits 0, and the drift sweep never
|
|
# touches this file. That is exactly how three real-HA-only facts stayed
|
|
# hidden while five gates were green:
|
|
#
|
|
# * a menu row id must equal its async_step_<id> method name (otherwise the
|
|
# real FlowManager raises when the row is dispatched),
|
|
# * the rendered step_id must equal the row id, and
|
|
# * async_show_menu takes no errors kwarg.
|
|
#
|
|
# So this job does two things the others cannot: it installs the real stack on
|
|
# Home Assistant's interpreter floor, and it FAILS LOUDLY if the tests are
|
|
# skipped instead of run. Do not "simplify" the guard at the bottom away — a
|
|
# test file that skips silently is indistinguishable from one that passes.
|
|
real-ha-config-flow:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
|
|
- name: Provision the interpreter Home Assistant needs
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
# Same floor as the ha-api-drift job: Home Assistant 2026.10.1
|
|
# declares Requires-Python >= 3.14.2 (tests/ha_api_drift.py holds our
|
|
# copy of that number). This resolves the newest 3.14.x patch.
|
|
python-version: "3.14"
|
|
|
|
- name: Ensure uv is available
|
|
run: |
|
|
set -euo pipefail
|
|
# uvx runs the exact command this gate was validated with locally;
|
|
# runner images are not guaranteed to ship it.
|
|
if ! command -v uvx >/dev/null 2>&1; then
|
|
python3 -m pip install --quiet uv \
|
|
|| python3 -m pip install --quiet --break-system-packages uv
|
|
fi
|
|
uvx --version
|
|
|
|
- name: Check the interpreter satisfies Home Assistant's floor
|
|
run: |
|
|
set -euo pipefail
|
|
# The floor comes from tests/ha_api_drift.py — one source of truth for
|
|
# this guard and the metadata pin in tests/test_ha_api_drift.py. uvx
|
|
# resolves the same interpreter the test step will use, so a too-old
|
|
# interpreter fails here rather than halfway through the install.
|
|
uvx --python 3.14 python - <<'PY'
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
sys.path.insert(0, str(Path("tests").resolve()))
|
|
from ha_api_drift import HOMEASSISTANT_PYTHON_FLOOR as FLOOR # noqa: E402
|
|
|
|
needed = ".".join(str(part) for part in FLOOR)
|
|
if sys.version_info < FLOOR:
|
|
print(
|
|
f"Home Assistant needs Python >= {needed}, but this job runs "
|
|
f"{sys.version.split()[0]}."
|
|
)
|
|
raise SystemExit(1)
|
|
print(f"interpreter ok: Python {sys.version.split()[0]} satisfies >= {needed}")
|
|
PY
|
|
|
|
- name: Real-HA config-flow tests
|
|
run: |
|
|
set -euo pipefail
|
|
LOG=/tmp/real-ha-config-flow.log
|
|
# Verbatim the command this gate is validated with locally; -rs is what
|
|
# makes a silent skip visible in the log.
|
|
set +e
|
|
uvx --python 3.14 --with 'homeassistant>=2026.10' --with adb-shell \
|
|
--with pytest-asyncio \
|
|
pytest tests/test_config_flow_real_ha.py -q -rs 2>&1 | tee "$LOG"
|
|
rc=${PIPESTATUS[0]}
|
|
set -e
|
|
if [ "$rc" -ne 0 ]; then
|
|
echo "::error::the real-HA config-flow tests exited $rc"
|
|
exit "$rc"
|
|
fi
|
|
|
|
# THE GUARD. Some pytest versions exit 0 for a file that skipped
|
|
# itself, so the exit code above is not enough. The last line of a -q
|
|
# run is the counts line ("26 passed in 1.6s" / "1 skipped in
|
|
# 0.01s"); it must report PASSED tests and nothing else.
|
|
summary=$(grep -v '^[[:space:]]*$' "$LOG" | tail -n 1)
|
|
echo "pytest summary: $summary"
|
|
case "$summary" in
|
|
[0-9]*" passed"*) ;;
|
|
*)
|
|
echo "::error::the real-HA config-flow tests did not run (summary: $summary)"
|
|
echo "The file skips itself when homeassistant/probatio is missing — a skip here is a FAILURE, not a pass."
|
|
exit 1
|
|
;;
|
|
esac
|
|
case "$summary" in
|
|
*skipped*|*failed*|*error*|*"no tests ran"*)
|
|
echo "::error::the real-HA config-flow run was not a clean pass (summary: $summary)"
|
|
exit 1
|
|
;;
|
|
esac
|