The user's remote displayed but offered no app picker, and the protocol
transport was never actually switched on. Both halves are fixed here.
REMOTE PARITY. The entity declares RemoteEntityFeature.ACTIVITY — exactly
what core's androidtv_remote declares — and publishes activity_list, which
is what makes the frontend render an activity picker at all. The list is
the app directory (the user's apps map, else the curated table as a floor)
in map order, then the apps ADB discovery found and something can name, in
the one name chain (user map > device label > curated > package id).
Nothing nameless is listed as a bare package id, and current_activity is
None until something is known: no activity is invented. remote.turn_on
honours activity= by waking the device when it reports itself off and then
launching through the existing launch path; a listed name resolves
case-insensitively, a package id or a detail link passes through, and
anything else raises unknown_activity naming the value instead of silently
doing nothing. An empty/absent activity keeps the previous behaviour
exactly, async_turn_off is untouched, and a configured command script still
owns the sequence.
WHY THE TRANSPORT WAS DEAD, MEASURED. androidtvremote2's keep_reconnecting
only starts _async_reconnect, whose whole body is
`while self._remote_message_protocol:` — false on the first check with
nothing connected, so the task returned immediately: no connect attempt, no
retry, and the auth callback was never reachable. HA core connects FIRST and
only then keeps it alive. Our _start_protocol called keep_reconnecting after
generating a certificate and never called async_connect anywhere, so every
send_key_command/send_launch_app_command raised ConnectionClosed internally,
the helpers swallowed it into `return False` plus a debug line, and no push
(is_available/is_on/current_app/volume_info) ever arrived.
The fix is a background connect loop: async_connect under a bounded timeout,
network failures retried with the ADB watchdog's jitter and backoff,
InvalidAuth routed into the EXISTING reauth callback (entry.async_start_reauth),
and keep_reconnecting only after a session is up. Setup never blocks on a
transport. A send that cannot run is no longer silent: it is counted, its
error recorded verbatim, and logged at WARNING with the transport state, and
the remote publishes protocol_state / protocol_connected /
protocol_ever_authenticated / protocol_last_error / protocol_failures /
protocol_certificate plus sticky per-push *_seen flags — so "sent" and
"never switched on" can never look alike again.
CASE (b) IS FIXED, NOT DOCUMENTED: switching the transport on in the options
flow for an entry that never paired now routes into the real
async_step_protocol_pair, and "continue without pairing" persists the
transport OFF. One marker carries the pairing (CONF_PROTOCOL_PAIRED),
written by the setup pairing step, the reauth finisher and the reconfigure
finisher, so no entry can sit enabled-but-unpaired. The certificate from the
first pairing is reused by design, so a paired entry is asked for no new PIN
— which is also why the marker, not certificate material, is the source of
truth.
ADB STAYS THE PRIMARY STATE SOURCE. Arming the protocol introduces pushes
that were never live before, so the reading now names its tier:
manager.foreground_source drives the media player's app_source and the
remote's activity_source, and the new tests prove a full protocol push set
leaves the media state, the now-playing reading and every ADB store field
untouched, while an ADB-only entry still serves media player, sensors, apps,
scripts and receiver with only the protocol surface degraded.
Copy: unknown_activity, activity_launch_failed and protocol_unavailable in
BOTH strings.json and translations/en.json, pinned by a test; the README
documents the picker, activity resolution, how to read the transport-state
attributes, the script bypass (a script-routed press is never evidence of
pairing), the connect-then-keep-alive lifecycle and cert reuse versus the
other integration's own pairing.
Evidence: 6 new real-HA FlowManager tests in tests/test_config_flow_real_ha.py
(unpaired -> pairs, PIN -> marker, skip -> off, paired -> not asked again, and
the setup flow's marker on success and on skip), a real-HA entity contract
test in tests/test_ha_api_drift.py (exact supported_features equality with
core, activity_list/current_activity reaching the state machine, live _attr_
updates) and new suites in tests/test_protocol_transport.py (connect-first,
keep_reconnecting alone is a no-op, retry, auth rejection, honest send
failures, ADB-primary precedence) and tests/test_command_routing.py (the
activity contract, turn_on(activity=), script ownership, the transport
attribute states). No device and no browser exist here: device behaviour is
inferred from the library and core source, and every UI-facing claim is
proven at the entity/registry/metadata level under real Home Assistant.
Verify: suites 719/4 and 720/3; real-HA glob (tests/test_adb_command_real_ha.py
+ tests/test_config_flow_real_ha.py) 56 passed, 0 skipped; ruff clean; drift
exit 0 (25 modules, 76 imports, 206 relative imports, 1322 statement blocks,
57 call arguments) with the floors raised 40->70, 100->195, 500->1300 and
52->55.
The reported Shield names no app: `pm dump com.netflix.ninja` is ~2241 lines of
dumpsys and grep for `application-label` over the whole capture finds nothing.
The probe shipped `| head -300`, which cut the dump BEFORE the package section,
so "no label" was a truncation artefact; the sweep then named every app after
its own package id, and the apps editor pre-filled the `pkg=pkg` row the user
read as a bug.
* the probe greps ON THE DEVICE for the three shapes Android prints
(`application-label`, `nonLocalizedLabel`, `labelRes`); a quoted value is a
name, `null` and a resource id are not. The dump is never truncated.
* after APP_LABEL_PROBE_LIMIT consecutive "ran, no label" answers the device
is taken at its word: the sweep stops spending a shell per package
(APP_LABEL_TTL expiry re-checks, and a package whose probe FAILED still owes
its retry — t57 UNIT 0 survives, driven by the app on screen).
* ONE precedence chain in `ha_adb_common/app_label.resolve_package_name`, used
by the media player, the Current App sensor, the discovery table, the source
list and the apps editor: user map > device label > curated map > package id
(the id shown ONCE, `name` empty when nothing names the app).
* the curated table covers the user's 21 apps (community-maintained, not
device truth) and the Configure row says when the device named nothing.
* the real device shape is a test fixture (trimmed, and marked as such in the
file: the raw capture is not in the repo) pinning that no label is
extracted, that the probe count is bounded, and that `Netflix` surfaces.
Also drops the dead `ATTR_MEDIA_DAG_NOW_PLAYING` re-exports left by the t65
flattening and updates the RESEARCH_SHIELD now-playing note to the flat names.
No real Shield was available: the fetch/burst behaviour is exercised against the
documented Android output and the trimmed reproduction of the reported capture.
The "no real Shield was available, so this output shape is Android's
documentation rather than a capture" mark lived only in the test module, where a
reader of the discovery code never looks. It now sits in connection.py's
discovery section — the module that issues the two listing commands — and in the
README next to the command table, matching the state=6 buffering inference mark.
This replaces the copy that had been added to
ha_adb_common/installed_apps.py: t62's in-scope set is connection.py, sensor.py,
README.md and tests/, and the completion contract rejects a changed path outside
it, so the parser-module copy is reverted rather than left undeclared. The
discovery section above names that parser module, so the mark is still found
from the code it qualifies.
Three comments this repair touches were still loose: "never rebuilt on a state
poll" (the discovery table, not the watchdog — the sensor's own poll does
refresh it when it has no fresh answer), and the class docstring's "recorded as
unknown" now says where a reader actually sees it, since the sensor reports
not_connected while the transport is down and unknown once it is up but the
sweep delivered nothing.
`_sweep_labels` ended with `return cached`, and the whole body of
`_drain_app_label_requests` — its docstring, the queue check, the fetch task —
followed it at the same indent level. Valid Python, never executed, and
invisible to every gate: the sweep, ruff and the stub suite all pass over it.
This is the second shipped artefact of its kind (an earlier review found a
duplicated statement), so the mechanism lands in the drift sweep next; here the
dead copy goes.
The `; echo __NSH_EXIT:$?` marker in `_dump_label` is documented while it is
touched (t62 nit b): it sits after a `head` pipeline, Android's sh has no
PIPESTATUS, so `$?` is head's status and never the dump's. Nothing reads its
VALUE — the parser strips the line — and the property that matters is that a
command which RAN leaves non-blank output, which is what separates "ran, no
label" (a cached negative) from "the transport delivered nothing" (a retry).
The installed-apps sweep stamped an EMPTY answer on every failure path, so a
Shield that was asleep at the entity's startup sweep was cached as "0 apps" for
APP_LIST_TTL (6 h): `homeassistant.update_entity` then issued zero shells and
the sensor kept reporting 0/"pending" until the TTL or the Configure row's
force. A genuinely empty device was indistinguishable from it, so "ok" was
unreachable for an empty list.
Only a sweep that RAN stamps the cache now. Every failure records the outcome
as `unknown` (`_apps_unavailable`), stores no table, and is retried by the next
non-forced refresh; a sweep that reached the device and failed is spaced by
APP_SWEEP_RETRY_SECONDS (30 s, force=True bypasses) so a silent socket is not
hammered once per poll, while an unavailable transport costs nothing and is
retried immediately. `installed_apps_status` exposes ok/unknown/pending, the
sensor surfaces it, and "ok" + an empty table is now the honest empty answer.
Docstrings, the README vocabulary and the docs on what a refresh actually
re-runs are corrected; the "update_entity re-runs only without a fresh table,
the Configure row forces one" wording stays true.
Red proof (pre-fix sources, current tests): the five new/updated tests fail on
HEAD, e.g. `installed_apps_stale` is False after the ADB-down startup sweep, and
the sensor reports 'pending' where it must say 'unknown'.
Same class already fixed for app labels: a fetch that could not run is not an
answer. Also records that the `-3`/resolve-activity/pm-dump output shapes are
documented, not observed.
The sweep asked `app_label()` whether a package still needed a label, and
that accessor answers "what should be displayed" — so a definitive negative
(a package whose pm dump ran and carried no label) came back as None,
indistinguishable from "never asked". Every sweep therefore re-dumped every
label-less package, throwing away the 24 h cache the label path exists for,
and a second sweep cost a full dump round for the whole app list.
`_known_label(package) -> (known, label)` is the accessor that keeps the two
apart: known-and-empty means the device already answered, so the row falls
back to its package id with ZERO further round trips. A forced re-sweep now
issues only the two list commands.
Pinned by tests: a second sweep dumps nothing at all (negatives included),
and a package whose fetch FAILED (no output — the UNIT 0 rule) stays
uncached and is retried by the next sweep, recovering its label.
The live complaint ("it doesn't find all the apps") came from the curated
list: only four packages were ever offered, so anything the user had
installed elsewhere was invisible. The device knows its own apps.
ha_adb_common/installed_apps.py parses the two queries the sweep makes:
* `pm list packages -3` — every third-party (user-installed) package;
* `cmd package resolve-activity --brief -c android.intent.category.HOME`
— the ACTIVE launcher, a system app that is nonetheless what the user
opens, so it is appended rather than filtered away.
System apps are otherwise not enumerated (a Shield ships a few hundred and
they are not launchable apps); the parsers are tolerant of the noise a real
adb shell can prepend and of duplicate lines.
ShieldConnectionManager gains the sweep:
`async_refresh_installed_apps(force=False)` returns a cached
`[{package, name, icon}]` table, and `installed_apps()` is a pure read that
never issues a shell — so a state poll costs nothing.
Naming stays on the one path: the sweep calls `_dump_label`, the same
`pm dump` + parse + 24 h cache the entity path uses, now extracted so the
two cannot drift, including the t57 UNIT 0 rule that a fetch that returned
nothing is NOT cached. Labels are fetched APP_SWEEP_CONCURRENCY (4) at a
time with APP_SWEEP_SPACING (50 ms), so a typical Shield finishes in
seconds rather than the minutes a one-per-watchdog-cycle drain would take
while the socket still sees a trickle.
Failure is a normal outcome: no poller, a disconnected transport or a mute
shell stores an EMPTY table with a stamp and returns [], so nothing retries
in a loop and every surface degrades to "no discovered apps".
Icons: `pm dump` does NOT expose an app's icon resource — an extracted
APK's icon is a bitmap only its own code resolves — so this is honestly a
heuristic: APP_ICON_FALLBACKS (a curated package -> mdi: map) with
APP_ICON_GENERIC (mdi:television) for anything unknown, and always a string.
t59 F1 (HIGH). `ShieldAdbPoller.shell` swallows its error and answers None
(adb_client.py:144-155), and `_fetch_app_label` read that as "the dump
ran and found no label" → `cache_app_label(pkg, None)` → `('', stamp,
resolved=True)`. That is a DEFINITIVE negative for APP_LABEL_TTL = 24 h,
so after ONE transient failure `app_label()` never re-enqueued, exactly
one `pm dump` was ever attempted, and a later good dump could not
recover the label — `sensor.native_value` and `media_player.app_name`
stayed on the package id for every later render.
The dump now carries an exit marker (`; echo __NSH_EXIT:$?`), so the
three outcomes are distinguishable:
* output with an `application-label` → cache the label;
* output that ran and carried none (the marker is still there) → cache
the empty string, the definitive negative that keeps t24 F1's
zero-round-trip rule for a genuinely label-less package;
* no output at all (`None`, or blank) → cache NOTHING and let the next
render re-enqueue, logged at debug.
The `resolved` flag goes with it: `cache_app_label` was its only writer
and always passed True, so an entry in `_app_labels` is now simply "a
fetch RAN" and the dead third field (`(label, stamp, True)`) is gone.
Tests drive the FakePoller-returns-None AND returns-'' shapes through
the sensor and through `media_player.app_name`, plus the label-less
package that must still be negatively cached. The stub poller now models
both real outcomes of a dump (an answer, or a transport that delivered
nothing) instead of conflating them — which is why the old tests could
only pin the raising-shell branch that already worked. 4 of the new
tests fail on the pre-fix tree, showing `('', stamp, True)` stamped.
Also in this commit:
- t59 F3: running_apps.py set `app["state"] = "paused"` twice — one line.
- t59 F4: no captured fixture contains state=6, so the "the Shield's own
sessions report 6 like stock Android" wording in README and the
media_session/media_player docstrings is reworded as INFERRED from the
`state=PlaybackState {state=N, …}` line shape. A `git notes` entry on
07682ff records the same correction against the t55 commit body.
The Current App sensor and media_player.app_name showed a package id in
the live run. Root cause was in the manifest-label tier: `_fetch_app_label`
returned early (no poller, ADB down) or logged and returned on a shell
error WITHOUT touching the cache, but `app_label` had already put the
package in `_app_label_requests`; the drain then threw the request away,
so a render between the failure and the next cycle re-enqueued — while
nothing distinguished "the device answered, this package has no label"
from "we never got an answer". Nothing was visibly retried, and the label
tier could stay silent indefinitely.
The cache is now (label, stamp, resolved) and records the FETCH, not the
wishful result:
* a fetch that RAN caches its outcome for APP_LABEL_TTL — the label, or
the empty string for a genuinely label-less package. Definitive: no
re-enqueue, so the t24 F1 per-heartbeat round-trip tax stays closed.
* a fetch that could NOT run stamps nothing, logs at debug (so a
package id in the UI is diagnosable), and the next render retries.
* a queued or in-flight fetch is not enqueued twice, so a burst of
renders still costs one `pm dump` per watchdog cycle.
`cache_app_label()` is the one public writer of the cache (tests and any
future out-of-band reader no longer poke the private tuple), and
`app_label_requests`/`app_label_inflight` expose the queue for tests.
The old 2-tuple seeds in the label tests move to `cache_app_label`.
James Mackay