From afc5f0a1d03b63e5aabe56a2c35b9f77528dd80d Mon Sep 17 00:00:00 2001 From: Levi Date: Thu, 8 Oct 2026 07:18:27 +0100 Subject: [PATCH] docs: add MVP plan and Shield firmware research findings --- docs/MVP_PLAN.md | 512 ++++++++++++++++++++++++++++++++++++++++ docs/RESEARCH_SHIELD.md | 396 +++++++++++++++++++++++++++++++ 2 files changed, 908 insertions(+) create mode 100644 docs/MVP_PLAN.md create mode 100644 docs/RESEARCH_SHIELD.md diff --git a/docs/MVP_PLAN.md b/docs/MVP_PLAN.md new file mode 100644 index 0000000..c13a26a --- /dev/null +++ b/docs/MVP_PLAN.md @@ -0,0 +1,512 @@ +# ha-nvidia-shield MVP Plan (Shield-only fork) + +Scope: Shield-only. Primary state = ADB media session tracking (`dumpsys media_session`), +persistent event-driven connection, with a fallback path for apps that don't publish +media sessions. ADB pairing flow stays intact. Companion doc: `docs/RESEARCH_SHIELD.md` +(firmware research results, fixtures, and the user's current YAML glue). + +**Decisions locked:** +- Option 1 adopted: absorb the remote protocol (`androidtvremote2`) — all-in-one device. +- TiviMate (`ar.tvplayer.tv`) publishes no media session → headline fallback app. +- `play_media(channel)` replicates core androidtv_remote behavior exactly + (digits, 0.1 s delay, no ENTER). + +--- + +## 1. Current state assessment + +### Architecture as found + +``` +config_flow.py ── user step → adb_pair step (one-shot connect test) +__init__.py ── no runtime object; hass.data[DOMAIN][entry_id] = entry (nothing!) +adb_session.py ── AdbMediaSessionPoller = AdbClientBase + poll() (thin) +media_player.py ── the real "coordinator": entity owns ADB socket, polling loop, + intent tracking, state tiering, Jellyfin/media-info, power logic +sensor.py ── 2 read-only mirror sensors +button.py ── Pair ADB (throwaway poller, connect→disconnect→reload) +jellyfin.py ── piggybacks on HA core Jellyfin coordinator cache (zero extra HTTP) +power.py ── sibling-sensor discovery for the smart plug +ha_adb_common/ ── vendored: adb_client (connect/shell/keyevents/media_session poll), + media_session parser, running_apps, intent dataclass, keys manager +``` + +There is **no DataUpdateCoordinator and no shared runtime object**. The entity +(`NvidiaShieldMediaPlayer`) *is* the coordinator: it creates the poller in +`async_added_to_hass`, runs `async_call_later` polling, and writes state itself. + +### What already works (keep) + +| Area | File | Notes | +|---|---|---| +| ADB key lifecycle: generate, migrate from legacy paths, namespace per domain | `ha_adb_common/keys.py` | Solid; pairing-flow compatible. Keep untouched. | +| Pairing UX: config-flow step + Pair ADB button + "approve on Shield" guidance | `config_flow.py`, `button.py` | Works; keep intact per constraint. | +| ADB connect via `adb_shell` (`AdbDeviceTcpAsync` + RSA signer in executor) | `adb_client.py` | Correct executor usage; blocking I/O off the loop. | +| Media session parsing: package/active/state/metadata from `dumpsys media_session`; foreground-app preference; playing>paused ordering | `media_session.py` | Works on Shield Android 11; version-drift risk (see fragility). | +| Optimistic UI: `PlaybackIntent` + self-heal + one retry + external-override detection | `intent.py`, `media_player.py` | Good design; survives rewrite. | +| State tiering: power → intent → Jellyfin → ADB session → ON | `media_player.py` `state` property | Sound priority model. | +| Jellyfin metadata piggyback (resolution/HDR/audio labels from core coordinator cache) | `jellyfin.py` | Zero-cost; keep. | +| Power: ADB wake as authority, plug "off" as hard off, sibling sensor discovery | `power.py`, `media_player.py` | Keep. | +| Running apps from `dumpsys activity recents` | `adb_client.py`, `running_apps.py` | Useful for Media DAG switcher. | +| Media commands: ADB keyevent when connected & Jellyfin inactive, else androidtv_remote | `media_player.py` `_media_cmd` | Sensible dual path; superseded by protocol-first routing in Phase 2. | + +### What is fragile + +1. **Connection ownership is in the entity.** The ADB socket, its poll timer and all + state live on `NvidiaShieldMediaPlayer`. Reload (options change triggers full + entry reload via `_async_update_listener`) tears down and re-handshakes the ADB + connection. Sensors have no access to ADB data. A second entity is impossible. +2. **Untracked in-flight poll tasks.** `_adb_poll` is dispatched via + `async_create_task` and never cancelled on unload; an in-flight `shell()` after + `disconnect()` hits `shell()`'s auto-reconnect → surprise reconnect after removal. +3. **Silent failure mode.** `shell()` swallows errors at DEBUG, returns `None`, marks + disconnected, and the next poll is 300 s later. Meanwhile `self._adb_awake` and + `last_info` keep their last values — the entity can report `playing` for minutes + while the device is asleep/dead. **No staleness/expiry on media session info.** +4. **Wake detection is a substring.** `"Awake" in dumpsys power result` — works on + this firmware (`mWakefulness=Awake/Asleep`, verified R5) but fragile against + future values like `Dozing`; switch to exact value match. +5. **Parser depends on exact dumpsys text** (`package=`, `active=true`, + `state=PlaybackState` `state=3/2`, `description=` positional split) and `head -200` + truncates the dump — sessions below the cut are lost. No position parsed from + `PlaybackState` despite it being present (verified R9: position/ms + uptime-ms + `updated`). +6. **Intent retry sends the wrong path.** `_on_intent_timeout` retries via + `_cmd()` (androidtv_remote) even when the original command succeeded via ADB. +7. **Dead/duplicated logic.** `jf_active` computed and unused in `_adb_poll`; + `_resolve_adb_host` duplicated in 3 places; `JELLYFIN_APP_IDS` defined but never + used (Wholphin routing); Current App sensor merely mirrors androidtv attributes. +8. **State writes gated on change.** `_adb_poll` writes HA state only when wake state + flipped or `last_info` exists — `running_apps` changes and info *clears* can be + missed; stale attribute windows. +9. **`power is False → IDLE` not OFF.** ADB-asleep maps to IDLE (standby) by design, + but the plug-off check happens before ADB, so semantics are split across two + places and easy to break. +10. **No tests, no CI lint gate** (ruff/pytest configured in `pyproject.toml` but + zero test files). A rewrite will be blind. +11. `iot_class: local_polling` + per-entity 10 s polling of **three** dumpsys + commands (power, media_session, activity recents) — wasted round trips when + nothing changed. + +### What's missing for MVP + +- Persistent connection manager with watchdog + backoff (nothing tracks connection + health; the only "watchdog" is the next poll). +- Event-driven trigger source (foreground-app change events) — today everything is + timer-driven. +- Fallback foreground/state path for apps without media sessions (TiviMate is the + headline case — verified no session, R2). +- Playback position from media sessions (+ uptime-anchored math, R9). +- Media session staleness handling (session gone → transition to idle). +- Tests + CI (lint + unit) to make the rewrite safe. + +--- + +## 2. Event-driven rewrite: keep or replace the polling layer? + +**Decision: replace the per-entity poll loop with an entry-scoped +`ShieldConnectionManager` that runs a hybrid event/heartbeat loop. Do not try to +make logcat a true push stream over `adb_shell` for MVP.** + +### Why not pure logcat push (yet) + +`adb_shell`'s `AdbDeviceTcpAsync.shell()` is **request → full-response**; there is no +incremental/streaming read API on the stable surface. Running `exec logcat` blocks +forever. A true stream means either hacking adb_shell internals (fragile across its +releases) or adding a dependency (violates constraints). Also, logcat does not +reliably carry *playback state* changes for third-party apps — it reliably carries +**foreground/app lifecycle** events (verified R8: `ActivityTaskManager: START u0 +{… cmp=/}` on every app open, including home-button → launcher). + +### The hybrid design (target architecture) + +``` +entry.runtime_data = ShieldConnectionManager + ├─ transport A: androidtvremote2 protocol client (Phase 2) + │ push callbacks: is_on, is_available, current_app, volume_info + │ commands: send_key_command, send_launch_app_command, IME + ├─ transport B: ADB (adb_shell) — persistent socket, no re-handshake per command + ├─ watchdog task: health ping + exponential backoff reconnect (5s→…→300s cap) + ├─ trigger listener: foreground-change events + │ (logcat -d -T1 ActivityTaskManager delta snapshots ~2–5 s cadence; + │ Phase 3 option: internals streaming if adb_shell ever supports it) + ├─ on trigger → cheap `dumpsys activity activities` (mResumedActivity confirm) + │ → if changed → `dumpsys media_session` fetch + ├─ heartbeat: `dumpsys media_session` every ~15–30 s while awake (safety net) + ├─ parsed StateStore: {is_on, foreground_pkg, sessions[], audio_playback[], + │ last_updated ts per field} + └─ event API: listener registration so entities are thin views +``` + +Net effect vs today: one persistent socket instead of reconnect-on-failure; +wake-ups driven by device events within seconds; heartbeat as fallback; wake check +folded into the foreground query instead of a separate grep; per-cycle cost drops +from 3 dumpsys commands × every 10 s to ~1 command × on-event + 1 × per heartbeat. + +### Blocking issues to fix before/during the rewrite (flagged) + +| # | Blocker | Where | Fix | +|---|---|---|---| +| B1 | Connection owned by entity; no shared runtime object | `media_player.py` `_setup_adb`, `__init__.py` | Move connection+state into `ShieldConnectionManager` stored in `entry.runtime_data`; entities subscribe. Mirror HA core androidtv's `entry.runtime_data` dataclass pattern. | +| B2 | Untracked `async_create_task` polls escape unload | `media_player.py` `_adb_poll_callback` | Manager owns its tasks; cancel on `async_stop` + `EVENT_HOMEASSISTANT_STOP`; guard `shell()` reconnect behind explicit lifecycle calls. | +| B3 | `shell()` auto-reconnect hides failure from callers | `adb_client.py` `shell` | Explicit error propagation; reconnect handled only by watchdog. Adopt androidtv core's `adb_decorator` shape: catch known ADB exceptions → log once → close → mark unavailable. | +| B4 | `head -200` truncation + parser text-coupling | `adb_client.py`, `media_session.py` | Remove `head`; tolerant parser w/ real firmware fixtures (captured — `docs/RESEARCH_SHIELD.md`). | +| B5 | Stale-state window (no TTL on `last_info`/`_adb_awake`) | `media_player.py` | StateStore carries timestamps; info older than TTL (e.g. 2×heartbeat) is expired. | +| B6 | Options change reloads entry → full reconnect churn | `__init__.py` update listener | Adopt core androidtv's pattern: only reload for structural options (`RELOAD_OPTIONS`), otherwise re-read options via `async_dispatcher_send`. | + +--- + +## 3. Fallback mechanism for apps without media sessions + +Options considered: + +| Option | Reliability | Cost | Notes | +|---|---|---|---| +| **A. Foreground app via `dumpsys activity activities` (`mResumedActivity` / `topResumedActivity`)** | High on Shield (Android 11/12 line) | 1 cheap command, on-event only | Deterministic current app; works for *every* app. Verified on this Shield (R4): `mResumedActivity` is the Android-11 pattern; `topResumedActivity` kept as newer-firmware fallback. | +| B. Intent tracking (Android usage/intent logs) | Low–medium | Higher parse cost | Requires AccessibilityService/app instrumentation on modern Android — not viable via ADB shell. Reject. | +| C. Last-known-state cache (retain last session per package with timestamp) | Medium (supplementary) | Memory only | No extra polling; answers "what was this app doing". | +| D. logcat foreground listener | High for events | Part of the listener work | Covered by the hybrid loop above; R8 verified `ActivityTaskManager: START` lines. | +| **E. "Audio playback" section of `dumpsys media_session`** | High as a *hint* | Zero (same command) | NEW from R2: lists recently-playing audio packages even with no session (`ar.tvplayer.tv` shows there). Confidence check that a foreground app is actually playing media. | + +**Recommendation: build A + C — with the Option-1 decision (§9), the absorbed +protocol client's `current_app` push is the *primary* foreground signal.** A +(`mResumedActivity` dumpsys) is the fallback when the protocol client is down +or silent; C (last-known-state cache) turns "foreground but no session" from +UNKNOWN into "in (idle / last known paused)"; E enriches the fallback tier +with a "media recently active" hint at zero round-trip cost. D lands as the +ADB-side trigger in Phase 1. + +Resulting state ladder for MVP: protocol is_on → intent → Jellyfin → media +session → foreground fallback (`current_app` push, then ADB `mResumedActivity`; +state = "on, in app X", with last-known metadata if any) → ON. Audio-playback +identity (E) enriches the fallback tier with a "media recently active" hint. + +--- + +## 4. Phased MVP plan + +### Phase 0 — Stabilize & trim (foundation; ~1–2 days) + +Goal: make the codebase safe to rewrite. No behavior change. + +Files: +- `custom_components/nvidia_shield/__init__.py` — introduce `entry.runtime_data` + placeholder (shared per-entry context object, initially just config mirror). +- `custom_components/nvidia_shield/const.py` — add TTL/staleness constants, + backoff constants; wire the currently-unused `JELLYFIN_APP_IDS` and add + `com.github.damontecres.wholphin` + `ar.tvplayer.tv` package constants. +- `custom_components/nvidia_shield/adb_session.py` + `ha_adb_common/adb_client.py` + — de-duplicate host resolution into one helper; make `shell()` propagate errors + (B3) while keeping current callers working; stop swallowing poll errors silently. +- `custom_components/nvidia_shield/media_player.py` — remove dead `jf_active`; + fix intent retry to use the path that originally succeeded; track+cancel poll + tasks on unload (B2); write state whenever any ADB-derived value changes. +- `tests/` (new) — parser fixtures (real `dumpsys media_session`, `activity + recents` outputs), intent-logic unit tests, config-flow smoke tests. Real + firmware fixtures are already captured — see the fixtures table in + `docs/RESEARCH_SHIELD.md`. +- `pyproject.toml`/CI — run ruff + pytest on PR. + +Risks: parser fixture drift across Shield firmware (mitigate: fixtures from 2+ +firmware versions, tolerant regexes). Fixing `shell()` error propagation can expose +previously hidden reconnect bugs — expected, ship together with the task-tracking +fix. + +Done when: all tests green; no in-flight task survives entity unload (test +proves it); intent retry honors command path; ruff clean. + +### Phase 1 — Core playback state + connection reliability (~3–5 days) + +Goal: reliable, fast, persistent connection; media session is the source of truth. + +Files: +- `custom_components/nvidia_shield/connection.py` (new) — `ShieldConnectionManager`: + persistent socket, watchdog + backoff reconnect, on-demand fetch API, StateStore + with timestamps, listener registration. API designed to host a second transport + (the remote protocol joins in Phase 2). +- `custom_components/nvidia_shield/adb_session.py` — becomes thin adapter over the + manager (keep `ensure_adb_key` re-export for config flow/button). +- `ha_adb_common/adb_client.py` — extract protocol-level commands (no loop + scheduling); add explicit `health_check()`. +- `ha_adb_common/media_session.py` — parse `PlaybackState` position (ms, with + uptime-anchored math — `updated` is uptime-ms per R9); add session-expiry at + store level. +- `media_player.py`, `sensor.py` — entities become views over StateStore; + state tiering unchanged; availability reflects manager connection health + (`_attr_available`). +- `manifest.json` — `iot_class: local_polling` stays until the protocol client + (push) lands in Phase 2, then becomes `local_push`. +- `config_flow.py` — pairing step reuses manager's test-connect (no behavior + change to flow). + +Risks: `adb_shell` async stability over long-lived connections (mitigate: +watchdog pings + teardown/rebuild on protocol errors); reconnect storms during +device reboots (backoff + jitter); intent logic regression (covered by Phase-0 +tests). + +Done when: device reboot recovers automatically within backoff window; play/pause +reflected in ≤1 heartbeat cycle (≤30 s) worst case, foreground-triggered changes +faster; unavailable/available transitions correct; no state staleness beyond TTL; +tests for manager (connect/reconnect/expiry) pass. + +### Phase 2 — All-in-one: absorb the remote protocol (androidtvremote2) (~4–6 days) + +Goal: one integration, one device. We own the Google remote protocol for +commands, power and availability; ADB remains the playback-state source. The +external androidtv / androidtv_remote integrations are no longer required. + +Files: +- `manifest.json` — add `androidtvremote2` requirement (`adb-shell` stays). +- `config_flow.py` — protocol pairing ported from core androidtv_remote: generate + cert if missing, fetch name + MAC (unique_id), PIN pair step (code shown on the + Shield), reauth flow on InvalidAuth (device hard reset). The existing ADB + pairing step stays intact — both pairings coexist in one flow. +- `connection.py` — manager now hosts both transports: protocol client + (`keep_reconnecting`, callbacks: is_on, is_available, current_app, volume_info) + + ADB client; one StateStore fed by both. +- `remote.py` (new platform) — protocol-backed remote entity (port of core: + SHORT/START_LONG/END_LONG command prefixes, num_repeats/delay/hold, activity = + current app). +- `media_player.py` — command routing switches to protocol keys; volume gains + absolute level (VOLUME_SET) + real mute state; availability and is_on come + from protocol callbacks; `_is_power_on` ladder: protocol is_on → ADB wake → + plug off. +- `media_player.py` `async_play_media` — §7 channel port via protocol + `send_key_command` digits (0.1 s delay) + task-cancel guard; ADB keyevent + fallback when the protocol is down. +- `__init__.py` — PLATFORMS gains REMOTE; entry migration (VERSION 2); close + both clients on HA stop / unload. + +Risks: two pairing experiences (ADB + PIN) need clear flow copy; +androidtvremote2's own reconnect task and our ADB watchdog must not fight each +other — one owner per transport; existing installs that wrap the external +androidtv_remote entity need a migration path (transitional fallback while it's +still configured); IME option defaults off. + +Done when: entry sets up standalone (no androidtv/androidtv_remote entries +needed); remote entity sends keys and launches apps; real volume level/mute +shown; channel selection works via protocol with ADB fallback; device hard reset +triggers reauth; ADB pairing unchanged; migration covered by tests. + +### Phase 3 — Fallback app state (~2–3 days, smaller now) + +Goal: every app gets a sensible state even without a media session. The protocol +client's `current_app` push is the primary foreground signal; ADB fills gaps. + +Files: +- `ha_adb_common/adb_client.py` (or new `foreground.py`) — `mResumedActivity` + parser (Android 11 primary pattern; `topResumedActivity` kept as a + newer-firmware fallback) — used only when protocol current_app is + None/unknown/stale. Do not parse `dumpsys window` focus first (multiple + `mCurrentFocus` lines incl. literal `null` — R4). +- `connection.py` — foreground priority in StateStore: protocol current_app push + > ADB `mResumedActivity`; last-known-state cache per package (option C); + audio-playback identity parsed from the same media_session dump (E). +- `ha_adb_common/media_session.py` — parse the "Audio playback (lastly played + comes first)" section into `{package: last_played_hint}`; ignore-list + `com.google.android.apps.mediashell` (Chromecast receiver process). +- `media_player.py` — state ladder gains the foreground tier; running_apps + enrichment with foreground/last-known info. +- `sensor.py` — Current App sensor reads StateStore (protocol push makes it + accurate without androidtv attribute reads). +- README — document the state ladder and fallback semantics; document the + migration away from mediasession2mqtt sensors + the universal-player YAML + glue (appendix in `docs/RESEARCH_SHIELD.md`). + +Risks: dumpsys format differences across Shield Experience releases (fixtures +from real devices; R4); apps that keep background sessions while another is in +the foreground (session-vs-foreground preference already handles); last-known +cache must never resurrect stale "playing". + +Done when: launching an app with no media session shows correct app + "on/idle" +within one update cycle; returning to an app with a session shows its state; +state tracks `current_app` push with no attribute polling; stale cache never +reports playing after >TTL; **`play_media` (`media_content_type: "channel"`) +selects TiviMate channels by number (protocol path, ADB fallback) per §7**; +tests cover all ladder branches. + +### Phase 4 — QoL (~ongoing) + +Files: +- Deep links — now cheap: `media_type: app/url` → protocol + `send_launch_app_command` (port of core pattern) + BrowseMedia app directory; + per-app `am start` extras via ADB only for apps the protocol can't launch + (data-driven map in const). +- Options: apps name/icon map editor (core androidtv_remote pattern), digit + delay, IME toggle, privacy scope; finish config entry migration if needed. +- Zeroconf discovery (`_androidtvremote2._tcp`) — auto-fill host, update entry on + IP change; `manifest.json` gains the `zeroconf` integration dependency. +- `adb_command` service + diagnostics (port androidtv patterns). +- Optional: logcat streaming via adb_shell internals if verified stable (behind + an option flag); otherwise keep snapshot approach. +- Trim: fold Current App sensor if StateStore covers it; drop + `running_apps_from_sessions` legacy; remove transitional external-entity code. + +Risks: deep-link semantics differ per app (intent extras) — keep the app map +data-driven; options migration bugs (test entry migration). + +Done when: deep links work for 2–3 reference apps; apps editor works; discovery +updates host on IP change; docs updated. + +--- + +## 5. Dependency & compatibility notes + +- Runtime deps: `adb-shell>=0.4.0` + `androidtvremote2` (adopted in Phase 2 — + small, HA-core-blessed); `zeroconf` integration dependency joins in Phase 4 + for discovery. Logcat *streaming* + is the only thing that might eventually + need internals or a new dep — explicitly deferred and flagged. +- Vendored `ha_adb_common` stays for HACS; keep the hisense_tv consumer in mind + when changing its API (add, don't break; the fork can diverge later but Phase 0–2 + changes should remain additive where possible). +- Pairing flow: ADB pairing (`async_step_adb_pair`, Pair ADB button, + `ensure_adb_key`, `.storage/nvidia_shield/adbkey` layout) is untouched in all + phases. The remote-protocol cert/PIN pairing (`.storage` certs) is *added* + alongside it in Phase 2. + +## 6. Sequencing summary + +| Phase | Theme | Primary outcome | +|---|---|---| +| 0 | Stabilize/trim, tests | Safe rewrite base; no behavior change | +| 1 | Persistent connection + session truth | Reliable fast playback state, auto-recovery | +| 2 | Absorb remote protocol (androidtvremote2) | All-in-one device; channel select; focused-app push | +| 3 | Foreground fallback + last-known state | Full-app coverage without sessions | +| 4 | Deep links, apps editor, discovery, diagnostics | Polish | + +--- + +## 7. TiviMate channel selection (`play_media` port) + +**Requirement:** retain `media_player.play_media` with `media_content_type: "channel"` +so TiviMate channels can be selected by number (e.g. `media_id: "1"`). This logic +lives today in the Android TV Remote integration; with Option 1 adopted we port +it onto our own protocol client (primary path) with ADB keyevents as fallback. + +### Port design + +- Add `MediaPlayerEntityFeature.PLAY_MEDIA` and implement `async_play_media(media_type, media_id)`. +- When `media_type == "channel"`: extract digits from `media_id`, then send each + digit as an ADB keyevent — `KEYCODE_0` = 7 … `KEYCODE_9` = 16 (digit n → keycode 7+n) — + with a configurable delay between digits (default ~0.5 s), then a confirm + keycode. Reference (verified from HA core `androidtv_remote/media_player.py`): + core sends digits ONLY — `media_id.isnumeric()`, cancels any prior in-flight + channel task (concurrency guard), `delay_secs=0.1`, and never sends a confirm + key: the device's channel overlay auto-selects. Port: digit n → keycode 7+n; + make ENTER an optional boolean (default off — replicates the androidtv_remote + behavior the user confirms works today). +- Other `media_type` values (`app`, `url`): launch via the protocol client's + `send_launch_app_command` once Phase 2 lands; until then reject unknown types + with a log warning. +- **Fallback chain:** protocol client down → ADB `input keyevent` digits (keycode + 7+n, slower but socket-independent) → transitional only: delegate to the + still-configured external androidtv_remote entity (migration path, removed in + Phase 4). Order: protocol first, ADB second, legacy third. +- Command path: protocol `send_key_command` (push-based, fast, no extra shell + round trip); ADB keyevents only as the fallback (works whenever the socket is + up, regardless of the remote protocol's state). +- Timing risk: too-fast digits can be dropped or the overlay can close early — + make the inter-digit delay a `keyvalue` option with a safe default (0.1 s, + core-verified); the live check (R7) remains optional. +- Service-call note: current HA syntax passes `data.media.media_content_id` / + `media_content_type`; `async_play_media` receives `"channel"` / `""` the + same as before. + +### Metadata implication + +Channel *metadata* (EPG name, title) for TiviMate is a separate question from +channel *selection*: + +- TiviMate: **no media session** (R2 verdict) → now-playing = Phase-3 foreground + fallback: state = "on, in TiviMate", title = `last_channel` cache when + play_media fires (cheap, honest, no EPG claim). Full EPG/EPG-logo data is out + of scope for the ADB-only MVP. +- Jellyfin-family clients: **Wholphin (`com.github.damontecres.wholphin`)** + publishes a proper session with usable metadata (R9) — routes through + `JELLYFIN_APP_IDS` to the rich-metadata path. +- Netflix publishes a stale inactive session (`active=false`, `state=null`) — + the parser must keep ignoring it; a fixture pins this behavior. + +### Where it lands + +- Phase 2 (the port target is the protocol client absorbed in Phase 2; TiviMate + state *display* comes from Phase 3's fallback). + +--- + +## 8. Local research on the Shield — STATUS + +**STATUS: DONE.** R1–R5, R8, R9 executed; R6 optional-not-run; R7 skipped by +decision. Full results, raw captures, and the user's current YAML glue are in +`docs/RESEARCH_SHIELD.md`. + +| Step | Status | Key result | +|---|---|---| +| Setup | ✅ | Shield `mdarcy`, Android 11, `RQ1A.210105.003`; custom launcher `com.spocky.projengmenu` | +| R1 | ✅ | TiviMate package = **`ar.tvplayer.tv`** (plan's `iv.android.tv` guess was wrong) | +| R2 | ✅ | **No media session** for TiviMate (branch 3). Netflix stale inactive session only. NEW: "Audio playback" section lists `ar.tvplayer.tv` | +| R3 | ⏳ partial | Wholphin + Netflix captured; YouTube/Spotify/Plex optional | +| R4 | ✅ | `mResumedActivity` is the Android-11 pattern; no `topResumedActivity`; `mCurrentFocus=null` trap | +| R5 | ✅ | `mWakefulness=Awake/Asleep` — switch to exact value match | +| R6 | ⏳ | Not run (optional; low risk) | +| R7 | ⏭ skipped | Replicate core behavior (digits, 0.1 s, no ENTER) — user's existing flow works | +| R8 | ✅ | `ActivityTaskManager: START` lines reliable incl. home button; user has `be.digitalia.mediasession2mqtt` installed | +| R9 | ✅ | `PlaybackState{state=3, position (ms), speed, updated (uptime ms)}` parseable; Wholphin session good (`description=Twirlywoos`) | + +--- + +## 9. HA core research: androidtv + androidtv_remote (what's worth taking) + +Sources reviewed on `dev`: +[androidtv](https://github.com/home-assistant/core/tree/dev/homeassistant/components/androidtv) · +[androidtv_remote](https://github.com/home-assistant/core/tree/dev/homeassistant/components/androidtv_remote). + +Key findings first: + +1. **`androidtv` (ADB component) has NO `async_play_media`** — channel-by-number + logic is not there. It lives in **`androidtv_remote/media_player.py`**, so §7's + port source is confirmed to be androidtv_remote (not androidtv). +2. **Both components ship `PLATFORMS = [MEDIA_PLAYER, REMOTE]`** — HA core's own + precedent for one device exposing a media_player + remote entity pair. That is + exactly the all-in-one shape requested. +3. `androidtv`'s remote entity is pure ADB (`input keyevent`) — a basic remote + entity needs no remote-protocol dependency (superseded by the Option-1 + decision; its logic survives as our command fallback path). + +### Worth taking from `androidtv` (ADB component) + +| Pattern | File | Fold into | +|---|---|---| +| `entry.runtime_data` dataclass: client + options snapshot | `__init__.py` | Phase 1 manager (B1) — canonical HA pattern to mirror | +| `ConfigEntryNotReady` on first connect failure → HA itself retries setup | `__init__.py` | Phase 1 — better than our in-entity 300 s retry | +| Close connection on `EVENT_HOMEASSISTANT_STOP` | `__init__.py` | Phase 1 manager lifecycle | +| Options change **without full reload**: only reload for `RELOAD_OPTIONS`; otherwise re-read options via `async_dispatcher_send` | `__init__.py` update_listener | Proper fix for B6 (reload churn) | +| `adb_decorator`: uniform exception handling → log once, `adb_close()`, mark entity unavailable | `entity.py` | Phase 1 — replaces swallow-at-DEBUG `shell()` (B3) | +| Reconnect-on-update + `_failed_connect_count` (log first failure only) | `media_player.py` `async_update` | Phase 1 watchdog detail | +| `select_source` = launch app; `!name` prefix = close app; `source_list` built from running apps | `media_player.py` | Phase 3/4 QoL — natural fit with our existing `running_apps` | +| `adb_command` service, diagnostics | `services.py`, `diagnostics.py` | Phase 4 QoL | + +### Worth taking from `androidtv_remote` (remote protocol) + +| Pattern | File | Fold into | +|---|---|---| +| **Channel logic (the §7 port source)**: `media_type == CHANNEL`, `media_id.isnumeric()`, cancel previous in-flight channel `asyncio.Task`, `_send_key_commands(list(media_id), delay_secs=0.1)` — digits only, **no ENTER** | `media_player.py` | §7 | +| `app`/`url` media types → launch app; BrowseMedia app directory | `media_player.py` | Phase 4 deep links + browse | +| Entities as thin callback-views over one runtime API object (available / is_on / current_app / volume callbacks) | `entity.py` | Phase 1 manager → entity design | +| `keep_reconnecting()` + trigger reauth when the device hard-resets (cert invalid) | `__init__.py` | Phase 1/2 watchdog: distinguish "unreachable" (backoff) from "unpaired" (reauth) | +| Zeroconf discovery (`_androidtvremote2._tcp`, MAC as unique_id, host auto-update on IP change) | `config_flow.py` | Phase 4 QoL | +| Cert+PIN pairing, certs stored in `.storage` | `config_flow.py`, `helpers.py` | Phase 2 | + +### All-in-one: the remote as an entity alongside the media_player — DECIDED + +Today we *wrap* an external `androidtv_remote` entity (resolving its host from +its config entry). **Decision: Option 1 adopted** — androidtvremote2 is excellent +for *commands* but weak on receiving playback state; our ADB session tracking +fills exactly that gap, making the two transports complementary. Phase 2 absorbs +the protocol: we own the connection, one device provides `media_player` + +`remote`, and the `current_app` push becomes the primary foreground signal — +fewer dumpsys round trips and a more reliable Phase-3 fallback. ADB remains the +state source; the ADB pairing flow stays intact. The ADB-only remote entity +(Option 2) is not built as a separate platform; its `input keyevent` logic +survives as the command fallback path inside `async_play_media`/command routing. diff --git a/docs/RESEARCH_SHIELD.md b/docs/RESEARCH_SHIELD.md new file mode 100644 index 0000000..2cf2850 --- /dev/null +++ b/docs/RESEARCH_SHIELD.md @@ -0,0 +1,396 @@ +# Shield firmware research results (§8 of MVP_PLAN.md, R1–R9) + +Captured from the user's Shield. These outputs are the ground truth for parser +fixtures and resolved the R2 verdict. Plan deltas applied to `MVP_PLAN.md` are +listed at the end. + +## Environment + +| Property | Value | +|---|---| +| Device | NVIDIA Shield (board `mdarcy`) | +| Android | 11 (`ro.build.version.release=11`) | +| Fingerprint | `NVIDIA/mdarcy/mdarcy:11/RQ1A.210105.003/7825230_4231.0265:user/release-keys` | +| SDK | 30 (Android 11; the `getprop ro.build.version.sdk` call failed on shell truncation — `etprop: inaccessible or not found` — but Android 11 ⇒ SDK 30) | + +Note: the home launcher is **not** the stock Shield launcher — it is +`com.spocky.projengmenu` (Project Eternity menu). Foreground detection must not +assume `com.google.android.tv.launcher`. + +--- + +## R1 — TiviMate package name: `ar.tvplayer.tv` + +``` +$ adb shell pm list packages | grep -iE 'tivi|^package:iv' +package:ar.tvplayer.tv +``` + +**Verdict:** TiviMate's package is `ar.tvplayer.tv` — **not** `iv.android.tv` as +the plan assumed. All match keys, fixtures, and constants use `ar.tvplayer.tv`. + +--- + +## R2 — Does TiviMate publish a media session? **NO** (verdict: branch 3) + +Full `dumpsys media_session` with TiviMate in the foreground playing a live +channel (captured twice, before/after zapping): + +``` +MEDIA SESSION SERVICE (dumpsys media_session) + + 9 sessions listeners. + Global priority session is null + User Records: + Record for full_user=0 + ... + Sessions Stack - have 1 sessions: + Netflix media session com.netflix.ninja/Netflix media session (userId=0) + ownerPid=5833, ownerUid=10080, userId=0 + package=com.netflix.ninja + launchIntent=null + mediaButtonReceiver=null + active=false + flags=3 + rating type=0 + controllers: 0 + state=null + audioAttrs=AudioAttributes: usage=USAGE_MEDIA content=CONTENT_TYPE_UNKNOWN flags=0x800 tags= bundle=null + volumeType=1, controlType=2, max=0, current=0 + metadata: null + queueTitle=null, size=0 + Audio playback (lastly played comes first) + uid=10117 packages=ar.tvplayer.tv + uid=10006 packages=com.google.android.apps.mediashell +``` + +**Findings:** + +1. **No TiviMate session block at all.** TiviMate registers no MediaSession — + it cannot be reached via the session stack. It is a pure Phase-3 fallback app. +2. The only session present is **Netflix's, stale/inactive**: `active=false`, + `state=null`, `metadata=null`. The current parser requires `active=true` + + metadata, so it correctly ignores it — but this exact dump must become a + fixture pinning that behavior ("inactive no-state sessions are ignored"). +3. The before/after-zap captures are **byte-identical** — zapping channels + changes nothing in `dumpsys media_session`. + +### NEW discovery — the "Audio playback" section + +The bottom of `dumpsys media_session` lists **apps that recently played audio**, +even with no MediaSession: + +``` +Audio playback (lastly played comes first) + uid=10117 packages=ar.tvplayer.tv + uid=10006 packages=com.google.android.apps.mediashell +``` + +- TiviMate shows up here while playing live TV. This is a **zero-cost + supplementary signal**: it comes from the *same* `dumpsys media_session` call + the integration already makes — no extra round trip. +- Use: confirm "this foreground app actually has/played media" (foreground = + TiviMate + recent audio playback ⇒ live TV likely playing). Distinguishes + "sitting in TiviMate's settings" from "watching a channel". +- Note `com.google.android.apps.mediashell` (Chromecast receiver process) also + appears — a parse-time ignore-list entry. + +--- + +## R4 — Foreground detection format (Android 11): `mResumedActivity` is the pattern + +``` +$ adb shell dumpsys activity activities | grep -iE 'resumed' + mResumedActivity: ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity + t8152} + state=RESUMED stopped=false delayedResume=false finishing=false + Resumed activities in task display areas (from top to bottom): + Resumed: ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity t8152} + ResumedActivity: ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity t8152} +``` + +``` +$ adb shell dumpsys window | grep -E 'mCurrentFocus|mFocusedApp' + mCurrentFocus=null + mFocusedApp=null + mCurrentFocus=Window{980f3f1 u0 ar.tvplayer.tv/ar.tvplayer.tv.ui.MainActivity} + mFocusedApp=ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity t8152} +``` + +**Findings:** + +1. **No `topResumedActivity` on this firmware** (that pattern is newer Android). + Primary pattern = `mResumedActivity: ActivityRecord{… u0 / …}`. + Keep `topResumedActivity` parsing for future-proofing, checked *after* + `mResumedActivity`. +2. `dumpsys window` contains **multiple** `mCurrentFocus` entries and the first + can be literally `mCurrentFocus=null` — a naive first-match parser breaks. + `mResumedActivity` is the reliable single source; window/focus parsing is a + secondary cross-check at most. +3. Package extraction: from the `u0 /` fragment of the + ActivityRecord line (handles multi-line wraps in the raw dump — match on the + joined block, not per-line). +4. Launcher foreground (for the "off/idle in launcher" distinction): with the + user's custom launcher focused, `mResumedActivity` shows + `com.spocky.projengmenu/.ui.home.MainActivity`. + +--- + +## R5 — Wake state: exact value match confirmed + +``` +Awake: mWakefulness=Awake + mWakefulnessChanging=false +Asleep: mWakefulness=Asleep + mWakefulnessChanging=false +``` + +**Verdict:** the current `"Awake" in result` substring heuristic happens to +work on this firmware, but switch to an exact value parse of the +`mWakefulness=` assignment (robust against future values like +`mWakefulness=Dozing`). + +--- + +## R6 — Recent-tasks: **not run** (optional; low risk) + +Still outstanding if `media_dag_running_apps` needs verification on this +firmware. + +--- + +## R7 — Channel digit keyevents: **skipped by decision** + +The user already uses `media_player.play_media` with +`media_content_type: channel` through the androidtv_remote integration and +reports it "works perfectly fine" — so we **replicate core's exact behavior** +rather than live-testing keyevent variants: + +- digits only (no confirm/ENTER key), inter-digit delay **0.1 s** (core default), +- cancel any prior in-flight channel task before sending, +- keep ENTER as an *optional* boolean (default off) in case a future app needs it. + +R7 stays in the doc as an optional verification step if protocol-path digits +ever misbehave. + +--- + +## R8 — Logcat event reliability: `ActivityTaskManager: START` lines confirmed + +``` +10-08 06:54:33.748 3693 5205 I ActivityTaskManager: START u0 +{act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] +flg=0x10200000 cmp=ar.tvplayer.tv/.ui.MainActivity} from uid 2000 + +10-08 06:54:50.439 3693 5212 I ActivityTaskManager: START u0 +{act=android.intent.action.MAIN +cat=[android.intent.category.LEANBACK_LAUNCHER] flg=0x10220000 +pkg=com.spocky.projengmenu cmp=com.spocky.projengmenu/.ui.home.MainActivity +(has extras)} from uid 10059 +``` + +**Findings:** + +1. `ActivityTaskManager: START u0 {… cmp=/}` lines are emitted on + every app open — including the **home button** (which sends a + `LEANBACK_LAUNCHER` START for the launcher). App-switch detection via logcat + delta snapshots is viable in both directions. +2. `WindowManager` lines were absent from the capture — rely on + `ActivityTaskManager` first. +3. The user has **`be.digitalia.mediasession2mqtt`** installed + ([MediaSession2MQTT](https://github.com/cbeyls/MediaSession2MQTT)) — a + notification-listener service that publishes media sessions over MQTT. It is + part of their current glue (see appendix) and is a candidate to retire once + our session + fallback path covers its sensors. + +--- + +## R9 — PlaybackState layout + a known-good Wholphin session + +``` +state=PlaybackState {state=3, position=17237, buffered position=70070, + speed=1.0, updated=1408392, actions=7339995, custom actions=[], active item + id=0, error=null} + audioAttrs=AudioAttributes: usage=USAGE_MEDIA content=CONTENT_TYPE_MOVIE flags=0x800 tags= bundle=null + volumeType=1, controlType=2, max=0, current=0 + metadata: size=11, description=Twirlywoos, null, null + queueTitle=null, size=1 +``` + +**Findings:** + +1. `PlaybackState` carries everything we planned to parse: `state=3` (playing), + `position=17237` (ms), `speed=1.0`, `updated=1408392` (**uptime ms**, needs + conversion — position math must account for the uptime-vs-wallclock gap; + mirror HA's convention by computing `position + (now - updated_time)` with + uptime anchoring). +2. No duration in PlaybackState — duration lives in metadata keys + (`android.media.metadata.DURATION` inside the size=11 metadata dict, beyond + the `description=` field the parser currently reads). Parsing the full + metadata key list is a Phase-1 stretch goal; `description=` gives + title/artist/album positionally as before. +3. The session above is **Wholphin** (`com.github.damontecres.wholphin`) — a + Jellyfin client that publishes a proper session with usable metadata. Add it + to `JELLYFIN_APP_IDS` (that constant is currently defined but **unused** — + wire it up) so Jellyfin-family clients route to the rich-metadata path. + +--- + +## play_media channel — current working service call (replicate exactly) + +```yaml +action: media_player.play_media +data: + media: + media_content_id: '{{ channel_number|int }}' + media_content_type: channel + metadata: {} +target: + entity_id: + - media_player.shield_android_tv +``` + +Note the current HA service syntax uses the `media:` block; the entity's +`async_play_media(media_type, media_id)` receives `"channel"` / `""` the same +as before. + +--- + +## Appendix — the user's current glue this integration replaces + +Recorded as requirements context. This is what works today and what MVP should +make unnecessary. + +### Template sensor (app name from the androidtv media_player) + +```yaml +template: + - triggers: + - trigger: homeassistant + event: start + - trigger: state + entity_id: + - media_player.android_tv_10_50_204_4 + attribute: app_name + sensor: + - name: Nvidia Shield App Name + unique_id: media_player_nvidia_shield_app_name + icon: "mdi:application" + state: > + {% set adb = "media_player.android_tv_10_50_204_4" %} + {% set adb_state = states(adb) %} + {% if adb_state not in ['unknown', 'unavailable', 'off'] %} + {{ state_attr(adb,'app_name') }} + {% elif adb_state in ['off'] %} + {{ "off" }} + {% else %} + {{ "unknown" }} + {% endif %} +``` + +### Universal media player (state merging + command routing) + +```yaml +media_player: + platform: universal + name: Nvidia Shield + device_class: tv + unique_id: media_player_nvidia_shield + children: + - media_player.android_tv_10_50_204_4 + browse_media_entity: media_player.android_tv_10_50_204_4 + state_template: >- + {% set shield_state = states('media_player.android_tv_10_50_204_4') %} + {% set known_app = states('sensor.mediasession2mqtt_2044_application_id') in ["com.google.android.youtube.tv", "com.github.damontecres.wholphin"] %} + {% set wholphin = is_state('sensor.nvidia_shield_app_name', 'Wholphin') %} + + {% if(shield_state == "unknown" or shield_state == "unavailable" or shield_state == "off") %} + off + {% elif wholphin %} + {% set playing_state = is_state('binary_sensor.jellyfin_nvidia_shield_playing', 'on') %} + {% set paused_state = is_state('binary_sensor.jellyfin_nvidia_shield_paused', 'on') %} + {% if paused_state %} + paused + {% elif playing_state %} + playing + {% else %} + idle + {% endif %} + {% elif known_app %} + {{ states("sensor.mediasession2mqtt_2044_playback_state") }} + {% else %} + {{ shield_state }} + {% endif %} + commands: + turn_on: + action: script.turn_on_nvidia_shield + turn_off: + action: script.turn_off_nvidia_shield + volume_mute: + action: switch.toggle + target: {entity_id: switch.tv_cabinet_livingroom_speaker_system} + volume_up: + action: script.volume_up_nvidia_shield + volume_down: + action: script.volume_down_nvidia_shield + volume_set: + action: script.volume_set_nvidia_shield + data: {level: "{{ volume_level }}"} + select_source: + action: script.set_specific_app_nvidia_shield + data: {app_name: "{{ source }}"} + attributes: + is_volume_muted: switch.inverse_tv_cabinet_speakers + volume_level: media_player.android_tv_10_50_204_4|volume_level + source: remote.shield_android_tv|current_activity + source_list: remote.shield_android_tv|activity_list +``` + +### What the MVP replaces vs. keeps + +| Current glue | MVP after Phase 2/3 | +|---|---| +| `sensor.nvidia_shield_app_name` template sensor | Current App sensor reads StateStore (protocol `current_app` push) | +| `mediasession2mqtt` sensors (`*_application_id`, `*_playback_state`) | ADB media sessions + fallback state (retire the MQTT bridge) | +| Universal player `state_template` app special-casing | Native state ladder | +| `browse_media_entity` from androidtv_remote | Phase 4 BrowseMedia app directory | +| `volume_*` scripts (speaker switch + receiver interplay) | **Stays as user automations** — but `volume_set` gains a real protocol path (Phase 2 VOLUME_SET); the receiver/speaker switch logic remains the user's or moves to media zone | +| `remote.shield_android_tv` for source/activity | Our own remote entity (Phase 2) | + +--- + +## Fixtures to save into `tests/fixtures/` + +| File | Content | +|---|---| +| `dumpsys_media_session_tivimate_playing.txt` | R2 capture 1 (Netflix stale session + audio playback listing) | +| `dumpsys_media_session_tivimate_zapped.txt` | R2 capture 2 (identical — pins zap-invariance) | +| `dumpsys_media_session_wholphin_playing.txt` | R9 capture (playing session with position/metadata) | +| `dumpsys_activity_activities_tivimate.txt` | R4 capture 1 | +| `dumpsys_activity_activities_launcher.txt` | R4 capture 2 (custom launcher) | +| `dumpsys_window_focused.txt` | R4 window capture | +| `dumpsys_power_wakefulness_{awake,asleep}.txt` | R5 captures | +| `logcat_activitytaskman_start.txt` | R8 capture | + +--- + +## Plan deltas applied to `MVP_PLAN.md` + +1. TiviMate package corrected everywhere: `ar.tvplayer.tv`. +2. R2 verdict resolved: no session → TiviMate is the headline Phase-3 fallback + app; README gets it as a known case. +3. **New fallback signal adopted:** "Audio playback (lastly played comes first)" + section of `dumpsys media_session` — zero extra round trip; parsed alongside + sessions; `com.google.android.apps.mediashell` in the ignore list. +4. Foreground parser: `mResumedActivity` primary (Android 11), + `topResumedActivity` newer-fallback; `mCurrentFocus=null` trap documented; + launcher special-casing not stock. +5. Wake parse: exact `mWakefulness=` value match. +6. `play_media(channel)`: replicate core behavior exactly (digits, 0.1 s, no + ENTER); R7 marked skipped-by-decision. +7. Wholphin (`com.github.damontecres.wholphin`) added to `JELLYFIN_APP_IDS`; + wiring that constant becomes a Phase-0/1 item. +8. Position math: `updated` is uptime-ms — needs uptime anchoring in Phase 1. +9. Migration note: retire mediasession2mqtt sensors + universal-player YAML once + Phase 3 lands (documented in README). +10. Netflix stale-session fixture pins "inactive no-state sessions ignored".