25 KiB
25 KiB
Research ideas for ha-nvidia-shield (features, patterns, TiviMate)
Status: COMPLETE. Companion docs: MVP_PLAN.md, RESEARCH_SHIELD.md.
Method: primary sources first — library/integration source code (androidtvremote2,
adb_shell, HA core), HA core GitHub issues/PRs, home-assistant.io integration docs,
home-assistant.io action docs, and community threads. Citations are inline markdown
links. Note: HA's Action classes (MediaPlayerEntityFeature.*) are what we set in
code; HA's Actions (media_player.play_media etc.) are what users call.
1. What users actually request (from HA core issues, community threads, library issues)
| # | Request | Source | Take for our MVP |
|---|---|---|---|
| U1 | Absolute volume set on Android TV players — media_player.volume_set repeatedly requested; HA core androidtv_remote closed it not_planned |
core#176505 | High demand, but the protocol v2 proto has no usable volume-set primitive shipped (it reports volume; see C-VOL). Our Phase 2 now ships `VOLUME_STEP |
| U2 | Launching any installed app (not just deep-linkable ones) and retrieving the installed-app list | androidtvremote2#17 | The official NVIDIA Shield app does retrieve the app list over another service; androidtvremote2 accepted market://launch?id=<pkg> as the app-launch workaround (now built into send_launch_app_command, see C-LAUNCH). App-list retrieval is not implementable through this client's protocol surface — a data-driven, options-defined app map is the right MVP substitute (already in plan) |
| U3 | Text/IME entry (typing into search boxes); behavior depends on the app and on whether the on-screen keyboard is open (one-at-a-time chars get replaced in Spotify; whole-string send works) | androidtvremote2#32 | Library supports send_text but behavior is app-dependent. Keep the IME option default off (as planned) and document the qualifier rather than trying to be clever |
| U4 | Voice search / voice commands | androidtvremote2#28 — closed; the library now ships start_voice()/VoiceStream, gated by enable_voice and device feature support |
Implemented in the lib (PCM 16-bit mono 8 kHz streamed as RemoteVoiceBegin/Payload/End). Post-MVP only; note exists so Phase 4 does not re-research |
| U5 | Deep-link launching of apps into specific sections (community guide table with ~60 known app URLs, plus how to discover one via adb shell cmd package dump PACKAGE and manifest string resolvers) |
deep-link guide | Directly validates Phase-4 deep links; send_launch_app_command accepts either a deep link or a bare package id (see C-LAUNCH). The guide's discovery recipe (cmd package dump <pkg>, grep <data, resolve @string/deeplink_schema) is worth copying into our docs/contributor workflow |
| U6 | BrowseMedia containing installed apps / activity list so users can launch apps from the media panel instead of scripts | core#117126 (androidtv_remote: "Implement Android TV Remote browse media with apps and activity list") | Validates Phase-4 BrowseMedia app directory; can be built from our own running_apps data or an options-defined list (no protocol app-list needed) |
| U7 | Channel-by-number play_media + channel up/down keys on streaming players (Roku, Bravia) | Roku docs, Bravia docs | play_media(media_content_type: channel) is the canonical way every TV integration does tuning (Roku, Bravia); our §7 digit path matches. Bonus Roku/Bravia idea: expose channel_up/channel_down as remote commands (protocol KEYCODE_CHANNEL_UP=166 / KEYCODE_CHANNEL_DOWN=167 — in the proto, no new work) |
| U8 | TiviMate: start it and switch to a specific channel from an HA automation | community thread (2025, active) | Exactly our §7 play_media(channel) requirement. The working recipe there: remote.turn_on with activity: ar.tvplayer.tv → delay ~7 s (app cold-start) → media_player.play_media with media_content_type: channel / media_content_id: "1" — note the wait; our play_media should not promise cold-start-to-channel in one shot (we don't own app launch speed). Follow-up request in the thread shows demand for tuning VOD/series pages too (unresolved anywhere) |
| U9 | Shield app-launcher cards for wall-panel remotes (Firemote adds TiviMate/SmartTube shortcuts); Harmony-style activity switching | HA-Firemote#48, Shield launcher blueprint | QoL signal for Phase-4 apps editor + source_list; also shows app-name→icon mapping is what users iterate on (supports data-driven app map + icon support) |
| U10 | Firemote / universal-remote-card users must hand-define every app shortcut because the Shield integration exposes no app directory | universal-remote-card discussion | Independent confirmation of U6/U9 demand; TiviMate + SmartTube shortcut request |
Also verified from an independent direction: MediaSession2MQTT#4 confirms TiviMate "doesn't capture anything" for notification-based playback detection on the Shield — consistent with our R2 no-media-session finding, and a reminder that projects tried and failed to get sessions out of TiviMate (see §5).
2. androidtvremote2 capabilities we are not using (verified from library source)
Source of truth read: androidtv_remote.py,
remotemessage.proto,
TvKeys.txt.
| Capability | API | MVP relevance |
|---|---|---|
| Key codes + length variants | send_key_command(key, direction) — SHORT/START_LONG/END_LONG; accepts int or "KEYCODE_..."/"POWER" strings; any code from the 300-strong proto enum |
Remote entity port (Phase 2) matches core's SHORT/START_LONG/END_LONG design |
| Text / IME | send_text(text); also send_key_command("text:<message>") routes into send_text |
Phase-2 IME option (default off). Known quirk: append-vs-replace inconsistency (#32) — document, don't ship cleverness |
| App launch | send_launch_app_command(app_link_or_app_id) — auto-prefixes market://launch?id= when the value has no URL scheme (C-LAUNCH) |
play_media(url)/play_media(app) (Phase 4): bare package names work with zero extra code; deep links like plex:// pass through unchanged |
| Push callbacks | add_is_on_updated_callback, add_current_app_updated_callback, add_volume_info_updated_callback, add_is_available_updated_callback |
Exactly the manager-listener pattern Phase 1/2 needs; current_app push is the Phase-3 primary foreground signal |
| Volume info (read-only) | volume_info → (level, max, muted) pushed on change |
Real mute state + numeric level readout in Phase 2 — but no volume_set; see C-VOL |
| Reconnect + reauth distinction | keep_reconnecting(invalid_auth_callback=<fn>) with internal exponential backoff 0.1 s → 30 s cap; raises InvalidAuth when certs are rejected (device hard reset) |
Feeds the transport design; one owner per transport — don't build a competing watchdog for the protocol socket; InvalidAuth → reauth flow, not retry storm |
| Device info | device_info (model, vendor, package_name, app_version) |
Free diagnostics data for Phase 4 (and a nice setup sanity check) |
| App-list push | Not implemented and not in the protocol this client speaks (#17). Verified against the full proto: the only app-ish messages are RemoteAppInfo (IME context) and RemoteAppLinkLaunchRequest — no app-list leafs |
Confirms: options-defined app name/icon map is the honest substitute; do not promise app-list discovery |
| IME sensitivity | Constructor enable_ime=True — "Needed for getting current_app; disable for devices that show 'Use keyboard on mobile device screen'" |
If we make IME configurable, document that disabling it can break the current_app push (which Phase 3 depends on) |
| Voice | start_voice() → VoiceStream (PCM 16-bit mono 8 kHz, 20 KB chunks), enable_voice=True |
Post-MVP |
RemoteSetVolumeLevel exists in the proto but is sent by noone (see C-VOL) |
— | Do not plan on it for VOLUME_SET |
3. Patterns worth borrowing from other HA integrations (docs verified)
All four integration docs read in full (Roku, Apple TV, Bravia, androidtv).
| Integration | Mechanism | What to borrow | Effort for us |
|---|---|---|---|
| Roku | play_media(media_content_type: channel) → tune by number (e.g. 5.1); select_source accepts app name or app id; BrowseMedia for apps; channel tuning for Roku TV devices; media_content_type: app with extra.content_id/extra.media_type for content deeplinks; remote commands incl. channel_up/channel_down; active app + active app id sensors; media_content_type: url + extra{format,name,thumbnail,artist_name} for PlayOnRoku direct playback; obtains app list via a published XML endpoint (/query/apps) |
Channel tuning semantic — exactly our channel port (validated). App deep link envelope shape: media_content_type: app with extra.content_id/extra.media_type is a richer pattern than raw URLs if we later want per-app content launches — record it but don't build it for MVP. Source list built from apps matches our select_source = launch app. Channel up/down as remote commands — bolt channel_up/channel_down onto Phase-2 remote command set (keycodes 166/167, free) |
Low–medium |
| Apple TV | play_media(media_content_type: url) with youtube://..., https://www.netflix.com/title/... "Deep Links" (canonical how-to + link-discovery tips via iOS share sheets); select_source + BrowseMedia "Apps" folder for launching apps; keyboard entity (binary_sensor "keyboard focused") + three text actions (set/append/clear_keyboard_text) driven by keyboard-focus push; remote entity with num_repeats/delay_secs/hold_secs |
play_media(url) deep-link semantics for Phase 4, direct port (mediatype url → send_launch_app_command). "Apps folder" validation for BrowseMedia. Keyboard-focus push → typed-text entry is Apple TV's answer to #32's append/replace ambiguity — a keyboard entity would need an IME-push equivalent the Shield protocol does not give us cheaply, so for MVP ship send_text only when IME is on (nothing fancier). Remote entity parameters (num_repeats/delay_secs/hold_secs) are the shape we already planned |
Medium (only if we chase keyboard); Low for the rest |
| Sony Bravia | play_media matches channel number, exact or partial app/channel name, or URI string; BrowseMedia lists installed apps + TV channels; remote commands incl. Num0..Num9 (digit-typing built into the remote platform); documented real-world limitation that TVs running Android/Google TV lose in-app playback info unless you pair with Google Cast (they recommend Universal Media Player with two children — exactly the problem our all-in-one design solves natively) |
Fuzzy matching on media_content_id (number OR exact/partial name) is a small and very forgiving UX win for channel — accept a numeric string strictly, but also accept exact channel/app names from an options-defined map if cheap. The Bravia doc explicitly documents the Cast+Universal-Media-Player workaround — citable pattern that our native ladder replaces for Shield users |
Low (for the simple variant) |
| androidtv (core, ADB/Fire TV) | select_source = launch app; ! prefix closes app (!com.netflix.ninja); options editors for apps list, exclude unknown, arbitrary ADB turn_on/turn_off commands, per-app state_detection_rules; screen-capture album art option; adb_command/upload/download actions; learn_sendevent helper |
! close-app prefix is a tiny, delightful, zero-new-dep feature — implement for free alongside our Phase-4 launch path; document it as an androidtv compatibility nicety. Arbitrary turn_on/turn_off commands as options is worth matching (we already have shell; gives users a knob without code). state_detection_rules (per-app playing/paused conditions over media_session_state, audio_state, wake_lock_size) is the blueprint for our last-known cache tier to become configurable if users need app-specific truth — out of ADB/MVP scope but record it. Screen-capture album art is the precedent for the screencap idea in §4 (Roku/Bravia do not need it; we do) |
Low |
Canonical patterns (all four):
select_source = launch app with source_list of apps; BrowseMedia with a flat
"Apps" directory; play_media channel digits or named matches; per-app
icon/name map; documented media_content_type vocabulary (channel, app, url,
music, video) — we should mirror this vocabulary exactly in Phase 2/4.
4. adb_shell capabilities we are not using (verified from adb_shell source)
Source of truth read: adb_shell/adb_device.py (the full file) and README.rst.
| Capability | API | MVP relevance |
|---|---|---|
| Streaming shell | AdbDevice.streaming_shell(command, decode=True) — a generator that yields lines as they arrive, until the connection is closed |
⚠️ Plan correction. MVP_PLAN.md §2 currently says "adb_shell's shell() is request→full-response; there is no incremental/streaming read API". That is no longer accurate on master — streaming_shell exists as a documented generator on AdbDevice. Two caveats before acting: (a) verify it is present in the resolved version we pin (repo last pushed 2024‑10; check release ≥ ours covers it — the version is old enough that the feature set may differ slightly); (b) verify the async device class exposing it (the source read was the sync AdbDevice; the async subclass typically mirrors). If both check out, MVP_PLAN's "Phase-4 optional: logcat streaming behind an option" becomes much cheaper than hacking adb_shell internals — and the deferred logcat snapshot approach gains a real path. Rank as the single highest-value adb_shell finding |
| FileSync | push(), pull(device_path, local_path, ...), list(device_path) — full FileSync protocol |
Enables clean pull-based screencap -p /local && pull work flows (crisper than shell screencap -p stdout parsing) for album art, debugging, or future thumbnails; zero new deps |
| Lazy shell decode | decode=False on shell |
Micro-win for binary/inspection commands |
| USB | AdbDeviceUsb (experimental, [usb] extra) |
Not for MVP (cradle-cable setup); note for future reference only |
| Long-lived TCP connect | AdbDeviceTcp(host, port, default_transport_timeout_s=...) + connect(rsa_keys=[signer], auth_timeout_s=...) with per-call timeouts |
What Phase 1 already relies on; the per-transaction timeout fields (transport_timeout_s/read_timeout_s/timeout_s) are worth being explicit about in the manager's health-check design rather than default-None everywhere |
5. TiviMate specifics
What TiviMate gives us: essentially nothing at the device level.
- Package
ar.tvplayer.tv; no media session (R2, verified twice — before/after a zap produced byte-identical output) and appears only in the "Audio playback" section ofdumpsys media_session. - MediaSession2MQTT#4: independent confirmation that TiviMate produces nothing captureable by notification/session bridges on the Shield.
- Channel-switch automation thread: the community's only working channel-select recipe is the digit-keyevent path we already planned (plus a ~7 s delay for app cold start).
- No deep link published by users in the deep-linking guide or elsewhere for
ar.tvplayer.tv— treat as "no shortcut"; if ever one surfaces,am startwith extras via ADB is the Phase-4 "apps the protocol can't launch" seam. - Channel list export: TiviMate does not expose a device-side API for the channel list the community can use — the ecosystem gets channels from the upstream M3U/XMLTV sources that TiviMate loads, not from the app. Community projects around TiviMate (playlists, EPG (EPGTalk), playlist auto-updaters) take their data from those links. MVP implication: if we ever want "channel N = " metadata, the data side belongs to the user's playlist/EPG pipeline (e.g. options field to store a channel list file), not to the integration scraping the TV.
- Channel/zap-change detection while watching: no project has reliable on-device
telemetry (the R2 capture shows the dump is zap-invariant). Realistic best
signals remain the ones we have chosen: A "Audio playback" presence (identity)
and
last_channelcache (what we last asked for). The Phase-1 optional logcat-on-a-device process channel may eventually reveal app-internal logs, but no public, stable, IPTV-app-level pattern exists to borrow.
Conclusion: our current design (digits via keys, foreground fallback + audio-playback hint + last-known cache) is at parity with what the community does; the only additions worth recording are the ~7 s app-cold-start note in automation docs and the "channel list data belongs outside the integration" decision.
6. Ranked ideas
Ranking by: (a) fits locked MVP decisions, (b) cost, (c) user-verifiable payoff. "Phase" is where it should land in the current plan.
| Rank | Idea | MVP fit | Effort | Phase |
|---|---|---|---|---|
| 1 | channel_up / channel_down on the remote+media_player command sets — free with the protocol's KEYCODE_CHANNEL_UP(166)/KEYCODE_CHANNEL_DOWN(167); Roku/Bravia precedent for making tuning feel native; zero new logic since we already send keycodes for digits |
High | Trivial (hours) | 2 (fall-out) |
| 2 | ! app-close prefix on select_source/play_media(app) — androidtv precedent (!com.netflix.ninja); one-stop fix for "app won't die" automation scripts; zero new deps or surfaces |
High | Small (half day incl. tests) | 4 |
| 3 | play_media(media_content_type: url) deep links via send_launch_app_command with a cloud-less data-driven deeplink map option (validated against the community-wide guide; the library accepts either raw deep link or bare package id) — plus the guide's adb shell cmd package dump trick documented for contributors to discover links per app |
High | Small–medium (~1 day incl. options editor) | 4 |
| 4 | BrowseMedia "Apps" directory from running_apps + options-defined app map (androidtv_remote/Apple TV/Bravia precedent) — the single most-requested QoL pattern in this space and the reason users hit external cards (#117126, HA-Firemote#48) | High | Medium (~1–2 days incl. icons + tests) | 4 |
| 5 | Volume readout hardening in Phase 2 — volume_info push sourced real volume_level and mute, with media_player.volume_set deliberately unsupported (C-VOL correction, already adopted); if we hear repeated user demand later, offer an ADB-step-loop helper service documented as such (not fake features) |
Medium (already handled) | done (plan note only) | 2 |
| 6 | adb_shell streaming_shell probe — spike: confirm streaming_shell exists at our pinned adb-shell version and on the async class; if yes, MVP_PLAN's deferred logcat-on-ADB idea is feasible via a documented generator rather than internal hacks; if no, keeps the snapshot-only plan honest. Cheap spike, high leverage on a currently-"impossible" feature |
Medium | Small (half-day spike) | 4 (optional path) |
| 7 | Bravia-style tolerant media_content_id matching for channel (accept 5.1-style floats and exact-name matches from the app/channel map) — small UX win, safe with a strict numeric path retained |
Medium | Small | 3/4 |
| 8 | Screen-capture album art (adb_shell screencap/pull) — androidtv precedent supplies the pattern; useful only for our PlayOnRoku-missing, Jellyfin-cached apps; keep it as a documented option like androidtv does, since it costs extra ADB shell round trips when playing and is privacy-scope-sensitive |
Medium | Medium (~1 day + options slosh) | Post-MVP |
| 9 | IME text entry surfaced as a remote/service integration — possible now that the library has send_text/text: syntax, but per #32 behavior is inconsistent; Apple TV solves this with a keyboard-focus push that the Shield protocol doesn't give us cheaply; keep IME an options toggle, not an entity |
Low for MVP | Medium | 4 (option only) |
| 10 | state_detection_rules-style per-app configurable state ladder (androidtv pattern) — only if/when users ask for app-specific truth beyond ADB sessions (e.g. IPTV VOD pages); nice future seam; don't build pre-emptively |
Low for MVP | Medium–large | Post-MVP |
| 11 | Voice (start_voice) — available in the library, enable_voice=True and device-feature-gated; not in our MVP scope, recorded so it's not re-researched |
Low | Medium | Post-MVP |
Key references for the rankings (primary, not forum threads only): core#176505 · #117126 · #17 · #32 · #28 · androidtvremote2 source · proto · adb_shell device · Roku · Apple TV · Bravia · androidtv · deep linking guide · community TiviMate thread · Firemote#48 · MediaSession2MQTT#4
7. Explicit MVP-plan corrections proposed (for Phase-3/4 lands)
- C-VOL (already adopted by captain).
volume_setis not protocol-native — the supported features in Phase 2 stayVOLUME_STEP|MUTE, with real numericvolume_levelandis_volume_mutedreadouts from the protocol'svolume_infopush. Absolute-volume demand, if it ever recurs, is addressed honestly via an ADBinput keyevent KEYCODE_VOLUME_*helper script, documented as such, not as a fake supported feature. - C-STREAM (new). MVP_PLAN §2's statement that adb_shell offers no
incremental/streaming read should be softened:
streaming_shellexists upstream (at least on master) and might make the deferred Phase-4 logcat idea a one-day spike instead of "needs new dependency". Verify our pinned version carries it and whether the async device class exposes it, then either upgrade to a "cheap Phase-4 item" or keep the deferred note with accurate wording. - C-LAUNCH (new, informational).
send_launch_app_commandalready accepts a bare package id by auto-prefixingmarket://launch?id=; Phase 4 needs no extra branch for "app" launches, only for deep links (where the value already has a scheme) and!-close via ADB (only when the protocol path is not enough). - C-TIVI (new). TiviMate has no usable on-device channel metadata or export
surface; the honest MVP display is "on, in TiviMate" +
last_channelcache from our own play_media calls. Channel-name metadata is a user-supplied playlist/EPG concern, out of scope (and out of ADB's reach).