ha-nvidia-shield/docs/RESEARCH_SHIELD.md

# Shield firmware research results (§8 of MVP_PLAN.md, R1–R9)

Captured from the user's Shield. These outputs are the ground truth for parser
fixtures and resolved the R2 verdict. Plan deltas applied to `MVP_PLAN.md` are
listed at the end.

## Environment

| Property | Value |
|---|---|
| Device | NVIDIA Shield (board `mdarcy`) |
| Android | 11 (`ro.build.version.release=11`) |
| Fingerprint | `NVIDIA/mdarcy/mdarcy:11/RQ1A.210105.003/7825230_4231.0265:user/release-keys` |
| SDK | 30 (Android 11; the `getprop ro.build.version.sdk` call failed on shell truncation — `etprop: inaccessible or not found` — but Android 11 ⇒ SDK 30) |

Note: the home launcher is **not** the stock Shield launcher — it is
`com.spocky.projengmenu` (Project Eternity menu). Foreground detection must not
assume `com.google.android.tv.launcher`.

---

## R1 — TiviMate package name: `ar.tvplayer.tv`

```
$ adb shell pm list packages | grep -iE 'tivi|^package:iv'
package:ar.tvplayer.tv
```

**Verdict:** TiviMate's package is `ar.tvplayer.tv` — **not** `iv.android.tv` as
the plan assumed. All match keys, fixtures, and constants use `ar.tvplayer.tv`.

---

## R2 — Does TiviMate publish a media session? **NO** (verdict: branch 3)

Full `dumpsys media_session` with TiviMate in the foreground playing a live
channel (captured twice, before/after zapping):

```
MEDIA SESSION SERVICE (dumpsys media_session)

 9 sessions listeners.
 Global priority session is null
 User Records:
 Record for full_user=0
   ...
   Sessions Stack - have 1 sessions:
     Netflix media session com.netflix.ninja/Netflix media session (userId=0)
       ownerPid=5833, ownerUid=10080, userId=0
       package=com.netflix.ninja
       launchIntent=null
       mediaButtonReceiver=null
       active=false
       flags=3
       rating type=0
       controllers: 0
       state=null
       audioAttrs=AudioAttributes: usage=USAGE_MEDIA content=CONTENT_TYPE_UNKNOWN flags=0x800 tags= bundle=null
       volumeType=1, controlType=2, max=0, current=0
       metadata: null
       queueTitle=null, size=0
 Audio playback (lastly played comes first)
   uid=10117 packages=ar.tvplayer.tv
   uid=10006 packages=com.google.android.apps.mediashell
```

**Findings:**

1. **No TiviMate session block at all.** TiviMate registers no MediaSession —
   it cannot be reached via the session stack. It is a pure Phase-3 fallback app.
2. The only session present is **Netflix's, stale/inactive**: `active=false`,
   `state=null`, `metadata=null`. The current parser requires `active=true` +
   metadata, so it correctly ignores it — but this exact dump must become a
   fixture pinning that behavior ("inactive no-state sessions are ignored").
3. The before/after-zap captures are **byte-identical** — zapping channels
   changes nothing in `dumpsys media_session`.

### NEW discovery — the "Audio playback" section

The bottom of `dumpsys media_session` lists **apps that recently played audio**,
even with no MediaSession:

```
Audio playback (lastly played comes first)
  uid=10117 packages=ar.tvplayer.tv
  uid=10006 packages=com.google.android.apps.mediashell
```

- TiviMate shows up here while playing live TV. This is a **zero-cost
  supplementary signal**: it comes from the *same* `dumpsys media_session` call
  the integration already makes — no extra round trip.
- Use: confirm "this foreground app actually has/played media" (foreground =
  TiviMate + recent audio playback ⇒ live TV likely playing). Distinguishes
  "sitting in TiviMate's settings" from "watching a channel".
- Note `com.google.android.apps.mediashell` (Chromecast receiver process) also
  appears — a parse-time ignore-list entry.

---

## R4 — Foreground detection format (Android 11): `mResumedActivity` is the pattern

```
$ adb shell dumpsys activity activities | grep -iE 'resumed'
    mResumedActivity: ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity
    t8152}
              state=RESUMED stopped=false delayedResume=false finishing=false
      Resumed activities in task display areas (from top to bottom):
        Resumed: ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity t8152}
      ResumedActivity: ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity t8152}
```

```
$ adb shell dumpsys window | grep -E 'mCurrentFocus|mFocusedApp'
    mCurrentFocus=null
      mFocusedApp=null
      mCurrentFocus=Window{980f3f1 u0 ar.tvplayer.tv/ar.tvplayer.tv.ui.MainActivity}
      mFocusedApp=ActivityRecord{4f0d030 u0 ar.tvplayer.tv/.ui.MainActivity t8152}
```

**Findings:**

1. **No `topResumedActivity` on this firmware** (that pattern is newer Android).
   Primary pattern = `mResumedActivity: ActivityRecord{… u0 <pkg>/<activity> …}`.
   Keep `topResumedActivity` parsing for future-proofing, checked *after*
   `mResumedActivity`.
2. `dumpsys window` contains **multiple** `mCurrentFocus` entries and the first
   can be literally `mCurrentFocus=null` — a naive first-match parser breaks.
   `mResumedActivity` is the reliable single source; window/focus parsing is a
   secondary cross-check at most.
3. Package extraction: from the `u0 <pkg>/<activity>` fragment of the
   ActivityRecord line (handles multi-line wraps in the raw dump — match on the
   joined block, not per-line).
4. Launcher foreground (for the "off/idle in launcher" distinction): with the
   user's custom launcher focused, `mResumedActivity` shows
   `com.spocky.projengmenu/.ui.home.MainActivity`.

---

## R5 — Wake state: exact value match confirmed

```
Awake:   mWakefulness=Awake
         mWakefulnessChanging=false
Asleep:  mWakefulness=Asleep
         mWakefulnessChanging=false
```

**Verdict:** the current `"Awake" in result` substring heuristic happens to
work on this firmware, but switch to an exact value parse of the
`mWakefulness=` assignment (robust against future values like
`mWakefulness=Dozing`).

---

## R6 — Recent-tasks: **not run** (optional; low risk)

Still outstanding if `media_dag_running_apps` needs verification on this
firmware.

---

## R7 — Channel digit keyevents: **skipped by decision**

The user already uses `media_player.play_media` with
`media_content_type: channel` through the androidtv_remote integration and
reports it "works perfectly fine" — so we **replicate core's exact behavior**
rather than live-testing keyevent variants:

- digits only (no confirm/ENTER key), inter-digit delay **0.1 s** (core default),
- cancel any prior in-flight channel task before sending,
- keep ENTER as an *optional* boolean (default off) in case a future app needs it.

R7 stays in the doc as an optional verification step if protocol-path digits
ever misbehave.

---

## R8 — Logcat event reliability: `ActivityTaskManager: START` lines confirmed

```
10-08 06:54:33.748  3693  5205 I ActivityTaskManager: START u0
{act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER]
flg=0x10200000 cmp=ar.tvplayer.tv/.ui.MainActivity} from uid 2000

10-08 06:54:50.439  3693  5212 I ActivityTaskManager: START u0
{act=android.intent.action.MAIN
cat=[android.intent.category.LEANBACK_LAUNCHER] flg=0x10220000
pkg=com.spocky.projengmenu cmp=com.spocky.projengmenu/.ui.home.MainActivity
(has extras)} from uid 10059
```

**Findings:**

1. `ActivityTaskManager: START u0 {… cmp=<pkg>/<activity>}` lines are emitted on
   every app open — including the **home button** (which sends a
   `LEANBACK_LAUNCHER` START for the launcher). App-switch detection via logcat
   delta snapshots is viable in both directions.
2. `WindowManager` lines were absent from the capture — rely on
   `ActivityTaskManager` first.
3. The user has **`be.digitalia.mediasession2mqtt`** installed
   ([MediaSession2MQTT](https://github.com/cbeyls/MediaSession2MQTT)) — a
   notification-listener service that publishes media sessions over MQTT. It is
   part of their current glue (see appendix) and is a candidate to retire once
   our session + fallback path covers its sensors.

---

## R9 — PlaybackState layout + a known-good Wholphin session

```
state=PlaybackState {state=3, position=17237, buffered position=70070,
    speed=1.0, updated=1408392, actions=7339995, custom actions=[], active item
    id=0, error=null}
      audioAttrs=AudioAttributes: usage=USAGE_MEDIA content=CONTENT_TYPE_MOVIE flags=0x800 tags= bundle=null
      volumeType=1, controlType=2, max=0, current=0
      metadata: size=11, description=Twirlywoos, null, null
      queueTitle=null, size=1
```

**Findings:**

1. `PlaybackState` carries everything we planned to parse: `state=3` (playing),
   `position=17237` (ms), `speed=1.0`, `updated=1408392` (**uptime ms**, needs
   conversion — position math must account for the uptime-vs-wallclock gap;
   mirror HA's convention by computing `position + (now - updated_time)` with
   uptime anchoring).
2. No duration in PlaybackState — duration lives in metadata keys
   (`android.media.metadata.DURATION` inside the size=11 metadata dict, beyond
   the `description=` field the parser currently reads). Parsing the full
   metadata key list is a Phase-1 stretch goal; `description=` gives
   title/artist/album positionally as before.
3. The session above is **Wholphin** (`com.github.damontecres.wholphin`) — a
   Jellyfin client that publishes a proper session with usable metadata. Add it
   to `JELLYFIN_APP_IDS` (that constant is currently defined but **unused** —
   wire it up) so Jellyfin-family clients route to the rich-metadata path.

---

## play_media channel — current working service call (replicate exactly)

```yaml
action: media_player.play_media
data:
  media:
    media_content_id: '{{ channel_number|int }}'
    media_content_type: channel
    metadata: {}
target:
  entity_id:
    - media_player.shield_android_tv
```

Note the current HA service syntax uses the `media:` block; the entity's
`async_play_media(media_type, media_id)` receives `"channel"` / `"<n>"` the same
as before.

---

## Appendix — the user's current glue this integration replaces

Recorded as requirements context. This is what works today and what MVP should
make unnecessary.

### Template sensor (app name from the androidtv media_player)

```yaml
template:
  - triggers:
    - trigger: homeassistant
      event: start
    - trigger: state
      entity_id:
        - media_player.android_tv_10_50_204_4
      attribute: app_name
    sensor:
      - name: Nvidia Shield App Name
        unique_id: media_player_nvidia_shield_app_name
        icon: "mdi:application"
        state: >
          {% set adb = "media_player.android_tv_10_50_204_4" %}
          {% set adb_state = states(adb) %}
          {% if adb_state not in ['unknown', 'unavailable', 'off'] %}
            {{ state_attr(adb,'app_name') }}
          {% elif adb_state in ['off'] %}
            {{ "off" }}
          {% else %}
            {{ "unknown" }}
          {% endif %}
```

### Universal media player (state merging + command routing)

```yaml
media_player:
  platform: universal
  name: Nvidia Shield
  device_class: tv
  unique_id: media_player_nvidia_shield
  children:
    - media_player.android_tv_10_50_204_4
  browse_media_entity: media_player.android_tv_10_50_204_4
  state_template: >-
    {% set shield_state = states('media_player.android_tv_10_50_204_4') %}
    {% set known_app = states('sensor.mediasession2mqtt_2044_application_id') in ["com.google.android.youtube.tv", "com.github.damontecres.wholphin"] %}
    {% set wholphin = is_state('sensor.nvidia_shield_app_name', 'Wholphin') %}

    {% if(shield_state == "unknown" or shield_state == "unavailable" or shield_state == "off") %}
      off
    {% elif wholphin %}
      {% set playing_state = is_state('binary_sensor.jellyfin_nvidia_shield_playing', 'on') %}
      {% set paused_state = is_state('binary_sensor.jellyfin_nvidia_shield_paused', 'on') %}
      {% if paused_state %}
        paused
      {% elif playing_state %}
        playing
      {% else %}
        idle
      {% endif %}
    {% elif known_app %}
      {{ states("sensor.mediasession2mqtt_2044_playback_state") }}
    {% else %}
      {{ shield_state }}
    {% endif %}
  commands:
    turn_on:
      action: script.turn_on_nvidia_shield
    turn_off:
      action: script.turn_off_nvidia_shield
    volume_mute:
      action: switch.toggle
      target: {entity_id: switch.tv_cabinet_livingroom_speaker_system}
    volume_up:
      action: script.volume_up_nvidia_shield
    volume_down:
      action: script.volume_down_nvidia_shield
    volume_set:
      action: script.volume_set_nvidia_shield
      data: {level: "{{ volume_level }}"}
    select_source:
      action: script.set_specific_app_nvidia_shield
      data: {app_name: "{{ source }}"}
  attributes:
    is_volume_muted: switch.inverse_tv_cabinet_speakers
    volume_level: media_player.android_tv_10_50_204_4|volume_level
    source: remote.shield_android_tv|current_activity
    source_list: remote.shield_android_tv|activity_list
```

### What the MVP replaces vs. keeps

| Current glue | MVP after Phase 2/3 |
|---|---|
| `sensor.nvidia_shield_app_name` template sensor | Current App sensor reads StateStore (protocol `current_app` push) |
| `mediasession2mqtt` sensors (`*_application_id`, `*_playback_state`) | ADB media sessions + fallback state (retire the MQTT bridge) |
| Universal player `state_template` app special-casing | Native state ladder |
| `browse_media_entity` from androidtv_remote | Phase 4 BrowseMedia app directory |
| `volume_*` scripts (speaker switch + receiver interplay) | **Stays as user automations** — but `volume_set` gains a real protocol path (Phase 2 VOLUME_SET); the receiver/speaker switch logic remains the user's or moves to media zone |
| `remote.shield_android_tv` for source/activity | Our own remote entity (Phase 2) |

---

## Fixtures to save into `tests/fixtures/`

| File | Content |
|---|---|
| `dumpsys_media_session_tivimate_playing.txt` | R2 capture 1 (Netflix stale session + audio playback listing) |
| `dumpsys_media_session_tivimate_zapped.txt` | R2 capture 2 (identical — pins zap-invariance) |
| `dumpsys_media_session_wholphin_playing.txt` | R9 capture (playing session with position/metadata) |
| `dumpsys_activity_activities_tivimate.txt` | R4 capture 1 |
| `dumpsys_activity_activities_launcher.txt` | R4 capture 2 (custom launcher) |
| `dumpsys_window_focused.txt` | R4 window capture |
| `dumpsys_power_wakefulness_{awake,asleep}.txt` | R5 captures |
| `logcat_activitytaskman_start.txt` | R8 capture |

---

## Plan deltas applied to `MVP_PLAN.md`

1. TiviMate package corrected everywhere: `ar.tvplayer.tv`.
2. R2 verdict resolved: no session → TiviMate is the headline Phase-3 fallback
   app; README gets it as a known case.
3. **New fallback signal adopted:** "Audio playback (lastly played comes first)"
   section of `dumpsys media_session` — zero extra round trip; parsed alongside
   sessions; `com.google.android.apps.mediashell` in the ignore list.
4. Foreground parser: `mResumedActivity` primary (Android 11),
   `topResumedActivity` newer-fallback; `mCurrentFocus=null` trap documented;
   launcher special-casing not stock.
5. Wake parse: exact `mWakefulness=` value match.
6. `play_media(channel)`: replicate core behavior exactly (digits, 0.1 s, no
   ENTER); R7 marked skipped-by-decision.
7. Wholphin (`com.github.damontecres.wholphin`) added to `JELLYFIN_APP_IDS`;
   wiring that constant becomes a Phase-0/1 item.
8. Position math: `updated` is uptime-ms — needs uptime anchoring in Phase 1.
9. Migration note: retire mediasession2mqtt sensors + universal-player YAML once
   Phase 3 lands (documented in README).
10. Netflix stale-session fixture pins "inactive no-state sessions ignored".

---

## Appendix — mediasession2mqtt retirement + universal-player glue (Phase 3)

The fork retires the surrounding glue once the state ladder (README, Phase 3)
covers what the external pieces provided:

| External piece | What it provided | Now native (file) |
|---|---|---|
| `mediasession2mqtt` bridge sensors | per-app now-playing pushed over MQTT from the Shield's notification listener | ADB media-session polling + the foreground tier (`connection.py`, `media_player.py`); the reading is published as separate top-level `now_playing_*` attributes, and `now_playing_via` names the tier (`jellyfin` \| `media_session` \| `app`) |
| Universal media player `media_dag_running_apps` template | app switcher list with playback states | `running_apps_from_tasks(..., last_known=...)` (`ha_adb_common/running_apps.py`) — enriched with `foreground`/`state`/`last_known_state`/`last_known_title` |
| Universal player "currently in app X" template | fallback state for no-session apps | the foreground ladder tier (`media_player.py` state property, tier 3.5) |
| External androidtv_remote entity for commands | remote keys | the protocol-backed `remote` entity (Phase 2) — the external entity remains only as the transitional last command tier, removed in Phase 4 |
| Last-channel / "recently played" sensors | what was on when the session vanished | the TTL-capped last-known-state cache + `_last_channel` + the audio-playback hint (`connection.py`) |

The migration path: point the automations/scripts at the integration's DAG
attributes (`media_dag_*`) instead of the MQTT sensors, then remove
`mediasession2mqtt` and the universal-player config entries. No YAML glue is
needed for the MVP feature set; the appendix exists so retirement can be
staged per household.

**TiviMate (`ar.tvplayer.tv`)** is the documented no-session case (R2): it
publishes no MediaSession on Shield Android 11. Its visibility comes from the
foreground tier (tier 3.5) with the audio-playback hint and the
last-known-state cache — never from a session, never resurrected past the
state TTL.