feat(image): opt-in screen capture as an image entity, binary-safe over ADB

The user asked for "an image entity of the device screen capture? Can that be
added". It is added as `image.<entry>_screen_capture`, and the three boundaries
are the feature:

- it NEVER drives state — the state is the capture timestamp, and the capture
  path reads the ADB poller and nothing else (pinned by a manager that raises on
  any other attribute, and by a real-HA run whose recorded state is the
  timestamp with no bytes anywhere in the attributes);
- it is OPT-IN and OFF by default (`screen_capture_enabled`, Main setup row) —
  with the option absent `async_setup_entry` adds no entity at all;
- it NEVER polls — HA's own `should_poll = False` for image entities is kept, so
  only an image fetch or `homeassistant.update_entity` captures; concurrent
  fetches share one shell and a capture is reused for 5 s.

BINARY-SAFE CAPTURE: `screencap -p | base64` over the existing shell, decoded
locally with `base64.b64decode` and gated on the PNG signature AND the IHDR
chunk. `AdbDevice.shell()` returns `str`, so raw PNG bytes cannot travel through
it; the alternative (a device temp file + the library's file-read path) needs the
raw AdbDevice and writes to the Shield's shared storage, where a crash between
write and delete leaves a screenshot behind. The base64 route leaves no
device-side artifact — nothing to clean up — at the cost of ~33% more bytes on
the wire.

HONEST FAILURES: ADB off / ADB down / a capture that returned no PNG / a
transport that raised all make the entity UNAVAILABLE with `capture_status`
(`adb_disabled` / `adb_disconnected` / `capture_failed`) — never a blank or
zero-byte image. `image_last_updated` moves only on success.

Tests: 25 new cases (a real 2x1 PNG fixture on the wire — bytes, never a str;
every non-PNG answer rejected; disconnected/failed/recovery paths; capture reuse,
coalescing and the expiry window; opt-in default OFF and on) plus 4 config-flow
cases (schema default False, off stays off, on is stored, and it can be turned
back off). The stub harness gains the image platform and `util.dt` faces, the
real-HA entity matrix gains this class, and the drift sweep's coverage test
demanded exactly that. README documents the cost, the opt-in and the boundary.
This commit is contained in:
James Mackay committed 2026-10-12 10:28:32 +01:00
1 parent a1ce28996e
commit f29f62ddae
13 files changed
+954 -3

No files matched your search

+48 -1
View File
@@ -171,6 +171,52 @@ documented, opt-in escape hatch for debugging and automation, registered at
setup and unregistered when the last participating entity unloads. It
rejects (logs a warning, sends nothing) when the ADB connection is down.
### Screen capture (opt-in, manual)
The integration can expose an `image` entity showing the Shield's current screen —
`sensor`-free and **off by default**. Turn it on in **Configure → Main setup →
"Add a screen capture image entity"**; the entity appears as
`image.<entry>_screen_capture` on the existing device.
Three things about it are deliberate, and they are the whole design:
- **It never drives state.** An `image` entity's state is the timestamp of its last
successful capture. Nothing about a capture writes to the media player, the
sensors, the power switch or the manager's store — the capture path reads the ADB
poller and nothing else. Your media player keeps saying what the DEVICE reported,
not what a screenshot happened to show.
- **It never polls.** HA sets `should_poll = False` for image entities and this one
keeps it: no scan interval, no timer. A capture happens only when something asks
for the bytes (the dashboard showing the entity picture) or when you call
`homeassistant.update_entity` on it. Back-to-back requests are coalesced into one
capture and a successful capture is reused for 5 seconds, so a dashboard repaint
cannot hammer the Shield.
- **What it costs.** Every fresh capture is a full-resolution screen grab
(`screencap -p`) of the current display, base64-encoded, pulled over the same ADB
socket the state polling uses, and it keeps that socket busy for the duration. On
a 1080p UI that is typically a few hundred kilobytes of PNG (roughly a third more
on the wire because of the base64). Treat it as a diagnostic surface you look at
when something looks wrong, not as a camera.
It is unavailable — never a blank image — whenever ADB is off, ADB is down, or the
last capture failed; the `capture_status` attribute says which (`ok`,
`adb_disabled`, `adb_disconnected`, `capture_failed`). An explicit
`homeassistant.update_entity` retries after a failure.
**How it captures, and why that matters.** The transport is `screencap -p |
base64` over the existing shell, decoded locally with `base64.b64decode` and
checked for the PNG signature and the `IHDR` chunk before anything is handed to HA.
The alternative — writing a temp PNG on the device and pulling it as a file — needs
the raw ADB device object and writes to the Shield's shared storage, where a crash
between write and delete leaves a screenshot behind; the base64 route leaves no
device-side artifact at all. The cost is the extra ~33% on the wire, and one thing
is **inferred rather than measured**: no Shield was available in the development
environment, so how the persistent socket copes with a few hundred kilobytes in one
`shell()` call has not been observed on hardware. The decode, the failure paths and
the entity contract are covered by tests, including a real-HA run that drives HA's
own `ImageEntity` and content-type validator.
## Replacing androidtv / androidtv_remote + the glue
This fork replaces the old `androidtv` media player, the `androidtv_remote`
@@ -310,7 +356,7 @@ and only **Save and close** commits:
| Row | What it holds |
| --- | --- |
| **Main setup** | Shield address, ADB port, `Read state over ADB`, `Add the Nvidia Remote Protocol`, `Show title and artist`, `Allow the adb_command action`, Jellyfin player, power switch |
| **Main setup** | Shield address, ADB port, `Read state over ADB`, `Add the Nvidia Remote Protocol`, `Show title and artist`, `Allow the adb_command action`, `Add a screen capture image entity` (off by default — see [Screen capture](#screen-capture-opt-in-manual)), Jellyfin player, power switch |
| **Receiver** | the receiver/soundbar entity and the input it uses |
| **Apps** | the apps editor — add, edit or remove one app, or paste the whole list (see below) |
| **Scripts** | one optional script per command, plus the remote-entity script, the channel digit delay and send-ENTER |
@@ -638,6 +684,7 @@ exactly as before.
| Current app (`sensor.current_app`) | `sensor` | Currently running app |
| Now playing (`sensor.now_playing`) | `sensor` | Formatted now-playing string, with its own flat attributes (below) |
| Pair ADB | `button` | Initiate or re-pair ADB connection |
| Screen capture (`image.screen_capture`) | `image` | The Shield's current screen — **opt-in, off by default**, manual/diagnostic only (see [Screen capture](#screen-capture-opt-in-manual)) |
### Now-playing attributes