feat(image): opt-in screen capture as an image entity, binary-safe over ADB
The user asked for "an image entity of the device screen capture? Can that be added". It is added as `image.<entry>_screen_capture`, and the three boundaries are the feature: - it NEVER drives state — the state is the capture timestamp, and the capture path reads the ADB poller and nothing else (pinned by a manager that raises on any other attribute, and by a real-HA run whose recorded state is the timestamp with no bytes anywhere in the attributes); - it is OPT-IN and OFF by default (`screen_capture_enabled`, Main setup row) — with the option absent `async_setup_entry` adds no entity at all; - it NEVER polls — HA's own `should_poll = False` for image entities is kept, so only an image fetch or `homeassistant.update_entity` captures; concurrent fetches share one shell and a capture is reused for 5 s. BINARY-SAFE CAPTURE: `screencap -p | base64` over the existing shell, decoded locally with `base64.b64decode` and gated on the PNG signature AND the IHDR chunk. `AdbDevice.shell()` returns `str`, so raw PNG bytes cannot travel through it; the alternative (a device temp file + the library's file-read path) needs the raw AdbDevice and writes to the Shield's shared storage, where a crash between write and delete leaves a screenshot behind. The base64 route leaves no device-side artifact — nothing to clean up — at the cost of ~33% more bytes on the wire. HONEST FAILURES: ADB off / ADB down / a capture that returned no PNG / a transport that raised all make the entity UNAVAILABLE with `capture_status` (`adb_disabled` / `adb_disconnected` / `capture_failed`) — never a blank or zero-byte image. `image_last_updated` moves only on success. Tests: 25 new cases (a real 2x1 PNG fixture on the wire — bytes, never a str; every non-PNG answer rejected; disconnected/failed/recovery paths; capture reuse, coalescing and the expiry window; opt-in default OFF and on) plus 4 config-flow cases (schema default False, off stays off, on is stored, and it can be turned back off). The stub harness gains the image platform and `util.dt` faces, the real-HA entity matrix gains this class, and the drift sweep's coverage test demanded exactly that. README documents the cost, the opt-in and the boundary.
This commit is contained in:
1 parent
a1ce28996e
commit
f29f62ddae
13 files changed
+954
-3
No files matched your search
@@ -171,6 +171,52 @@ documented, opt-in escape hatch for debugging and automation, registered at
|
||||
setup and unregistered when the last participating entity unloads. It
|
||||
rejects (logs a warning, sends nothing) when the ADB connection is down.
|
||||
|
||||
### Screen capture (opt-in, manual)
|
||||
|
||||
The integration can expose an `image` entity showing the Shield's current screen —
|
||||
`sensor`-free and **off by default**. Turn it on in **Configure → Main setup →
|
||||
"Add a screen capture image entity"**; the entity appears as
|
||||
`image.<entry>_screen_capture` on the existing device.
|
||||
|
||||
Three things about it are deliberate, and they are the whole design:
|
||||
|
||||
- **It never drives state.** An `image` entity's state is the timestamp of its last
|
||||
successful capture. Nothing about a capture writes to the media player, the
|
||||
sensors, the power switch or the manager's store — the capture path reads the ADB
|
||||
poller and nothing else. Your media player keeps saying what the DEVICE reported,
|
||||
not what a screenshot happened to show.
|
||||
- **It never polls.** HA sets `should_poll = False` for image entities and this one
|
||||
keeps it: no scan interval, no timer. A capture happens only when something asks
|
||||
for the bytes (the dashboard showing the entity picture) or when you call
|
||||
`homeassistant.update_entity` on it. Back-to-back requests are coalesced into one
|
||||
capture and a successful capture is reused for 5 seconds, so a dashboard repaint
|
||||
cannot hammer the Shield.
|
||||
- **What it costs.** Every fresh capture is a full-resolution screen grab
|
||||
(`screencap -p`) of the current display, base64-encoded, pulled over the same ADB
|
||||
socket the state polling uses, and it keeps that socket busy for the duration. On
|
||||
a 1080p UI that is typically a few hundred kilobytes of PNG (roughly a third more
|
||||
on the wire because of the base64). Treat it as a diagnostic surface you look at
|
||||
when something looks wrong, not as a camera.
|
||||
|
||||
It is unavailable — never a blank image — whenever ADB is off, ADB is down, or the
|
||||
last capture failed; the `capture_status` attribute says which (`ok`,
|
||||
`adb_disabled`, `adb_disconnected`, `capture_failed`). An explicit
|
||||
`homeassistant.update_entity` retries after a failure.
|
||||
|
||||
**How it captures, and why that matters.** The transport is `screencap -p |
|
||||
base64` over the existing shell, decoded locally with `base64.b64decode` and
|
||||
checked for the PNG signature and the `IHDR` chunk before anything is handed to HA.
|
||||
The alternative — writing a temp PNG on the device and pulling it as a file — needs
|
||||
the raw ADB device object and writes to the Shield's shared storage, where a crash
|
||||
between write and delete leaves a screenshot behind; the base64 route leaves no
|
||||
device-side artifact at all. The cost is the extra ~33% on the wire, and one thing
|
||||
is **inferred rather than measured**: no Shield was available in the development
|
||||
environment, so how the persistent socket copes with a few hundred kilobytes in one
|
||||
`shell()` call has not been observed on hardware. The decode, the failure paths and
|
||||
the entity contract are covered by tests, including a real-HA run that drives HA's
|
||||
own `ImageEntity` and content-type validator.
|
||||
|
||||
|
||||
## Replacing androidtv / androidtv_remote + the glue
|
||||
|
||||
This fork replaces the old `androidtv` media player, the `androidtv_remote`
|
||||
@@ -310,7 +356,7 @@ and only **Save and close** commits:
|
||||
|
||||
| Row | What it holds |
|
||||
| --- | --- |
|
||||
| **Main setup** | Shield address, ADB port, `Read state over ADB`, `Add the Nvidia Remote Protocol`, `Show title and artist`, `Allow the adb_command action`, Jellyfin player, power switch |
|
||||
| **Main setup** | Shield address, ADB port, `Read state over ADB`, `Add the Nvidia Remote Protocol`, `Show title and artist`, `Allow the adb_command action`, `Add a screen capture image entity` (off by default — see [Screen capture](#screen-capture-opt-in-manual)), Jellyfin player, power switch |
|
||||
| **Receiver** | the receiver/soundbar entity and the input it uses |
|
||||
| **Apps** | the apps editor — add, edit or remove one app, or paste the whole list (see below) |
|
||||
| **Scripts** | one optional script per command, plus the remote-entity script, the channel digit delay and send-ENTER |
|
||||
@@ -638,6 +684,7 @@ exactly as before.
|
||||
| Current app (`sensor.current_app`) | `sensor` | Currently running app |
|
||||
| Now playing (`sensor.now_playing`) | `sensor` | Formatted now-playing string, with its own flat attributes (below) |
|
||||
| Pair ADB | `button` | Initiate or re-pair ADB connection |
|
||||
| Screen capture (`image.screen_capture`) | `image` | The Shield's current screen — **opt-in, off by default**, manual/diagnostic only (see [Screen capture](#screen-capture-opt-in-manual)) |
|
||||
|
||||
### Now-playing attributes
|
||||
|
||||
|
||||
Reference in new issue
Block a user