init

README.md

@@ -0,0 +1,372 @@
+# dsh-recall
+
+Local-embedding code recall for **DeepSeek Harness**. One tool, `recall`, that
+answers "where is this handled?" with ranked **excerpts** — file path, line range,
+and a bounded slice of the code — instead of you reading whole files into the
+conversation.
+
+## Why this exists
+
+A long agent session does not get expensive because the model reasons. It gets
+expensive because **every step re-sends the accumulated transcript**, and in a
+coding session most of that transcript is file contents the agent has already
+read.
+
+Measured on one real DSH install over 32 days, from the `dsh-cost-meter` ledger:
+
+| | |
+|---|---|
+| total spend | **$160.27** (32 days — $5.01/day, ~$150/month) |
+| model calls | 39,804 |
+| prompt cache hit rate | **99.18%** — caching was already saturated |
+| cost from **re-reading cached context** | **65.1%** |
+| cost from new input | 16.9% |
+| cost from output (60% of it reasoning) | 18.0% |
+| **share of context that is message history** | **96.0%** (system 0.8%, tools 3.2%) |
+| average context re-read **per call** | **332,000 tokens** |
+
+That reframes the problem. Caching had nothing left to give, and trimming the
+system prompt or tool definitions would have chased 4% of the context. The money
+was in **how much history each step carried**. A tool that adds a 40-line excerpt
+instead of a 900-line file does not just save those tokens once — it saves them
+on *every subsequent step of the session*, because the context it never added is
+never re-sent.
+
+## What it does, measured
+
+Run against `ha-runtime` (684 indexable files) with the default 8-result limit:
+
+```
+query     : how does the occupancy state machine decide a room is occupied
+returned  : 13,075 chars   ~3,269 tokens
+source    : 78,669 chars  ~19,667 tokens  across the 4 files the results came from
+ratio     : 6.0x smaller
+```
+
+All eight results landed inside `src/controllers/room/occupancy/state_machine/`
+— the correct module — best cosine similarity **0.61**. Scoring a warm index takes
+about a millisecond; the only per-query network cost is one embedding call.
+
+## Install
+
+```sh
+dsh plugin --profile web add /path/to/dsh-recall
+```
+
+Then point it at your embedding endpoint. Anything Ollama-compatible works, and
+the endpoint must be stated because `127.0.0.1:11434` is a convention rather than
+a fact:
+
+```yaml
+# your profile's cordis.patch.yml
+- id: dsh-recall
+  name: dsh-recall
+  config:
+    embeddingApiBase: http://10.50.1.2:11434
+```
+
+Or edit every setting from the UI. The plugin ships **two halves**, and the settings
+page draws a card only when it has both:
+
+- the **host half** (`index.js`) serves a `dsh-recall` settings namespace;
+- the **browser half** (`client/client.js`) claims that same namespace with a
+  collapsible card under **Settings → Plugins → Plugin configuration**.
+
+A served namespace with no card renders nothing at all, and a card whose namespace
+is not served is never dispatched — so both are required. A restart is needed for a
+newly installed browser half to appear, because client bundle bytes are snapshotted
+when the profile activates.
+
+Every value is also checkable from the tool, without touching a file:
+
+```
+recall  action=configure  config={"embeddingApiBase":"http://10.50.1.2:11434"}
+```
+
+The API key is declared `role('secret')` on the host, so it is stripped from every
+settings response — the card can show that a key is set without ever receiving it,
+and leaving it blank on save means "unchanged" rather than "clear".
+
+Pull the model once:
+
+```sh
+ollama pull unclemusclez/jina-embeddings-v2-base-code
+```
+
+## The tool
+
+`recall` — three actions plus configuration.
+
+| action | what it does |
+|---|---|
+| `search` *(default)* | answers `query`, warming a bounded slice of the index first |
+| `index` | builds or refreshes the whole index; **background job by default** |
+| `status` | index and endpoint state, touching nothing |
+| `configure` | writes settings through the settings namespace |
+
+Parameters: `query`, `action`, `limit`, `path` (restrict to a subdirectory),
+`run_in_background`.
+
+### What it shows you
+
+Two surfaces, both reading a host projection rather than polling:
+
+- a **chip in the session header** — `Recall 3 · 40× · 457 KB kept out`. No price
+  on it, deliberately: at the measured rate a whole conversation's avoided
+  context is a fraction of a penny, and a chip advertising that reads as the tool
+  being useless rather than as the arithmetic being honest. The tooltip carries
+  the full comparison.
+- a **right-sidebar Recall tab** — the same figures with room for what they are
+  not: *"An estimate, and an upper bound: it assumes a whole file would have been
+  read each time. Token counts are bytes ÷ 4."*
+
+Both render nothing when there is nothing to say — a conversation that never used
+recall costs no layout at all, which matters for the header corner because the
+framework lays it out only while its occupant renders something.
+
+The data path is `presentationMeta` → the `tool/result` event's `meta` → a
+projection unit (`recallUsage`) with a wire view the browser reads through
+`useProjection`. No new event type, no side channel, and nothing leaves the
+machine.
+
+The tool description steers the model on when *not* to use it: prefer `grep` when
+you already know the symbol, `read` when you need the whole file, and do not
+follow up with a `read` of a range you were just handed.
+
+## Focus: letting a conversation see less
+
+A workspace-wide index has to rank across everything, and in a codebase built
+from many near-identical modules that is where embedding similarity fails. On the
+repository this was written against there are **18 controllers** — same idioms,
+same naming, same shape — across 228k lines, and a query about one returns
+plausible fragments of all of them.
+
+Narrowing the index to one of them removes the confusability rather than ranking
+through it:
+
+| | |
+|---|---|
+| median controller scope | **114 chunks, 2.4% of the workspace** |
+| candidate pool | 4,738 → 114 chunks, a **41.6× smaller haystack** |
+| cold start (projected from measured throughput) | **3.1 min → 5s** |
+
+Set it per call, or configure a default:
+
+```
+recall  action=index  focus=src/controllers/room/occupancy
+```
+
+```yaml
+focus: src/controllers/room/occupancy   # or leave empty to resolve by marker
+focusMarkers: [CONTROLLER.md]           # files that make a directory its own scope
+```
+
+Resolution is most-explicit-first: an explicit `focus`, then the nearest
+enclosing directory carrying a marker file, then the workspace. The last fallback
+is the pre-focus behaviour, so adopting this cannot regress a deployment that
+never configures it. **`recall status` lists every scope it can find**, so
+choosing one is a matter of reading the list.
+
+Two things worth knowing, both measured:
+
+- **Scoping hides shared code.** 41.6% of that workspace — 1,972 chunks of shared
+  types, services and infrastructure — sits outside every controller. A weak
+  result inside a scope says so and tells you to retry without it, because a
+  scope is a window, not a relevance filter.
+- **Choosing a scope is explicit, never inferred.** A session sitting at the
+  repository root is not thereby *about* any one controller, and silently
+  confining it to whichever scope sorted first would hide code it needs. So
+  `status` lists them and a caller names one.
+
+## Two opt-in local-model steps
+
+Both are **off by default**, and both are built so that any failure is a
+fallback rather than an error:
+
+| setting | what it does | default |
+|---|---|---|
+| `expansionEnabled` | asks the local model for alternative phrasings, ranks each, and fuses the orders | `false` |
+| `rerankEnabled` | has the local model score the top candidates against the query | `false` |
+
+They are off because **their benefit has not been measured against a real
+model**, and a feature that spends latency on an unquantified gain is not a
+default worth having. Turn them on and run `scripts/baseline.mjs` if you want to
+find out.
+
+### Why the failures matter more than the features
+
+Both run on the critical path of a search, against a model on another box that
+can be slow, wedged, or absent. That is not hypothetical here — the endpoint
+this was developed against spent the whole project in exactly that state. So the
+contracts are narrow:
+
+- **expansion always returns the original query**, first, whatever happens. The
+  worst case is a search with no expansion, never a failed search.
+- **re-ranking returns `undefined` for "no opinion"**, which is deliberately not
+  the same as zero. Replacing a good embedding score with a zero because the
+  model timed out would turn a working search into an empty one.
+- both use their **own deadline** inside the caller's signal, so a slow local
+  model gives up and lets the search continue: cancelling mean stop, being slow
+  does not.
+- a non-finite timeout is **clamped, not propagated** — `AbortSignal.timeout(NaN)`
+  throws, and the catch below it would report that as "the model had nothing to
+  say". A misconfiguration must not wear the disguise of a slow model.
+
+Tested, including the wedged case: a socket that opens and never answers must
+give up on its own deadline rather than pass the wait on.
+
+## Measuring it
+
+`scripts/baseline.mjs` is the before/after instrument, and it reads two stores
+because they answer different questions:
+
+```sh
+node scripts/baseline.mjs          # human report
+node scripts/baseline.mjs --json   # machine readable
+```
+
+- the **cost ledger** knows requests, cache-read tokens, and therefore what each
+  request carried;
+- the **session projection cache** knows the surface each session reached, and
+  whether it used recall at all.
+
+```
+BASELINE
+  sessions                      109
+  requests                      39,706
+  avg context/request (pooled)  339,182    every request counts once
+  avg context/request (typical) 155,175    every session counts once
+  cost                          $160.11
+```
+
+**Both averages are reported on purpose.** They differ by a factor of two here,
+because the largest sessions dominate the pooled figure — quoting only one would
+mislead in whichever direction it happened to favour. Which to use depends on
+the question: pooled is what maps to money, typical is what a new conversation
+should expect.
+
+Sessions that used recall form the "after" group; everything else is the
+baseline. While no session has used it, the report says so rather than printing
+a comparison of nothing against nothing.
+
+## How it works
+
+**Chunking is structure-aware.** Splitting on a fixed line count produces units
+that begin mid-function; their embedding sits between two topics and matches
+neither. Chunks are cut at column-0 declarations instead, neighbouring small
+declarations are merged, and a chunk that absorbs several of them is labelled with
+all of their names — because for a small file that label is the module's only
+description in the index. Minified and generated files are detected and skipped
+rather than indexed as noise.
+
+**Ranking is hybrid, fused by rank.** Pure vector search is weak on code: an
+embedding of `getUserById` sits near every other lookup in the repo. Lexical
+scoring is weak in the opposite direction. The two are combined by reciprocal
+rank fusion rather than by adding normalised scores — min-max normalising a
+cluster of unrelated similarities stretches noise into a confident-looking
+ranking, and rank fusion cannot make that mistake.
+
+**The index is incremental and lives outside your repository.** Files are keyed by
+content hash; only changed files are re-embedded. The SQLite index sits in
+`$DSH_HOME/dsh-recall/workspaces/<key>/` — never in the repo, because for a
+300k-line project it is tens of megabytes. Vectors are stored unit-length, so
+scoring is a dot product.
+
+**Cold start is bounded, and honest.** A search reads and hashes everything
+(fast), then embeds only what fits its time budget, and reports exactly what is
+still pending. `apply` never blocks on embedding a repository. In the measurement
+above, the first search spent its 90s budget on 309 of 684 files (2,257 chunks);
+a full `index` finished the remainder in about 75s more.
+
+**Changing the model or the vector width discards the index** rather than silently
+mixing two incompatible vector spaces into one ranking.
+
+## Honest limitations
+
+- **The 6.0x figure is a comparison, not a guarantee.** It measures what the tool
+  returned against the whole files those results came from. An unusually careful
+  agent might have read only one of those files, so treat it as an upper bound on
+  the saving rather than a promise.
+- **Token figures are `characters / 4`.** An approximation, not a measurement.
+- **Excerpts are capped at 40 lines by default**, so most results in the
+  measurement above were truncated. The excerpt identifies the location; a
+  *targeted* follow-up `read` of that range is still sometimes the right move. The
+  saving is in not reading the other 90% of each file.
+- **A locally built index costs minutes of CPU** on a large repository. It costs
+  no API money, and only once.
+- **Retrieval quality is the model's quality.** A different embedding model will
+  rank differently. `scripts/smoke.mjs` is the check that the top hit is actually
+  correct.
+- **A file edited since it was embedded is flagged `[stale]`**, not silently
+  trusted.
+
+## Operational notes
+
+- The **only outbound request** this plugin makes is to the embedding endpoint you
+  configure. No source code goes anywhere else, and there is no telemetry.
+- The endpoint receives your source code. Point it at infrastructure you trust —
+  that is the whole reason it is explicit configuration rather than a default.
+- The workspace walk **never follows a symlink**, so a link out of the repository
+  cannot cause the machine to be indexed.
+- The index is a cache. Deleting `$DSH_HOME/dsh-recall/` costs one re-embed and
+  nothing else.
+
+### Give the endpoint room
+
+A cold index is a sustained CPU load — hundreds of batched requests. The defaults
+(`embeddingBatchSize: 16`, `embeddingConcurrency: 2`) are deliberately
+conservative, and they were chosen the hard way.
+
+**What happened during development.** An index run at concurrency 4 against a
+small Ollama host that was *also* holding a generation model resident
+(`expires_at` set to the year 2319 — pinned forever) left the host unable to load
+the embedding model. It did not error: it accepted connections and never
+answered. `ollama ps` afterwards showed only the pinned model. That needs a
+restart of the Ollama service on that host; nothing this plugin can do reaches it.
+
+Three things now guard against it:
+
+- **Lower concurrency and batch size by default.** If you raise them, raise them
+  on hardware you know.
+- **`embeddingKeepAlive`** (default `5m`) keeps the embedding model resident for
+  the length of a run instead of letting it be evicted and reloaded between
+  requests.
+- **A short timeout, and a probe.** A wedged endpoint does not refuse a
+  connection, so a long timeout turns a broken backend into a tool call that
+  appears to hang. `recall status` issues a single no-retry probe with a 10-second
+  ceiling and reports whether the endpoint **answered** — which is a different
+  claim from "is configured", and the one worth having.
+
+If `status` reports `endpoint did NOT answer`, check `ollama ps` on that host
+before assuming the plugin is at fault.
+
+## Development
+
+```sh
+npm test              # 91 tests, no network required
+node scripts/smoke.mjs [workspace] [query]     # live retrieval check
+node scripts/measure.mjs [workspace] [query] [budgetMs]
+```
+
+The `lib/` tests run anywhere. `test/config.test.js` and `test/plugin.test.js`
+import `index.js`, which imports `@deepseek-ai/dsh-tools` and
+`@deepseek-ai/schemastery` — resolvable once the plugin is installed into a
+profile (or via a local `node_modules` link to the profile's copies, which the
+`.gitignore` excludes).
+
+`test/plugin.test.js` validates every tool result against **the tool's own
+declared output schema**, which is `additionalProperties: false`. That is not
+pedantry: the host enforces the schema before the model sees a result, so an
+internal field leaking into the returned object is a failed call. It caught
+exactly that during development.
+
+`test/config.test.js` covers the load-time contract. Cordis rejects a `Config`
+before `apply` ever runs, so a broken schema cannot be reported from inside the
+plugin — the profile simply refuses to boot. Every other test in the suite calls
+`apply` directly and would not notice.
+
+`test/client.test.js` runs the browser half in Node by stubbing `window` and
+`require('react')`. That is worth more than it sounds: the two failure modes most
+likely to reach you silently are structural — a root element that is not an `<li>`,
+and a card that does not start collapsed — and both are invisible until someone
+opens Settings. It also pins that an untouched API key is never written or unset.