commit 0964819cdc3140465c8b5a5c724fdff104c42bb0 Author: NotExpectedYet Date: Sat Sep 26 05:35:29 2026 +0100 init diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..f7dabe1 --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +node_modules/ +*.sqlite +*.sqlite-wal +*.sqlite-shm diff --git a/README.md b/README.md new file mode 100644 index 0000000..92ecb1d --- /dev/null +++ b/README.md @@ -0,0 +1,372 @@ +# dsh-recall + +Local-embedding code recall for **DeepSeek Harness**. One tool, `recall`, that +answers "where is this handled?" with ranked **excerpts** — file path, line range, +and a bounded slice of the code — instead of you reading whole files into the +conversation. + +## Why this exists + +A long agent session does not get expensive because the model reasons. It gets +expensive because **every step re-sends the accumulated transcript**, and in a +coding session most of that transcript is file contents the agent has already +read. + +Measured on one real DSH install over 32 days, from the `dsh-cost-meter` ledger: + +| | | +|---|---| +| total spend | **$160.27** (32 days — $5.01/day, ~$150/month) | +| model calls | 39,804 | +| prompt cache hit rate | **99.18%** — caching was already saturated | +| cost from **re-reading cached context** | **65.1%** | +| cost from new input | 16.9% | +| cost from output (60% of it reasoning) | 18.0% | +| **share of context that is message history** | **96.0%** (system 0.8%, tools 3.2%) | +| average context re-read **per call** | **332,000 tokens** | + +That reframes the problem. Caching had nothing left to give, and trimming the +system prompt or tool definitions would have chased 4% of the context. The money +was in **how much history each step carried**. A tool that adds a 40-line excerpt +instead of a 900-line file does not just save those tokens once — it saves them +on *every subsequent step of the session*, because the context it never added is +never re-sent. + +## What it does, measured + +Run against `ha-runtime` (684 indexable files) with the default 8-result limit: + +``` +query : how does the occupancy state machine decide a room is occupied +returned : 13,075 chars ~3,269 tokens +source : 78,669 chars ~19,667 tokens across the 4 files the results came from +ratio : 6.0x smaller +``` + +All eight results landed inside `src/controllers/room/occupancy/state_machine/` +— the correct module — best cosine similarity **0.61**. Scoring a warm index takes +about a millisecond; the only per-query network cost is one embedding call. + +## Install + +```sh +dsh plugin --profile web add /path/to/dsh-recall +``` + +Then point it at your embedding endpoint. Anything Ollama-compatible works, and +the endpoint must be stated because `127.0.0.1:11434` is a convention rather than +a fact: + +```yaml +# your profile's cordis.patch.yml +- id: dsh-recall + name: dsh-recall + config: + embeddingApiBase: http://10.50.1.2:11434 +``` + +Or edit every setting from the UI. The plugin ships **two halves**, and the settings +page draws a card only when it has both: + +- the **host half** (`index.js`) serves a `dsh-recall` settings namespace; +- the **browser half** (`client/client.js`) claims that same namespace with a + collapsible card under **Settings → Plugins → Plugin configuration**. + +A served namespace with no card renders nothing at all, and a card whose namespace +is not served is never dispatched — so both are required. A restart is needed for a +newly installed browser half to appear, because client bundle bytes are snapshotted +when the profile activates. + +Every value is also checkable from the tool, without touching a file: + +``` +recall action=configure config={"embeddingApiBase":"http://10.50.1.2:11434"} +``` + +The API key is declared `role('secret')` on the host, so it is stripped from every +settings response — the card can show that a key is set without ever receiving it, +and leaving it blank on save means "unchanged" rather than "clear". + +Pull the model once: + +```sh +ollama pull unclemusclez/jina-embeddings-v2-base-code +``` + +## The tool + +`recall` — three actions plus configuration. + +| action | what it does | +|---|---| +| `search` *(default)* | answers `query`, warming a bounded slice of the index first | +| `index` | builds or refreshes the whole index; **background job by default** | +| `status` | index and endpoint state, touching nothing | +| `configure` | writes settings through the settings namespace | + +Parameters: `query`, `action`, `limit`, `path` (restrict to a subdirectory), +`run_in_background`. + +### What it shows you + +Two surfaces, both reading a host projection rather than polling: + +- a **chip in the session header** — `Recall 3 · 40× · 457 KB kept out`. No price + on it, deliberately: at the measured rate a whole conversation's avoided + context is a fraction of a penny, and a chip advertising that reads as the tool + being useless rather than as the arithmetic being honest. The tooltip carries + the full comparison. +- a **right-sidebar Recall tab** — the same figures with room for what they are + not: *"An estimate, and an upper bound: it assumes a whole file would have been + read each time. Token counts are bytes ÷ 4."* + +Both render nothing when there is nothing to say — a conversation that never used +recall costs no layout at all, which matters for the header corner because the +framework lays it out only while its occupant renders something. + +The data path is `presentationMeta` → the `tool/result` event's `meta` → a +projection unit (`recallUsage`) with a wire view the browser reads through +`useProjection`. No new event type, no side channel, and nothing leaves the +machine. + +The tool description steers the model on when *not* to use it: prefer `grep` when +you already know the symbol, `read` when you need the whole file, and do not +follow up with a `read` of a range you were just handed. + +## Focus: letting a conversation see less + +A workspace-wide index has to rank across everything, and in a codebase built +from many near-identical modules that is where embedding similarity fails. On the +repository this was written against there are **18 controllers** — same idioms, +same naming, same shape — across 228k lines, and a query about one returns +plausible fragments of all of them. + +Narrowing the index to one of them removes the confusability rather than ranking +through it: + +| | | +|---|---| +| median controller scope | **114 chunks, 2.4% of the workspace** | +| candidate pool | 4,738 → 114 chunks, a **41.6× smaller haystack** | +| cold start (projected from measured throughput) | **3.1 min → 5s** | + +Set it per call, or configure a default: + +``` +recall action=index focus=src/controllers/room/occupancy +``` + +```yaml +focus: src/controllers/room/occupancy # or leave empty to resolve by marker +focusMarkers: [CONTROLLER.md] # files that make a directory its own scope +``` + +Resolution is most-explicit-first: an explicit `focus`, then the nearest +enclosing directory carrying a marker file, then the workspace. The last fallback +is the pre-focus behaviour, so adopting this cannot regress a deployment that +never configures it. **`recall status` lists every scope it can find**, so +choosing one is a matter of reading the list. + +Two things worth knowing, both measured: + +- **Scoping hides shared code.** 41.6% of that workspace — 1,972 chunks of shared + types, services and infrastructure — sits outside every controller. A weak + result inside a scope says so and tells you to retry without it, because a + scope is a window, not a relevance filter. +- **Choosing a scope is explicit, never inferred.** A session sitting at the + repository root is not thereby *about* any one controller, and silently + confining it to whichever scope sorted first would hide code it needs. So + `status` lists them and a caller names one. + +## Two opt-in local-model steps + +Both are **off by default**, and both are built so that any failure is a +fallback rather than an error: + +| setting | what it does | default | +|---|---|---| +| `expansionEnabled` | asks the local model for alternative phrasings, ranks each, and fuses the orders | `false` | +| `rerankEnabled` | has the local model score the top candidates against the query | `false` | + +They are off because **their benefit has not been measured against a real +model**, and a feature that spends latency on an unquantified gain is not a +default worth having. Turn them on and run `scripts/baseline.mjs` if you want to +find out. + +### Why the failures matter more than the features + +Both run on the critical path of a search, against a model on another box that +can be slow, wedged, or absent. That is not hypothetical here — the endpoint +this was developed against spent the whole project in exactly that state. So the +contracts are narrow: + +- **expansion always returns the original query**, first, whatever happens. The + worst case is a search with no expansion, never a failed search. +- **re-ranking returns `undefined` for "no opinion"**, which is deliberately not + the same as zero. Replacing a good embedding score with a zero because the + model timed out would turn a working search into an empty one. +- both use their **own deadline** inside the caller's signal, so a slow local + model gives up and lets the search continue: cancelling mean stop, being slow + does not. +- a non-finite timeout is **clamped, not propagated** — `AbortSignal.timeout(NaN)` + throws, and the catch below it would report that as "the model had nothing to + say". A misconfiguration must not wear the disguise of a slow model. + +Tested, including the wedged case: a socket that opens and never answers must +give up on its own deadline rather than pass the wait on. + +## Measuring it + +`scripts/baseline.mjs` is the before/after instrument, and it reads two stores +because they answer different questions: + +```sh +node scripts/baseline.mjs # human report +node scripts/baseline.mjs --json # machine readable +``` + +- the **cost ledger** knows requests, cache-read tokens, and therefore what each + request carried; +- the **session projection cache** knows the surface each session reached, and + whether it used recall at all. + +``` +BASELINE + sessions 109 + requests 39,706 + avg context/request (pooled) 339,182 every request counts once + avg context/request (typical) 155,175 every session counts once + cost $160.11 +``` + +**Both averages are reported on purpose.** They differ by a factor of two here, +because the largest sessions dominate the pooled figure — quoting only one would +mislead in whichever direction it happened to favour. Which to use depends on +the question: pooled is what maps to money, typical is what a new conversation +should expect. + +Sessions that used recall form the "after" group; everything else is the +baseline. While no session has used it, the report says so rather than printing +a comparison of nothing against nothing. + +## How it works + +**Chunking is structure-aware.** Splitting on a fixed line count produces units +that begin mid-function; their embedding sits between two topics and matches +neither. Chunks are cut at column-0 declarations instead, neighbouring small +declarations are merged, and a chunk that absorbs several of them is labelled with +all of their names — because for a small file that label is the module's only +description in the index. Minified and generated files are detected and skipped +rather than indexed as noise. + +**Ranking is hybrid, fused by rank.** Pure vector search is weak on code: an +embedding of `getUserById` sits near every other lookup in the repo. Lexical +scoring is weak in the opposite direction. The two are combined by reciprocal +rank fusion rather than by adding normalised scores — min-max normalising a +cluster of unrelated similarities stretches noise into a confident-looking +ranking, and rank fusion cannot make that mistake. + +**The index is incremental and lives outside your repository.** Files are keyed by +content hash; only changed files are re-embedded. The SQLite index sits in +`$DSH_HOME/dsh-recall/workspaces//` — never in the repo, because for a +300k-line project it is tens of megabytes. Vectors are stored unit-length, so +scoring is a dot product. + +**Cold start is bounded, and honest.** A search reads and hashes everything +(fast), then embeds only what fits its time budget, and reports exactly what is +still pending. `apply` never blocks on embedding a repository. In the measurement +above, the first search spent its 90s budget on 309 of 684 files (2,257 chunks); +a full `index` finished the remainder in about 75s more. + +**Changing the model or the vector width discards the index** rather than silently +mixing two incompatible vector spaces into one ranking. + +## Honest limitations + +- **The 6.0x figure is a comparison, not a guarantee.** It measures what the tool + returned against the whole files those results came from. An unusually careful + agent might have read only one of those files, so treat it as an upper bound on + the saving rather than a promise. +- **Token figures are `characters / 4`.** An approximation, not a measurement. +- **Excerpts are capped at 40 lines by default**, so most results in the + measurement above were truncated. The excerpt identifies the location; a + *targeted* follow-up `read` of that range is still sometimes the right move. The + saving is in not reading the other 90% of each file. +- **A locally built index costs minutes of CPU** on a large repository. It costs + no API money, and only once. +- **Retrieval quality is the model's quality.** A different embedding model will + rank differently. `scripts/smoke.mjs` is the check that the top hit is actually + correct. +- **A file edited since it was embedded is flagged `[stale]`**, not silently + trusted. + +## Operational notes + +- The **only outbound request** this plugin makes is to the embedding endpoint you + configure. No source code goes anywhere else, and there is no telemetry. +- The endpoint receives your source code. Point it at infrastructure you trust — + that is the whole reason it is explicit configuration rather than a default. +- The workspace walk **never follows a symlink**, so a link out of the repository + cannot cause the machine to be indexed. +- The index is a cache. Deleting `$DSH_HOME/dsh-recall/` costs one re-embed and + nothing else. + +### Give the endpoint room + +A cold index is a sustained CPU load — hundreds of batched requests. The defaults +(`embeddingBatchSize: 16`, `embeddingConcurrency: 2`) are deliberately +conservative, and they were chosen the hard way. + +**What happened during development.** An index run at concurrency 4 against a +small Ollama host that was *also* holding a generation model resident +(`expires_at` set to the year 2319 — pinned forever) left the host unable to load +the embedding model. It did not error: it accepted connections and never +answered. `ollama ps` afterwards showed only the pinned model. That needs a +restart of the Ollama service on that host; nothing this plugin can do reaches it. + +Three things now guard against it: + +- **Lower concurrency and batch size by default.** If you raise them, raise them + on hardware you know. +- **`embeddingKeepAlive`** (default `5m`) keeps the embedding model resident for + the length of a run instead of letting it be evicted and reloaded between + requests. +- **A short timeout, and a probe.** A wedged endpoint does not refuse a + connection, so a long timeout turns a broken backend into a tool call that + appears to hang. `recall status` issues a single no-retry probe with a 10-second + ceiling and reports whether the endpoint **answered** — which is a different + claim from "is configured", and the one worth having. + +If `status` reports `endpoint did NOT answer`, check `ollama ps` on that host +before assuming the plugin is at fault. + +## Development + +```sh +npm test # 91 tests, no network required +node scripts/smoke.mjs [workspace] [query] # live retrieval check +node scripts/measure.mjs [workspace] [query] [budgetMs] +``` + +The `lib/` tests run anywhere. `test/config.test.js` and `test/plugin.test.js` +import `index.js`, which imports `@deepseek-ai/dsh-tools` and +`@deepseek-ai/schemastery` — resolvable once the plugin is installed into a +profile (or via a local `node_modules` link to the profile's copies, which the +`.gitignore` excludes). + +`test/plugin.test.js` validates every tool result against **the tool's own +declared output schema**, which is `additionalProperties: false`. That is not +pedantry: the host enforces the schema before the model sees a result, so an +internal field leaking into the returned object is a failed call. It caught +exactly that during development. + +`test/config.test.js` covers the load-time contract. Cordis rejects a `Config` +before `apply` ever runs, so a broken schema cannot be reported from inside the +plugin — the profile simply refuses to boot. Every other test in the suite calls +`apply` directly and would not notice. + +`test/client.test.js` runs the browser half in Node by stubbing `window` and +`require('react')`. That is worth more than it sounds: the two failure modes most +likely to reach you silently are structural — a root element that is not an `
  • `, +and a card that does not start collapsed — and both are invisible until someone +opens Settings. It also pins that an untouched API key is never written or unset. diff --git a/client/client.js b/client/client.js new file mode 100644 index 0000000..37c9fee --- /dev/null +++ b/client/client.js @@ -0,0 +1,644 @@ +/** + * dsh-recall — browser half: the card in Settings → Plugins → Plugin configuration. + * + * The Host half registers a `dsh-recall` settings namespace. This file registers + * a card under that exact namespace key. Both are required: the tab pairs the + * namespaces the Host serves with the cards the browser claims, so a served + * namespace with no card draws nothing at all, and a card whose namespace is not + * served is never dispatched. + * + * The chrome follows the framework's `PluginCard`: the root is an `
  • ` (the tab + * renders every card into a `
      `), the header is its own `