Document what gets indexed, with the measured numbers

This commit is contained in:
dsh-slopo agent committed 2026-09-22 13:23:49 +01:00
1 parent 22be1e819a
commit 6b9d91b436
1 file changed
+39
+39
View File
@@ -175,6 +175,45 @@ Point `ignoreFile` at e.g. `<repo>/slopo.ignore.txt` if you would rather those
decisions were versioned with the code — they are a judgement about *your* decisions were versioned with the code — they are a judgement about *your*
codebase, and arguably belong in review. codebase, and arguably belong in review.
## What gets indexed
Slopo does **not** read `.gitignore` — it walks the tree you point it at. Pointed
at a repository root with no exclusions, it will happily parse every file under
`node_modules`. That is not a review of your code; it is a review of your
dependencies, and it is slow enough to matter.
Measured on one real repository:
| scope | files | code units | embedding at the reference rate |
| --- | --- | --- | --- |
| nothing excluded | ~1,300 | **113,768** | **~7.9 hours** |
| default excludes | 562 | 8,607 | ~36 minutes |
| default excludes + a server-only scope | **165** | **1,519** | **~6 minutes** |
So there are two independent safeguards.
**Default excludes**, on unless you turn them off: `node_modules`, `.git`,
`dist`, `build`, `out`, `coverage`, `.venv`, `venv`, `__pycache__`, `target`,
`vendor`, and minified bundles. Your own `sourceDirExclude` is **added** to
these rather than replacing them — deliberately, because a settings patch
replaces a whole value, so setting one pattern would otherwise silently
re-enable the 13x blow-up.
**A budget on the backlog.** `maxIndexUnits` (default 5,000) refuses to start an
embed that exceeds it, naming the count, the estimate and the lever:
> 113,768 code units still need embedding across 1,300 files. At the reference
> rate that is about 7.9 hours, against a budget of 5,000.
Nothing has been embedded at that point. Raise the budget, or narrow the scope —
setting `maxIndexUnits` to 0 removes the limit. The count comes from reading
slopo's own database, not from parsing its output, because `Indexed N code units`
reports only what was *newly* indexed: on a warm database it reads zero while the
backlog is still full, which is exactly the state a refusal leaves behind.
Every action indexes first so the gate runs before anything expensive, and
`status` reports both the effective excludes and the budget.
## Things worth knowing ## Things worth knowing
**Five settings are frozen at first index.** `source_dir`, `embedding_model`, **Five settings are frozen at first index.** `source_dir`, `embedding_model`,