diff --git a/README.md b/README.md index 124234e..fdda454 100644 --- a/README.md +++ b/README.md @@ -175,6 +175,45 @@ Point `ignoreFile` at e.g. `/slopo.ignore.txt` if you would rather those decisions were versioned with the code — they are a judgement about *your* codebase, and arguably belong in review. +## What gets indexed + +Slopo does **not** read `.gitignore` — it walks the tree you point it at. Pointed +at a repository root with no exclusions, it will happily parse every file under +`node_modules`. That is not a review of your code; it is a review of your +dependencies, and it is slow enough to matter. + +Measured on one real repository: + +| scope | files | code units | embedding at the reference rate | +| --- | --- | --- | --- | +| nothing excluded | ~1,300 | **113,768** | **~7.9 hours** | +| default excludes | 562 | 8,607 | ~36 minutes | +| default excludes + a server-only scope | **165** | **1,519** | **~6 minutes** | + +So there are two independent safeguards. + +**Default excludes**, on unless you turn them off: `node_modules`, `.git`, +`dist`, `build`, `out`, `coverage`, `.venv`, `venv`, `__pycache__`, `target`, +`vendor`, and minified bundles. Your own `sourceDirExclude` is **added** to +these rather than replacing them — deliberately, because a settings patch +replaces a whole value, so setting one pattern would otherwise silently +re-enable the 13x blow-up. + +**A budget on the backlog.** `maxIndexUnits` (default 5,000) refuses to start an +embed that exceeds it, naming the count, the estimate and the lever: + +> 113,768 code units still need embedding across 1,300 files. At the reference +> rate that is about 7.9 hours, against a budget of 5,000. + +Nothing has been embedded at that point. Raise the budget, or narrow the scope — +setting `maxIndexUnits` to 0 removes the limit. The count comes from reading +slopo's own database, not from parsing its output, because `Indexed N code units` +reports only what was *newly* indexed: on a warm database it reads zero while the +backlog is still full, which is exactly the state a refusal leaves behind. + +Every action indexes first so the gate runs before anything expensive, and +`status` reports both the effective excludes and the budget. + ## Things worth knowing **Five settings are frozen at first index.** `source_dir`, `embedding_model`,