From a8e577ae386c4e19f0913bc117d3e6d6b537f872 Mon Sep 17 00:00:00 2001 From: dsh-slopo agent Date: Tue, 22 Sep 2026 06:11:17 +0100 Subject: [PATCH] Document dsh-slopo, including the traps that cost real time The provider prefix is not cosmetic: the openai/ prefix makes LiteLLM apply OpenAI parameter validation to any model behind it, and slopo always passes dimensions, so a local OpenAI-compatible server needs its own prefix. Five settings are frozen into the database at first index, so changing the embedding model means re-embedding everything. And a review cannot bootstrap the database at all, because agent-review opens the index read-only. --- README.md | 225 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..853f71b --- /dev/null +++ b/README.md @@ -0,0 +1,225 @@ +# dsh-slopo + +[slopo](https://github.com/rafal-qa/slopo) — an embedding-based duplicate-code +detector — as a DeepSeek Harness bundle. + +Slopo finds non-obvious duplicated logic: similar implementations of the same +problem that are not copy-paste, and that a human reviewer, a coding agent and a +clone detector all miss. This bundle gives you three user-triggered skills and +one configuration-aware tool, so you can run it against whichever workspace you +are in without remembering flags. + +## What it adds + +| Contribution | What it is | +| --- | --- | +| `/slopo-review` | Slopo reports similar code involving your Git changes, and the agent reviews them as a note-taker. | +| `/slopo-analyze-ignore` | Triages the whole codebase's clusters, writing the obvious false positives into the ignore file. | +| `/slopo-analyze-one` | Reports one cluster at a time — optionally by hash — and judges whether it is worth acting on. | +| the `slopo` tool | Resolves the CLI, materialises the config, runs the work, and turns slopo's terse failures into something actionable. | +| the `dsh-slopo` settings namespace | The embedding backend and per-workspace state, editable at runtime. | + +All three skills are **user-invocable and deliberately not model-invocable**, +matching slopo's own agent configs. Typing `/slopo-review` anywhere in a message +injects that skill; the model will not reach for one on its own. Every one of +these commands can spend minutes embedding, so the human decides when. + +## Install + +```sh +dsh plugin --profile web add git+https://git.notexpectedyet.com/Not-Expected-Yet/dsh-slopo.git +``` + +Then restart the profile — a new bundle's modules are resolved at startup. + +Plain ESM, no build step, and therefore no `prepare` script and no `allowBuilds` +entry to authorise. That is deliberate: `dsh plugin add git+…` fetches the source +tree rather than build output, so a `prepare` script would make every user +allowlist a build before the first install can succeed. + +## Configure it + +Nothing is defaulted. Slopo itself requires a source directory, an embedding +model and its dimensions, and guessing an embedding backend would mean guessing +where money is spent — so the plugin starts unconfigured and says so. + +Ask the tool what it wants: + +```jsonc +{ "action": "status" } +``` + +Then set it up: + +```jsonc +{ + "action": "configure", + "config": { + "embeddingModel": "ollama/unclemusclez/jina-embeddings-v2-base-code:latest", + "embeddingDimensions": 768, + "embeddingApiBase": "http://127.0.0.1:11434" + } +} +``` + +Values land in the `dsh-slopo` section of `$DSH_HOME/settings.yaml`. + +| Setting | Meaning | +| --- | --- | +| `embeddingModel` | LiteLLM model string. **Include the provider prefix** — see below. | +| `embeddingDimensions` | Must match what the model actually returns. Slopo freezes it at first index. | +| `embeddingApiBase` | Provider base URL. Becomes slopo's `embedding_params.api_base`. | +| `embeddingApiKey` | Optional. Passed to slopo in its environment, **never written to disk**. | +| `embeddingParamsJson` | Extra LiteLLM embedding params as a JSON object string. | +| `sourceDir` | Code to index. Defaults to the session's working directory. | +| `sourceDirExclude` | gitignore-style paths to skip. | +| `autoInstall` | Install slopo into `$DSH_HOME` with `uv` on first use. Default `true`. | +| `binary` | An explicit slopo executable, overriding discovery. | +| `stateDir` | Override the per-workspace state directory. | +| `ignoreFile` | Override the ignore file path. | +| `embeddingRequestDelay` | Milliseconds between embedding requests, for rate-limited providers. | +| `timeoutMs` | Cooperative budget for a foreground run. | + +### Choose the provider prefix carefully + +The provider prefix is not cosmetic — it selects which LiteLLM validator runs. + +``` +openai/ ✗ UnsupportedParamsError: Setting dimensions is not supported + for OpenAI `text-embedding-3` and later models +ollama/ ✓ works +``` + +Slopo always passes `dimensions`, and the `openai/` prefix makes LiteLLM apply +OpenAI-specific parameter validation to *any* model behind it. A local +OpenAI-compatible server therefore needs its own prefix (`ollama/`, +`hosted_vllm/`), not `openai/`. This is the single most likely thing to trip you +up on first setup, and slopo reports it only as `Error: Embedding failed.` on +stdout — the real message is in the agent log, which the tool reads for you. + +### A local backend costs nothing + +A local Ollama server is a good default: free, private, no API key and no +egress. Measured throughput on a small CPU box is roughly **230 ms per unique +unit** (~4/second), with repeated inputs served from cache. Budget minutes, not +seconds, for a first index of a real repository. + +## Use it + +Type the skill name in a message: + +``` +/slopo-review +/slopo-review main +/slopo-analyze-one +``` + +Or drive the tool directly: + +| Action | What it does | +| --- | --- | +| `status` | Resolved settings, index state, binary, and what is still missing. | +| `configure` | Patch the settings namespace. | +| `prepare` | `slopo index` then `slopo embed` — the slow, possibly costly phase. | +| `review` | Similar code involving Git changes (`base` defaults to `HEAD`). | +| `analyze` | Triage the whole codebase. | +| `analyze-one` | One cluster, optionally by `cluster` hash. | +| `update` | Upgrade the slopo CLI. | + +The long actions run as **background jobs** by default (pass `foreground: true` +to run inline). Embedding takes minutes, and slopo writes its progress to a log +file rather than to stdout, so there is nothing to stream — the job handle +carries the log path instead. + +## Where things live + +Everything is under `$DSH_HOME`, and nothing is written into your repository: + +``` +$DSH_HOME/dsh-slopo/ +├── uv-tools/ # the uv-managed slopo install and its shim +├── uv-python/ # the CPython 3.12 that slopo requires +└── workspaces/-/ + ├── slopo.conf.yaml # generated; edits here are overwritten + ├── slopo.db # the index and embeddings + ├── slopo.ignore.txt # dismissed cluster hashes + └── slopo.agent.log # full errors, progress +``` + +`$DSH_HOME` is not an arbitrary choice. In a containerised harness it is usually +the only persistent mount, while `~/.local` — where `uv tool install` puts things +by default — lives on the container's writable layer and is lost when the +container is *recreated*, not merely restarted. Setting `UV_TOOL_DIR`, +`UV_TOOL_BIN_DIR` and `UV_PYTHON_INSTALL_DIR` into `$DSH_HOME` makes the install +survive that, and installation is lazy rather than on boot so a fresh container +self-heals on first use. + +Point `ignoreFile` at e.g. `/slopo.ignore.txt` if you would rather those +decisions were versioned with the code — they are a judgement about *your* +codebase, and arguably belong in review. + +## Things worth knowing + +**Five settings are frozen at first index.** `source_dir`, `embedding_model`, +`embedding_dimensions` and both node-count thresholds are written into +`slopo.db` on the first `index`, and every later run refuses to mix generations: + +``` +Error: Configuration mismatch. +``` + +Changing any of them means deleting the database and re-embedding everything — +real time, and real money on a hosted backend. The tool recognises this failure +and says so rather than relaying the two-word stderr line. + +**A review cannot bootstrap the database.** `agent-review` opens the index with a +read-only open, so on a fresh workspace it fails with `No data in database. The +tool wasn't initialized.` Run `prepare` first. Indexing is pure local parsing and +free; only embedding reaches the network. + +**Slopo refuses more than 1000 unembedded units** inside an agent call, to +protect agent latency. The tool translates that into an offer to run `prepare` +in the background. + +**Slopo's stderr is deliberately unhelpful.** It prints one short non-revealing +line for an agent and writes the real error to the agent log. The tool reads the +log tail and maps the known failures — missing database, configuration mismatch, +oversized backlog, missing or invalid config, stale index, git failure, +embedding failure — onto a message and a next step. + +**`--config-version 1` is an alarm, not boilerplate.** It makes slopo fail if its +agent-facing contract has moved on, which is the signal that the skill text in +`skills/` needs re-porting from upstream. + +## Slopo is an external process, not a library + +It is installed separately, invoked through its own CLI, and never imported or +modified. Slopo is AGPL-3.0-or-later and does not accept patches, so its +command-line interface is the only contract worth building on — and executing an +unmodified, separately installed program is a materially different proposition +from linking against it. + +## Development + +```sh +node --test test/*.test.js # unit + integration, mock harness +DSH_SLOPO_E2E=1 npm test # also drives the real CLI end to end +``` + +The integration tests load the real plugin module against a mock Cordis context +and drive the tool through a real shell into the real slopo binary, so a wrong +assumption about a dsh service surfaces as a failing test rather than as a plugin +that silently does nothing. Two real bugs were caught this way: a config +renderer that would write the API key to disk if handed one, and `--config-version` +placed before the subcommand, which slopo rejects with a usage error. + +`node_modules/@deepseek-ai` is a local symlink to the installed harness so the +`@deepseek-ai/*` imports resolve during tests. It is gitignored and is not a +declared dependency — an out-of-tree bundle that depended on an in-box package +would install a second copy that drifts from the host's. + +## Licence + +AGPL-3.0-or-later, matching slopo. The three skill bodies are ports of slopo's +own agent configurations rather than new prose; `NOTICE.md` records exactly what +changed and why the licence follows.