Document dsh-slopo, including the traps that cost real time
The provider prefix is not cosmetic: the openai/ prefix makes LiteLLM apply OpenAI parameter validation to any model behind it, and slopo always passes dimensions, so a local OpenAI-compatible server needs its own prefix. Five settings are frozen into the database at first index, so changing the embedding model means re-embedding everything. And a review cannot bootstrap the database at all, because agent-review opens the index read-only.
This commit is contained in:
dsh-slopo agent committed
1 parent
3373e0b4f6
commit
a8e577ae38
1 file changed
+225
@@ -0,0 +1,225 @@
|
||||
# dsh-slopo
|
||||
|
||||
[slopo](https://github.com/rafal-qa/slopo) — an embedding-based duplicate-code
|
||||
detector — as a DeepSeek Harness bundle.
|
||||
|
||||
Slopo finds non-obvious duplicated logic: similar implementations of the same
|
||||
problem that are not copy-paste, and that a human reviewer, a coding agent and a
|
||||
clone detector all miss. This bundle gives you three user-triggered skills and
|
||||
one configuration-aware tool, so you can run it against whichever workspace you
|
||||
are in without remembering flags.
|
||||
|
||||
## What it adds
|
||||
|
||||
| Contribution | What it is |
|
||||
| --- | --- |
|
||||
| `/slopo-review` | Slopo reports similar code involving your Git changes, and the agent reviews them as a note-taker. |
|
||||
| `/slopo-analyze-ignore` | Triages the whole codebase's clusters, writing the obvious false positives into the ignore file. |
|
||||
| `/slopo-analyze-one` | Reports one cluster at a time — optionally by hash — and judges whether it is worth acting on. |
|
||||
| the `slopo` tool | Resolves the CLI, materialises the config, runs the work, and turns slopo's terse failures into something actionable. |
|
||||
| the `dsh-slopo` settings namespace | The embedding backend and per-workspace state, editable at runtime. |
|
||||
|
||||
All three skills are **user-invocable and deliberately not model-invocable**,
|
||||
matching slopo's own agent configs. Typing `/slopo-review` anywhere in a message
|
||||
injects that skill; the model will not reach for one on its own. Every one of
|
||||
these commands can spend minutes embedding, so the human decides when.
|
||||
|
||||
## Install
|
||||
|
||||
```sh
|
||||
dsh plugin --profile web add git+https://git.notexpectedyet.com/Not-Expected-Yet/dsh-slopo.git
|
||||
```
|
||||
|
||||
Then restart the profile — a new bundle's modules are resolved at startup.
|
||||
|
||||
Plain ESM, no build step, and therefore no `prepare` script and no `allowBuilds`
|
||||
entry to authorise. That is deliberate: `dsh plugin add git+…` fetches the source
|
||||
tree rather than build output, so a `prepare` script would make every user
|
||||
allowlist a build before the first install can succeed.
|
||||
|
||||
## Configure it
|
||||
|
||||
Nothing is defaulted. Slopo itself requires a source directory, an embedding
|
||||
model and its dimensions, and guessing an embedding backend would mean guessing
|
||||
where money is spent — so the plugin starts unconfigured and says so.
|
||||
|
||||
Ask the tool what it wants:
|
||||
|
||||
```jsonc
|
||||
{ "action": "status" }
|
||||
```
|
||||
|
||||
Then set it up:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"action": "configure",
|
||||
"config": {
|
||||
"embeddingModel": "ollama/unclemusclez/jina-embeddings-v2-base-code:latest",
|
||||
"embeddingDimensions": 768,
|
||||
"embeddingApiBase": "http://127.0.0.1:11434"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Values land in the `dsh-slopo` section of `$DSH_HOME/settings.yaml`.
|
||||
|
||||
| Setting | Meaning |
|
||||
| --- | --- |
|
||||
| `embeddingModel` | LiteLLM model string. **Include the provider prefix** — see below. |
|
||||
| `embeddingDimensions` | Must match what the model actually returns. Slopo freezes it at first index. |
|
||||
| `embeddingApiBase` | Provider base URL. Becomes slopo's `embedding_params.api_base`. |
|
||||
| `embeddingApiKey` | Optional. Passed to slopo in its environment, **never written to disk**. |
|
||||
| `embeddingParamsJson` | Extra LiteLLM embedding params as a JSON object string. |
|
||||
| `sourceDir` | Code to index. Defaults to the session's working directory. |
|
||||
| `sourceDirExclude` | gitignore-style paths to skip. |
|
||||
| `autoInstall` | Install slopo into `$DSH_HOME` with `uv` on first use. Default `true`. |
|
||||
| `binary` | An explicit slopo executable, overriding discovery. |
|
||||
| `stateDir` | Override the per-workspace state directory. |
|
||||
| `ignoreFile` | Override the ignore file path. |
|
||||
| `embeddingRequestDelay` | Milliseconds between embedding requests, for rate-limited providers. |
|
||||
| `timeoutMs` | Cooperative budget for a foreground run. |
|
||||
|
||||
### Choose the provider prefix carefully
|
||||
|
||||
The provider prefix is not cosmetic — it selects which LiteLLM validator runs.
|
||||
|
||||
```
|
||||
openai/<model> ✗ UnsupportedParamsError: Setting dimensions is not supported
|
||||
for OpenAI `text-embedding-3` and later models
|
||||
ollama/<model> ✓ works
|
||||
```
|
||||
|
||||
Slopo always passes `dimensions`, and the `openai/` prefix makes LiteLLM apply
|
||||
OpenAI-specific parameter validation to *any* model behind it. A local
|
||||
OpenAI-compatible server therefore needs its own prefix (`ollama/`,
|
||||
`hosted_vllm/`), not `openai/`. This is the single most likely thing to trip you
|
||||
up on first setup, and slopo reports it only as `Error: Embedding failed.` on
|
||||
stdout — the real message is in the agent log, which the tool reads for you.
|
||||
|
||||
### A local backend costs nothing
|
||||
|
||||
A local Ollama server is a good default: free, private, no API key and no
|
||||
egress. Measured throughput on a small CPU box is roughly **230 ms per unique
|
||||
unit** (~4/second), with repeated inputs served from cache. Budget minutes, not
|
||||
seconds, for a first index of a real repository.
|
||||
|
||||
## Use it
|
||||
|
||||
Type the skill name in a message:
|
||||
|
||||
```
|
||||
/slopo-review
|
||||
/slopo-review main
|
||||
/slopo-analyze-one
|
||||
```
|
||||
|
||||
Or drive the tool directly:
|
||||
|
||||
| Action | What it does |
|
||||
| --- | --- |
|
||||
| `status` | Resolved settings, index state, binary, and what is still missing. |
|
||||
| `configure` | Patch the settings namespace. |
|
||||
| `prepare` | `slopo index` then `slopo embed` — the slow, possibly costly phase. |
|
||||
| `review` | Similar code involving Git changes (`base` defaults to `HEAD`). |
|
||||
| `analyze` | Triage the whole codebase. |
|
||||
| `analyze-one` | One cluster, optionally by `cluster` hash. |
|
||||
| `update` | Upgrade the slopo CLI. |
|
||||
|
||||
The long actions run as **background jobs** by default (pass `foreground: true`
|
||||
to run inline). Embedding takes minutes, and slopo writes its progress to a log
|
||||
file rather than to stdout, so there is nothing to stream — the job handle
|
||||
carries the log path instead.
|
||||
|
||||
## Where things live
|
||||
|
||||
Everything is under `$DSH_HOME`, and nothing is written into your repository:
|
||||
|
||||
```
|
||||
$DSH_HOME/dsh-slopo/
|
||||
├── uv-tools/ # the uv-managed slopo install and its shim
|
||||
├── uv-python/ # the CPython 3.12 that slopo requires
|
||||
└── workspaces/<name>-<hash8>/
|
||||
├── slopo.conf.yaml # generated; edits here are overwritten
|
||||
├── slopo.db # the index and embeddings
|
||||
├── slopo.ignore.txt # dismissed cluster hashes
|
||||
└── slopo.agent.log # full errors, progress
|
||||
```
|
||||
|
||||
`$DSH_HOME` is not an arbitrary choice. In a containerised harness it is usually
|
||||
the only persistent mount, while `~/.local` — where `uv tool install` puts things
|
||||
by default — lives on the container's writable layer and is lost when the
|
||||
container is *recreated*, not merely restarted. Setting `UV_TOOL_DIR`,
|
||||
`UV_TOOL_BIN_DIR` and `UV_PYTHON_INSTALL_DIR` into `$DSH_HOME` makes the install
|
||||
survive that, and installation is lazy rather than on boot so a fresh container
|
||||
self-heals on first use.
|
||||
|
||||
Point `ignoreFile` at e.g. `<repo>/slopo.ignore.txt` if you would rather those
|
||||
decisions were versioned with the code — they are a judgement about *your*
|
||||
codebase, and arguably belong in review.
|
||||
|
||||
## Things worth knowing
|
||||
|
||||
**Five settings are frozen at first index.** `source_dir`, `embedding_model`,
|
||||
`embedding_dimensions` and both node-count thresholds are written into
|
||||
`slopo.db` on the first `index`, and every later run refuses to mix generations:
|
||||
|
||||
```
|
||||
Error: Configuration mismatch.
|
||||
```
|
||||
|
||||
Changing any of them means deleting the database and re-embedding everything —
|
||||
real time, and real money on a hosted backend. The tool recognises this failure
|
||||
and says so rather than relaying the two-word stderr line.
|
||||
|
||||
**A review cannot bootstrap the database.** `agent-review` opens the index with a
|
||||
read-only open, so on a fresh workspace it fails with `No data in database. The
|
||||
tool wasn't initialized.` Run `prepare` first. Indexing is pure local parsing and
|
||||
free; only embedding reaches the network.
|
||||
|
||||
**Slopo refuses more than 1000 unembedded units** inside an agent call, to
|
||||
protect agent latency. The tool translates that into an offer to run `prepare`
|
||||
in the background.
|
||||
|
||||
**Slopo's stderr is deliberately unhelpful.** It prints one short non-revealing
|
||||
line for an agent and writes the real error to the agent log. The tool reads the
|
||||
log tail and maps the known failures — missing database, configuration mismatch,
|
||||
oversized backlog, missing or invalid config, stale index, git failure,
|
||||
embedding failure — onto a message and a next step.
|
||||
|
||||
**`--config-version 1` is an alarm, not boilerplate.** It makes slopo fail if its
|
||||
agent-facing contract has moved on, which is the signal that the skill text in
|
||||
`skills/` needs re-porting from upstream.
|
||||
|
||||
## Slopo is an external process, not a library
|
||||
|
||||
It is installed separately, invoked through its own CLI, and never imported or
|
||||
modified. Slopo is AGPL-3.0-or-later and does not accept patches, so its
|
||||
command-line interface is the only contract worth building on — and executing an
|
||||
unmodified, separately installed program is a materially different proposition
|
||||
from linking against it.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
node --test test/*.test.js # unit + integration, mock harness
|
||||
DSH_SLOPO_E2E=1 npm test # also drives the real CLI end to end
|
||||
```
|
||||
|
||||
The integration tests load the real plugin module against a mock Cordis context
|
||||
and drive the tool through a real shell into the real slopo binary, so a wrong
|
||||
assumption about a dsh service surfaces as a failing test rather than as a plugin
|
||||
that silently does nothing. Two real bugs were caught this way: a config
|
||||
renderer that would write the API key to disk if handed one, and `--config-version`
|
||||
placed before the subcommand, which slopo rejects with a usage error.
|
||||
|
||||
`node_modules/@deepseek-ai` is a local symlink to the installed harness so the
|
||||
`@deepseek-ai/*` imports resolve during tests. It is gitignored and is not a
|
||||
declared dependency — an out-of-tree bundle that depended on an in-box package
|
||||
would install a second copy that drifts from the host's.
|
||||
|
||||
## Licence
|
||||
|
||||
AGPL-3.0-or-later, matching slopo. The three skill bodies are ports of slopo's
|
||||
own agent configurations rather than new prose; `NOTICE.md` records exactly what
|
||||
changed and why the licence follows.
|
||||
Reference in new issue
Block a user