dsh-slopo/README.md

# dsh-slopo

[slopo](https://github.com/rafal-qa/slopo) — an embedding-based duplicate-code
detector — as a DeepSeek Harness bundle.

Slopo finds non-obvious duplicated logic: similar implementations of the same
problem that are not copy-paste, and that a human reviewer, a coding agent and a
clone detector all miss. This bundle gives you three user-triggered skills and
one configuration-aware tool, so you can run it against whichever workspace you
are in without remembering flags.

## What it adds

| Contribution | What it is |
| --- | --- |
| `/slopo-review` | Slopo reports similar code involving your Git changes, and the agent reviews them as a note-taker. |
| `/slopo-analyze-ignore` | Triages the whole codebase's clusters, writing the obvious false positives into the ignore file. |
| `/slopo-analyze-one` | Reports one cluster at a time — optionally by hash — and judges whether it is worth acting on. |
| the `slopo` tool | Resolves the CLI, materialises the config, runs the work, and turns slopo's terse failures into something actionable. |
| the `dsh-slopo` settings namespace | The embedding backend and per-workspace state, editable at runtime. |

All three skills are **user-invocable and deliberately not model-invocable**,
matching slopo's own agent configs. Typing `/slopo-review` anywhere in a message
injects that skill; the model will not reach for one on its own. Every one of
these commands can spend minutes embedding, so the human decides when.

## Install

```sh
dsh plugin --profile web add git+https://git.notexpectedyet.com/Not-Expected-Yet/dsh-slopo.git
```

Then restart the profile — a new bundle's modules are resolved at startup.

Plain ESM, no build step, and therefore no `prepare` script and no `allowBuilds`
entry to authorise. That is deliberate: `dsh plugin add git+…` fetches the source
tree rather than build output, so a `prepare` script would make every user
allowlist a build before the first install can succeed.

## Configure it

Nothing is defaulted. Slopo itself requires a source directory, an embedding
model and its dimensions, and guessing an embedding backend would mean guessing
where money is spent — so the plugin starts unconfigured and says so.

Ask the tool what it wants:

```jsonc
{ "action": "status" }
```

Then set it up:

```jsonc
{
  "action": "configure",
  "config": {
    "embeddingModel": "ollama/unclemusclez/jina-embeddings-v2-base-code:latest",
    "embeddingDimensions": 768,
    "embeddingApiBase": "http://127.0.0.1:11434"
  }
}
```

Values land in the `dsh-slopo` section of `$DSH_HOME/settings.yaml`.

| Setting | Meaning |
| --- | --- |
| `embeddingModel` | LiteLLM model string. **Include the provider prefix** — see below. |
| `embeddingDimensions` | Must match what the model actually returns. Slopo freezes it at first index. |
| `embeddingApiBase` | Provider base URL. Becomes slopo's `embedding_params.api_base`. |
| `embeddingApiKey` | Optional. Passed to slopo in its environment, **never written to disk**. |
| `embeddingParamsJson` | Extra LiteLLM embedding params as a JSON object string. |
| `sourceDir` | Code to index. Defaults to the session's working directory. |
| `sourceDirExclude` | gitignore-style paths to skip. |
| `autoInstall` | Install slopo into `$DSH_HOME` with `uv` on first use. Default `true`. |
| `binary` | An explicit slopo executable, overriding discovery. |
| `stateDir` | Override the per-workspace state directory. |
| `ignoreFile` | Override the ignore file path. |
| `embeddingRequestDelay` | Milliseconds between embedding requests, for rate-limited providers. |
| `timeoutMs` | Cooperative budget for a foreground run. |

### Choose the provider prefix carefully

The provider prefix is not cosmetic — it selects which LiteLLM validator runs.

```
openai/<model>   ✗  UnsupportedParamsError: Setting dimensions is not supported
                    for OpenAI `text-embedding-3` and later models
ollama/<model>   ✓  works
```

Slopo always passes `dimensions`, and the `openai/` prefix makes LiteLLM apply
OpenAI-specific parameter validation to *any* model behind it. A local
OpenAI-compatible server therefore needs its own prefix (`ollama/`,
`hosted_vllm/`), not `openai/`. This is the single most likely thing to trip you
up on first setup, and slopo reports it only as `Error: Embedding failed.` on
stdout — the real message is in the agent log, which the tool reads for you.

### A local backend costs nothing

A local Ollama server is a good default: free, private, no API key and no
egress. Measured throughput on a small CPU box is roughly **230 ms per unique
unit** (~4/second), with repeated inputs served from cache. Budget minutes, not
seconds, for a first index of a real repository.

## Use it

Type the skill name in a message:

```
/slopo-review
/slopo-review main
/slopo-analyze-one
```

Or drive the tool directly:

| Action | What it does |
| --- | --- |
| `status` | Resolved settings, index state, binary, and what is still missing. |
| `configure` | Patch the settings namespace. |
| `prepare` | `slopo index` then `slopo embed` — the slow, possibly costly phase. |
| `review` | Similar code involving Git changes (`base` defaults to `HEAD`). |
| `analyze` | Triage the whole codebase. |
| `analyze-one` | One cluster, optionally by `cluster` hash. |
| `update` | Upgrade the slopo CLI. |

The long actions run as **background jobs** by default (pass `foreground: true`
to run inline). Embedding takes minutes, and slopo writes its progress to a log
file rather than to stdout, so there is nothing to stream — the job handle
carries the log path instead.

## Where things live

Everything is under `$DSH_HOME`, and nothing is written into your repository:

```
$DSH_HOME/dsh-slopo/
├── uv-tools/                    # the uv-managed slopo install and its shim
├── uv-python/                   # the CPython 3.12 that slopo requires
└── workspaces/<name>-<hash8>/
    ├── slopo.conf.yaml          # generated; edits here are overwritten
    ├── slopo.db                 # the index and embeddings
    ├── slopo.ignore.txt         # dismissed cluster hashes
    └── slopo.agent.log          # full errors, progress
```

`$DSH_HOME` is not an arbitrary choice. In a containerised harness it is usually
the only persistent mount, while `~/.local` — where `uv tool install` puts things
by default — lives on the container's writable layer and is lost when the
container is *recreated*, not merely restarted. Setting `UV_TOOL_DIR`,
`UV_TOOL_BIN_DIR` and `UV_PYTHON_INSTALL_DIR` into `$DSH_HOME` makes the install
survive that, and installation is lazy rather than on boot so a fresh container
self-heals on first use.

Point `ignoreFile` at e.g. `<repo>/slopo.ignore.txt` if you would rather those
decisions were versioned with the code — they are a judgement about *your*
codebase, and arguably belong in review.

## Things worth knowing

**Five settings are frozen at first index.** `source_dir`, `embedding_model`,
`embedding_dimensions` and both node-count thresholds are written into
`slopo.db` on the first `index`, and every later run refuses to mix generations:

```
Error: Configuration mismatch.
```

Changing any of them means deleting the database and re-embedding everything —
real time, and real money on a hosted backend. The tool recognises this failure
and says so rather than relaying the two-word stderr line.

**A review cannot bootstrap the database.** `agent-review` opens the index with a
read-only open, so on a fresh workspace it fails with `No data in database. The
tool wasn't initialized.` Run `prepare` first. Indexing is pure local parsing and
free; only embedding reaches the network.

**Slopo refuses more than 1000 unembedded units** inside an agent call, to
protect agent latency. The tool translates that into an offer to run `prepare`
in the background.

**Slopo's stderr is deliberately unhelpful.** It prints one short non-revealing
line for an agent and writes the real error to the agent log. The tool reads the
log tail and maps the known failures — missing database, configuration mismatch,
oversized backlog, missing or invalid config, stale index, git failure,
embedding failure — onto a message and a next step.

**`--config-version 1` is an alarm, not boilerplate.** It makes slopo fail if its
agent-facing contract has moved on, which is the signal that the skill text in
`skills/` needs re-porting from upstream.

## Slopo is an external process, not a library

It is installed separately, invoked through its own CLI, and never imported or
modified. Slopo is AGPL-3.0-or-later and does not accept patches, so its
command-line interface is the only contract worth building on — and executing an
unmodified, separately installed program is a materially different proposition
from linking against it.

## Development

```sh
node --test test/*.test.js          # unit + integration, mock harness
DSH_SLOPO_E2E=1 npm test            # also drives the real CLI end to end
```

The integration tests load the real plugin module against a mock Cordis context
and drive the tool through a real shell into the real slopo binary, so a wrong
assumption about a dsh service surfaces as a failing test rather than as a plugin
that silently does nothing. Two real bugs were caught this way: a config
renderer that would write the API key to disk if handed one, and `--config-version`
placed before the subcommand, which slopo rejects with a usage error.

`node_modules/@deepseek-ai` is a local symlink to the installed harness so the
`@deepseek-ai/*` imports resolve during tests. It is gitignored and is not a
declared dependency — an out-of-tree bundle that depended on an in-box package
would install a second copy that drifts from the host's.

## Licence

AGPL-3.0-or-later, matching slopo. The three skill bodies are ports of slopo's
own agent configurations rather than new prose; `NOTICE.md` records exactly what
changed and why the licence follows.