dsh-slopo/README.md

# dsh-slopo

[slopo](https://github.com/rafal-qa/slopo) — an embedding-based duplicate-code
detector — as a DeepSeek Harness bundle.

Slopo finds non-obvious duplicated logic: similar implementations of the same
problem that are not copy-paste, and that a human reviewer, a coding agent and a
clone detector all miss. This bundle gives you three user-triggered skills and
one configuration-aware tool, and a settings card. Run it against whichever workspace you
are in without remembering flags.

## What it adds

| Contribution | What it is |
| --- | --- |
| `/slopo-review` | Slopo reports similar code involving your Git changes, and the agent reviews them as a note-taker. |
| `/slopo-analyze-ignore` | Triages the whole codebase's clusters, writing the obvious false positives into the ignore file. |
| `/slopo-analyze-one` | Reports one cluster at a time — optionally by hash — and judges whether it is worth acting on. |
| the `slopo` tool | Resolves the CLI, materialises the config, runs the work, and turns slopo's terse failures into something actionable. |
| the `dsh-slopo` settings namespace | The embedding backend and per-workspace state, registered on the Host. |
| the settings card | A form for all of it, in Settings → Plugins → Plugin configuration. |
All three skills are **user-invocable and deliberately not model-invocable**,
matching slopo's own agent configs. Typing `/slopo-review` anywhere in a message
injects that skill; the model will not reach for one on its own. Every one of
these commands can spend minutes embedding, so the human decides when.

## Install

```sh
dsh plugin --profile web add git+https://git.notexpectedyet.com/Not-Expected-Yet/dsh-slopo.git
```

Then restart the profile. A new bundle's modules are resolved at startup, and a
client half's bytes are snapshotted at activation — so the settings card only
appears after that restart.

Plain ESM, no build step, and therefore no `prepare` script and no `allowBuilds`
entry to authorise. That is deliberate: `dsh plugin add git+…` fetches the source
tree rather than build output, so a `prepare` script would make every user
allowlist a build before the first install can succeed.

## Settings

The plugin contributes a card to **Settings → Plugins → Plugin configuration**,
keyed on the `dsh-slopo` settings namespace the Host registers. Every setting in
the table below is editable there, overridden fields are marked, and Reset
returns a field to the deployment default rather than to empty.

The card behaves the way the other plugin cards do: it starts collapsed, its
header is a button that discloses the controls in place, the header marks it when
it holds unsaved edits — visible even while collapsed, since staged edits outlive
being folded away — and a save that lands collapses it again.

The card is the bundle's second half — the browser half. It needs no build step
and ships as plain ESM in the module-loader wrapper, but the platform snapshots
client bundles at activation, so **a client half only appears after the profile
restarts.** Without it the namespace still works — it is editable in
`$DSH_HOME/settings.yaml` and through the tool's `configure` action — there is
simply no form drawn for it, because the tab renders the intersection of served
namespaces and registered cards.

## Configure it

Nothing is defaulted. Slopo itself requires a source directory, an embedding
model and its dimensions, and guessing an embedding backend would mean guessing
where money is spent — so the plugin starts unconfigured and says so.

Ask the tool what it wants:

```jsonc
{ "action": "status" }
```

Then set it up:

```jsonc
{
  "action": "configure",
  "config": {
    "embeddingModel": "ollama/unclemusclez/jina-embeddings-v2-base-code:latest",
    "embeddingDimensions": 768,
    "embeddingApiBase": "http://127.0.0.1:11434"
  }
}
```

Values land in the `dsh-slopo` section of `$DSH_HOME/settings.yaml`.

| Setting | Meaning |
| --- | --- |
| `embeddingModel` | LiteLLM model string. **Include the provider prefix** — see below. |
| `embeddingDimensions` | Must match what the model actually returns. Slopo freezes it at first index. |
| `embeddingApiBase` | Provider base URL. Becomes slopo's `embedding_params.api_base`. |
| `embeddingApiKey` | Optional. Passed to slopo in its environment, **never written to disk**. |
| `embeddingParamsJson` | Extra LiteLLM embedding params as a JSON object string. |
| `sourceDir` | Code to index. Defaults to the session's working directory. |
| `sourceDirExclude` | gitignore-style paths to skip. |
| `autoInstall` | Install slopo into `$DSH_HOME` with `uv` on first use. Default `true`. |
| `binary` | An explicit slopo executable, overriding discovery. |
| `stateDir` | Override the per-workspace state directory. |
| `ignoreFile` | Override the ignore file path. |
| `embeddingRequestDelay` | Milliseconds between embedding requests, for rate-limited providers. |
| `timeoutMs` | Cooperative budget for a foreground run. |

### Choose the provider prefix carefully

The provider prefix is not cosmetic — it selects which LiteLLM validator runs.

```
openai/<model>   ✗  UnsupportedParamsError: Setting dimensions is not supported
                    for OpenAI `text-embedding-3` and later models
ollama/<model>   ✓  works
```

Slopo always passes `dimensions`, and the `openai/` prefix makes LiteLLM apply
OpenAI-specific parameter validation to *any* model behind it. A local
OpenAI-compatible server therefore needs its own prefix (`ollama/`,
`hosted_vllm/`), not `openai/`. This is the single most likely thing to trip you
up on first setup, and slopo reports it only as `Error: Embedding failed.` on
stdout — the real message is in the agent log, which the tool reads for you.

### A local backend costs nothing

A local Ollama server is a good default: free, private, no API key and no
egress. Measured throughput on a small CPU box is roughly **230 ms per unique
unit** (~4/second), with repeated inputs served from cache. Budget minutes, not
seconds, for a first index of a real repository.

## Use it

Type the skill name in a message:

```
/slopo-review
/slopo-review main
/slopo-analyze-one
```

Or drive the tool directly:

| Action | What it does |
| --- | --- |
| `status` | Everything: resolved settings, index state, the binary and its version, plus a live pre-flight against the embedding endpoint. |
| `configure` | Patch the settings namespace. |
| `prepare` | `slopo index` then `slopo embed` — the slow, possibly costly phase. |
| `review` | Similar code involving Git changes (`base` defaults to `HEAD`). |
| `analyze` | Triage the whole codebase. |
| `analyze-one` | One cluster, optionally by `cluster` hash. |
| `update` | Upgrade the slopo CLI. |

`prepare` — the warm-up — starts a **background job** by default, because embedding
takes minutes and slopo writes its progress to a log file rather than to stdout,
so there is nothing to stream; the job handle carries the log path instead. The
other actions return their report directly, and any of them can be backgrounded
with `run_in_background: true` — the same parameter name `dsh-tool-bash` uses.

That asymmetry is deliberate. A completion notice is best-effort: a **killed** job
deliberately produces none, and an agent whose turn has already ended may have the
notice injected rather than be woken for it. So the action that returns a report
the caller can use immediately is worth more than a job id it has to remember to
collect — and only the genuinely long step depends on being watched.

## Where things live

Everything is under `$DSH_HOME`, and nothing is written into your repository:

```
$DSH_HOME/dsh-slopo/
├── uv-tools/                    # the uv-managed slopo install and its shim
├── uv-python/                   # the CPython 3.12 that slopo requires
└── workspaces/<name>-<hash8>/
    ├── slopo.conf.yaml          # generated; edits here are overwritten
    ├── slopo.db                 # the index and embeddings
    ├── slopo.ignore.txt         # dismissed cluster hashes
    └── slopo.agent.log          # full errors, progress
```

`$DSH_HOME` is not an arbitrary choice. In a containerised harness it is usually
the only persistent mount, while `~/.local` — where `uv tool install` puts things
by default — lives on the container's writable layer and is lost when the
container is *recreated*, not merely restarted. Setting `UV_TOOL_DIR`,
`UV_TOOL_BIN_DIR` and `UV_PYTHON_INSTALL_DIR` into `$DSH_HOME` makes the install
survive that, and installation is lazy rather than on boot so a fresh container
self-heals on first use.

Point `ignoreFile` at e.g. `<repo>/slopo.ignore.txt` if you would rather those
decisions were versioned with the code — they are a judgement about *your*
codebase, and arguably belong in review.

## What gets indexed

Slopo does **not** read `.gitignore` — it walks the tree you point it at. Pointed
at a repository root with no exclusions, it will happily parse every file under
`node_modules`. That is not a review of your code; it is a review of your
dependencies, and it is slow enough to matter.

Measured on one real repository:

| scope | files | code units | embedding at the reference rate |
| --- | --- | --- | --- |
| nothing excluded | ~1,300 | **113,768** | **~7.9 hours** |
| default excludes | 562 | 8,607 | ~36 minutes |
| default excludes + a server-only scope | **165** | **1,519** | **~6 minutes** |

So there are two independent safeguards.

**Default excludes**, on unless you turn them off: `node_modules`, `.git`,
`dist`, `build`, `out`, `coverage`, `.venv`, `venv`, `__pycache__`, `target`,
`vendor`, and minified bundles. Your own `sourceDirExclude` is **added** to
these rather than replacing them — deliberately, because a settings patch
replaces a whole value, so setting one pattern would otherwise silently
re-enable the 13x blow-up.

**A budget on the backlog.** `maxIndexUnits` (default 5,000) refuses to start an
embed that exceeds it, naming the count, the estimate and the lever:

> 113,768 code units still need embedding across 1,300 files. At the reference
> rate that is about 7.9 hours, against a budget of 5,000.

Nothing has been embedded at that point. Raise the budget, or narrow the scope —
setting `maxIndexUnits` to 0 removes the limit. The count comes from reading
slopo's own database, not from parsing its output, because `Indexed N code units`
reports only what was *newly* indexed: on a warm database it reads zero while the
backlog is still full, which is exactly the state a refusal leaves behind.

Every action indexes first so the gate runs before anything expensive, and
`status` reports both the effective excludes and the budget.

## Things worth knowing

**Five settings are frozen at first index.** `source_dir`, `embedding_model`,
`embedding_dimensions` and both node-count thresholds are written into
`slopo.db` on the first `index`, and every later run refuses to mix generations:

```
Error: Configuration mismatch.
```

Changing any of them means deleting the database and re-embedding everything —
real time, and real money on a hosted backend. The tool recognises this failure
and says so rather than relaying the two-word stderr line.

**A review cannot bootstrap the database.** `agent-review` opens the index with a
read-only open, so on a fresh workspace it fails with `No data in database. The
tool wasn't initialized.` Run `prepare` first. Indexing is pure local parsing and
free; only embedding reaches the network.

**Slopo refuses more than 1000 unembedded units** inside an agent call, to
protect agent latency. The tool translates that into an offer to run `prepare`
in the background.

**Slopo's stderr is deliberately unhelpful.** It prints one short non-revealing
line for an agent and writes the real error to the agent log. The tool reads the
log tail and maps the known failures — missing database, configuration mismatch,
oversized backlog, missing or invalid config, stale index, git failure,
embedding failure — onto a message and a next step.

**`--config-version 1` is an alarm, not boilerplate.** It makes slopo fail if its
agent-facing contract has moved on, which is the signal that the skill text in
`skills/` needs re-porting from upstream.
**The sandbox can block slopo's state.** Slopo's index, config and log live
under `$DSH_HOME`, which is outside the workspace that a `workspace-write`
sandbox permits writing to. The tool detects that and refuses *before* running,
naming both ways out, rather than letting a write fail halfway through an embed:
run the session under the `danger-full-access` preset, or set `stateDir` to a
directory inside the workspace.

**What the tool does with the sandbox policy, and why it matters.** The mounted
`ctx.shell` on a confined deployment is `dsh-bash-sandbox`, whose `resolve()`
stamps `request.sandboxPolicy ?? ctx.sandboxPolicy.resolve()`. Omitting the
policy therefore does *not* mean "unconfined" — it silently substitutes the
**deployment** default (`DSH_PERMISSION_MODE ?? 'workspace-write'`), which is not
the session's preset. That is what makes a `danger-full-access` session fail with
`SandboxUnavailableError` on a host with no sandbox runner. The tool resolves the
session's policy the way `dsh-tool-bash` does and passes it per call.

## Slopo is an external process, not a library

It is installed separately, invoked through its own CLI, and never imported or
modified. Slopo is AGPL-3.0-or-later and does not accept patches, so its
command-line interface is the only contract worth building on — and executing an
unmodified, separately installed program is a materially different proposition
from linking against it.

## Development

```sh
node --test test/*.test.js          # unit + integration, mock harness
DSH_SLOPO_E2E=1 npm test            # also drives the real CLI end to end
```

The integration tests load the real plugin module against a mock Cordis context
and drive the tool through a real shell into the real slopo binary, so a wrong
assumption about a dsh service surfaces as a failing test rather than as a plugin
that silently does nothing. Two real bugs were caught this way: a config
renderer that would write the API key to disk if handed one, and `--config-version`
placed before the subcommand, which slopo rejects with a usage error.

The browser half has no browser to run in, so its tests evaluate the bundle the
way the real module loader does — inside `window.__ModuleLoader__.load` — with a
fake `require` that answers `react` and throws on anything else. That last part is
deliberate: it checks the card stays inside the platform's seed table, which is
the constraint a hand-written client half is most likely to break. The card is
then rendered and its handlers driven, which is not a substitute for looking at
it but does cover the wiring it cannot work without.

`node_modules/@deepseek-ai` is a local symlink to the installed harness so the
`@deepseek-ai/*` imports resolve during tests. It is gitignored and is not a
declared dependency — an out-of-tree bundle that depended on an in-box package
would install a second copy that drifts from the host's.

## Releases and updating

### The workflow

`.gitea/workflows/release.yml` runs on every push to `master`:

1. works out the range since the last tag;
2. decides whether the push releases at all;
3. bumps `package.json`, prepends a `CHANGELOG.md` section, commits, and tags
   `vX.Y.Z`;
4. creates the Gitea release with those notes as the body.

The bump is a patch unless a commit in the release says otherwise:

| in a commit subject | effect |
|---|---|
| `[minor]` | release as a minor version |
| `[major]` | release as a major version |
| `[skip release]` | this push publishes nothing |

With no tags yet, the version already in `package.json` is what gets tagged — the
first release should match what the code says, not invent a number.

Release notes come from the commit subjects in the range. Conventional prefixes
(`feat:`, `fix:`, `docs:`) are grouped under headings when at least half the
commits carry one; otherwise the notes are a flat list. Prose commit subjects get
no invented headings.

The logic lives in `.gitea/scripts/release.sh` so it runs without a runner, and
`npm run test:release` exercises it — 31 checks, including an empty range, a
repository with no tags yet, and a patch number already at 9.

### Making `dsh plugin update` work

A plugin installed as a bare `git+https://…/repo.git` is resolved to a **commit**
and pinned in the profile's lockfile. Verified against a local remote: `pnpm
update <name>` *does* move that to the branch head, so the mechanism is not
broken — but it tracks every commit, including half-finished ones, and it needs
fetch access to the remote.

Adding a version selector makes it track **releases** instead:

```jsonc
// $DSH_HOME/profiles/<profile>/package.json
"dsh-slopo": "git+https://git.notexpectedyet.com/Not-Expected-Yet/dsh-slopo.git#semver:^0.1.0"
```

`#semver:^0.1.0` resolves the highest tag in range. Verified locally: installed
`v0.1.1` where the branch was already at `v0.1.2`-plus-untagged-work, and
`pnpm update` then moved it to a newly pushed `v0.1.2`.

Two things to know, both of which look like "the update did nothing":

- **The remote must be fetchable.** `pnpm update` cannot see a new tag it cannot
  fetch. If `git ls-remote <url>` asks for a username, fix that first — the same
  credential push needs.
- **A restart is required.** Client bundle bytes are snapshotted when the profile
  activates, so a hot update installs new files but keeps serving the old ones
  until `dsh --profile <profile>` comes back up.

## Licence

AGPL-3.0-or-later, matching slopo. The three skill bodies are ports of slopo's
own agent configurations rather than new prose; `NOTICE.md` records exactly what
changed and why the licence follows.