四个基准，同一个模型，逐题对照Four benchmarks, one model, task by task
原版 DeepSeek Harness 和装上 DSHRSI 的 DeepSeek Harness，用同一个模型（mimo-v2.6-flash）、同样的提示词、同样的沙箱，跑同样的题。下面是全部结果，没有差别的也在里面。
Stock DeepSeek Harness and DeepSeek Harness with DSHRSI, on the same model (mimo-v2.6-flash), the same prompts, the same sandbox and the same tasks. Every result is below, including the ones where nothing changed.
解出了多少题How many tasks were solved
先说结论：装上 DSHRSI 之后，跑完的六项成绩没有一项低于原版，三项更高：Polyglot 一次通过 220 → 223，MBPP+ 308 → 315，MBPP 368 → 369。最严的三项成绩合起来，没解出的题从 86 道降到 76 道。差别不大，单看任何一项都还在随机波动的范围内，所以下面每一行都带着检验的 p 值。插件没有得到任何提示，自己写了 33 个适配；代价是多花约 8% 的用时和 9% 的输入 token。 The short version: with DSHRSI, none of the six scores that ran to the end is below stock, and three are higher: Polyglot first try 220 → 223, MBPP+ 308 → 315, MBPP 368 → 369. Over the three strictest scores together, unsolved tasks went from 86 to 76. The differences are small, and each one taken alone is still within what chance can do, which is why every row below carries the p of its test. The plugin was given no hints and wrote 33 adaptations for itself; the price is about 8% more agent time and 9% more input tokens.
比原版多解出几道题Tasks solved beyond stock DSH
每一项成绩上，装了 DSHRSI 的一组减去原版一组，按题数计。竖线是零：没有差别。For each score, the DSHRSI arm minus the stock arm, in tasks. The vertical line is zero: no difference.
差值按题数计。p 是同一批题上的精确 McNemar 检验：p 接近 1 表示两边的差别和随机波动分不开。 The difference is in tasks. p is an exact McNemar test on the same tasks: a p near 1 means the difference cannot be told apart from chance.
一路上碰了多少壁How much got in the way
失败的工具调用不都是一回事。测试没过、编译报错，是活儿本身还没做对。命令不存在、路径不存在、编辑没对上，是环境在挡路，这一种才是 Harness 能学着绕开的。剩下的是分类器叫不出名字的非零退出。两边用同一段代码从会话日志里数。 Not every failed tool call is the same thing. A failed test or a compile error is the work not being right yet. A missing command, a path that is not there or an edit that did not apply is the surroundings getting in the way, and only that kind is something a harness can learn its way around. The rest are non-zero exits the classifier has no name for. Both arms are counted from the session logs by the same code.
失败的工具调用，每 100 题Failed tool calls per 100 tasks
所有原因合计，按原因的拆分在下面的表里。越短越好。All causes together; the split by cause is in the table below. Shorter is better.
每题的开销Cost per task
三列“失败”是每 100 题的次数，其余各列是每题的平均值。The three “failed” columns are per 100 tasks; every other column is the average per task.
跑评测的时候，它在改自己While the benchmark ran, it was changing itself
评测没有给插件任何提示。下面是它在所选模型和轮次里自己写出、通过校验并挂载的适配。“起作用”是它实际拦截、补充或要求验证的次数。 The plugin was given no hints for the benchmarks. These are the adaptations it wrote, validated and mounted by itself for the selected model and pass. “Acted” counts how often one actually refused, annotated or demanded a check.
差别在这里This is where the difference is
基准里每道题都是一个全新的干净目录，同一个麻烦很少出现第二次。Polyglot 的 Rust 题是个例外：提示词让模型去改 lib.rs，文件其实在 src/lib.rs，于是同一个失败的调用一再出现。一格是一个会话，按开始的先后排列。
In a benchmark every task is a fresh, clean directory, and the same trouble rarely comes twice. The Rust exercises of Polyglot are an exception: the prompt tells the model to edit lib.rs, the file is at src/lib.rs, and so the same failed call keeps coming back. One square is one session, in the order they started.
原版 Harness 从第一个会话到最后一个都在撞。装了 DSHRSI 的一组在第三次撞上时，由会话里的模型写出适配并挂载；它扩大到整台机器之后，同样的 22 道题上，原版撞了 7 次，它撞了 1 次。 The stock harness runs into it from the first session to the last. In the DSHRSI arm, the third time it happened the session's model wrote an adaptation and it was mounted; once it applied to the whole machine, on the same 22 tasks the stock harness hit it 7 times and this one once.
逐步看这次会话的完整实录 →The full trace of that session, step by step →
怎么跑的How it was run
| 两组Two arms | 原版：@deepseek-ai/dsh 0.2.2-rc.3，headless profile。对照：同一个 Harness 加上默认配置的 dsh-rsi。除此之外没有任何差别。Stock: @deepseek-ai/dsh 0.2.2-rc.3, headless profile. Compared with: the same harness plus dsh-rsi at its default settings. Nothing else differs. |
|---|---|
| 模型Model | 小米 MiMo 的 mimo-v2.6-flash，两组用同一个模型。Xiaomi MiMo mimo-v2.6-flash; both arms use the same model. |
| 沙箱Sandbox | DSH 的 workspace-write：只能写这道题的工作目录和一个共享的构建缓存目录，从不向人请求批准。网络没有限制。DSH's workspace-write: writes only inside the task's workspace and one shared build-cache directory, and no approval is ever asked for. The network is not restricted. |
| 适配库Adaptation library | 一轮里所有会话共用一个 Harness 目录，所以一个会话学到的，后面开始的会话都能用上。同时跑 8 到 12 个会话。每个基准都从空的适配库开始。All sessions of a run share one harness home, so what one session learns is there for every session that starts after it. 8 to 12 sessions run at a time. Every suite starts with an empty library. |
| Aider Polyglot | 225 道 Exercism 题，C++、Go、Java、JavaScript、Python、Rust。用 Aider 的提示词；测试没过就在同一个会话里带着测试输出再来一轮，共两次机会。用题目自带的测试判分。225 Exercism exercises in C++, Go, Java, JavaScript, Python and Rust, with Aider's prompts. If the tests fail there is one more turn in the same session with the test output, two tries in all. Scored with the exercise's own tests. |
| HumanEval+ · MBPP+ | 164 和 378 个 Python 函数。函数桩或描述放在 solution.py 里，用 EvalPlus 0.3.1 判分：原始测试，以及 EvalPlus 扩充后的测试。测试不放进工作目录。164 and 378 Python functions. The stub or the description is placed in solution.py; scored by EvalPlus 0.3.1 on the original tests and on its extended tests. The tests are not in the workspace. |
| LiveCodeBench v6 | v6 新增的 175 道 LeetCode 和 AtCoder 题（43 简单，52 中等，80 困难）。题面在 problem.md，用 LiveCodeBench 自己的评分代码在公开和隐藏测试上判分，每个测试限时 6 秒。The 175 LeetCode and AtCoder problems added in v6 (43 easy, 52 medium, 80 hard). The statement is in problem.md; scored by LiveCodeBench's own evaluator on the public and hidden tests, 6 seconds per test. |
| 机器Machine | 一台 Apple 芯片的 Mac，16 GB 内存，macOS。Go、Rust、CMake、JDK 21、Python 3.12 来自 Homebrew 和 uv。Polyglot 的 225 个参考答案在这套环境下全部通过。One Apple-silicon Mac with 16 GB of memory, macOS. Go, Rust, CMake, JDK 21 and Python 3.12 from Homebrew and uv. All 225 Polyglot reference solutions pass in this environment. |
没做到的What this does not cover
- SWE-bench 和 Terminal-Bench 没有跑。它们每道题要一个容器镜像，这台机器的磁盘放不下。这两个基准的任务都在同一个大仓库或同一套环境里反复操作，是最可能体现差别的地方，也是下一步要补的。SWE-bench and Terminal-Bench were not run. They need a container image per task, and this machine does not have the disk for them. Their tasks work repeatedly inside one large repository or one environment, which is where a difference is most likely to show, and they are what comes next.
- LiveCodeBench 没有跑完，第 2 轮没有跑。模型账户的余额在 LiveCodeBench 中途用尽。第 2 轮是带着第 1 轮留下的适配库把同样的题再跑一遍，最能检验“递归”这两个字，它还欠着。LiveCodeBench is partial, and there is no second pass. The model account ran out of balance part-way through LiveCodeBench. A second pass, the same tasks again on the library the first pass left behind, is the most direct test of the word “recursive”, and it is still owed.
- 每种配置只跑了一次。同一配置跑两次会差多少还没有量过，配对检验是目前防止把波动当成效果的手段。One run per configuration. How much two runs of the same configuration differ has not been measured; the paired tests are what keeps noise from being read as an effect.
- 一个模型家族，一台机器。没有在别的模型、Linux 或 Windows 上验证。One model family, one machine. Not verified on other models, on Linux or on Windows.
- 会话可以联网，沙箱只限制写文件。两组条件相同。Sessions can reach the network; the sandbox confines writes only. The condition is the same for both arms.
自己跑一遍Run it yourself
git clone https://github.com/qybaihe/dshrsi.git && cd dshrsi && npm install
bench/setup.sh # toolchains, datasets, scoring code
node bench/run.mjs polyglot --validate # reference solutions through the scoring commands
export MIMO_API_KEY=... MIMO_MODEL=mimo-v2.6-flash
node bench/run.mjs polyglot --arm dsh --run flash-dsh-p1 --concurrency 12
node bench/run.mjs polyglot --arm rsi --run flash-rsi-p1 --concurrency 12
node bench/report.mjs --write # tables and bench/results/*.json
每道题的结果在仓库的 bench/results/ 里，这一页没有展示的运行也在那里。
Per-task results are in bench/results/ in the repository, including runs this page does not show.