RSI · Recursive Self-Improvement

递归自我改进,落在 Harness 这一层Recursive self-improvement, at the level of the harness

递归自我改进说的是:一个系统用自己的能力改进自己,改进之后的系统再去做下一轮改进。DSHRSI 把这个闭环接在智能体真正干活的地方,也就是 Harness 上。 Recursive self-improvement means a system uses its own abilities to improve itself, and the improved system carries out the next round. DSHRSI closes that loop where an agent does its work: in the harness.

先把词说清楚The term, first

要同时满足三件事,才配叫 RSIThree things have to hold at once

01 · SELF

改的是自己It changes itself

不是人在外面调参数,也不是另一个系统来训练它。动手的和被改的是同一个系统。No one tunes it from outside and no other system trains it. What acts and what is changed are the same system.

02 · IMPROVEMENT

改完确实更好The change is an improvement

“更好”要能检验。改动先过关,装上之后还要拿后来的运行来对账。Better has to be checkable. A change passes a gate first, and later runs are held against it afterwards.

03 · RECURSIVE

这一轮的产出是下一轮的起点One round's output starts the next

改进不是做一次就停。改过的系统继续运行,继续发现问题,继续改,改进叠在改进上。It does not stop after one change. The changed system keeps running, keeps finding trouble and keeps changing, improvement on top of improvement.

递归在哪Where the recursion is

四个环,一个套一个Four loops, each inside the next

每个环的产出,都是外面那个环的输入。What one loop produces is what the loop around it works with.

Generations of the harness: each session runs on the harness the sessions before it left behind G0 原版 DSHStock DSH 什么都还没学Nothing learned yet G1 + 适配 A+ adaptation A 会话 3 当轮写出并挂载Written and mounted in session 3 A v1 G2 A 被重写,+ BA rewritten, + B A 没拦住两次,改严了A did not hold twice: stricter now A v2 B v1 G3 … 写进仓库,传给别人In the repository, for others 别的机器从 G3 接着往下改Other machines continue from G3 .dsh-rsi/ 每一代 Harness 跑出来的记录,决定下一代怎么改The record each generation leaves decides how the next one changes 会话 1, 2, 3 … 都跑在前面的会话留下的那一版 Harness 上Sessions 1, 2, 3 … each run on the harness the sessions before them left behind
  1. 环一:一轮任务之内Loop one: inside a turn

    同一个调用失败两次,Harness 当场让模型写一段适配,过校验关,下一次模型请求之前就挂上。这一轮剩下的部分,已经跑在改过的 Harness 上。The same call fails twice. The harness has the model write an adaptation on the spot, passes it through the gate, and mounts it before the next model request. The rest of the turn already runs on the changed harness.

  2. 环二:会话与会话之间Loop two: from session to session

    下一次会话一开始就带着适配库里的全部内容。它遇到的新麻烦,是在已经改过的 Harness 上遇到的;它学到的东西,又成为再下一次会话的起点。以前的会话见过同一个麻烦,这次动手的门槛就更低。The next session starts with everything in the library. The new trouble it meets, it meets on a harness that has already changed, and what it learns is where the session after it starts. Trouble that earlier sessions met lowers the threshold for acting on it.

  3. 环三:改进本身被改进Loop three: the improvement gets improved

    适配装上之后,同一个麻烦在装着它的会话里又出现两次,就触发修订。模型拿到现有源码和“它没拦住”的证据,写出更严的一版,旧版留着可以回滚。起过作用、没出过错的转正;出错三次的退回上一版,没有上一版才隔离。When the same trouble occurs twice in sessions that had the adaptation mounted, a revision starts. The model gets the current source and the evidence that it did not hold, and writes a stricter version; the old one is kept for rollback. What acts without faults is promoted. What throws three times goes back to the version before it, and is quarantined only when there is none.

  4. 环四:从一个项目到一台机器,再到一个团队Loop four: from a project to a machine to a team

    在第二个工作区碰到同一个麻烦,适配的作用范围从工作区扩大到全机。项目层的适配转正后写进仓库的 .dsh-rsi/,同事接受之后,他们的 Harness 从这一版继续往下改。Met in a second workspace, an adaptation widens from the workspace to the whole machine. A promoted project-layer adaptation is written to .dsh-rsi/ in the repository; once colleagues accept it, their harness continues from that version.

为什么是 Harness 层Why the harness

同样是“让智能体变好”,改的地方不一样Three places to make an agent better

上下文层:提示词、记忆、SkillContext: prompts, memory, skills 权重层:微调Weights: fine-tuning Harness 层:DSHRSIHarness: DSHRSI
改的是什么What changes模型读到的文字Text the model reads模型参数Model parameters围绕模型运行的代码Code that runs around the model
多久生效Takes effect下一次请求Next request一次训练之后After a training run下一次请求,同一轮任务之内Next request, inside the same turn
能不能强制Can it enforce不能,模型可以不照做No, the model may not follow it不能保证No guarantee能,在工具调用执行之前拒绝Yes, a call is refused before it runs
谁来写Who writes it人,或模型自己记笔记A person, or the model taking notes训练流程A training pipeline会话里的模型,经过校验关The session's model, through a gate
看得懂、撤得回吗Readable, reversible看得懂,撤得回Both都很难Neither, in practice一个文件一个适配,可关、可回滚、可删除One file per adaptation: switch off, roll back, delete

实测里,模型在被明确告知之后仍有 3 次会话想跑那条不存在的命令。提示词拦不住的这 3 次,都在执行前被 Harness 拒绝了。In the measured runs, the model still tried the missing command in 3 sessions after it had been told. A prompt could not stop those three; the harness refused each one before it ran.

边界Where it stops

哪些地方没有递归,以及为什么What is not recursive, and why

把话说满很容易。下面这些是 DSHRSI 现在不做的。Overclaiming would be easy. These are the things DSHRSI does not do today.

模型权重不动The model's weights do not change

没有训练,没有微调。变的只是 Harness。换一个模型,学到的适配照样生效。No training and no fine-tuning. Only the harness changes. Swap the model, and what was learned still applies.

裁判不改自己The judge does not rewrite itself

判断点、触发策略和校验关是固定的代码,适配改不了它们。这是有意的:能自我修改的部分,必须由不能被它修改的部分来把关。Detectors, the trigger policy and the validation gate are fixed code that no adaptation can change. That is deliberate: the part that modifies itself has to be checked by a part it cannot modify.

只能收紧,不能放行It can tighten, never loosen

适配可以补充信息、拒绝、询问、要求验证。没有任何一种能力可以放行 Harness 本来会拒绝的操作。改写结果、新增工具、执行命令要等人批准。An adaptation can add information, refuse, ask, or demand verification. No capability allows what the harness would refuse. Rewriting results, adding tools and running commands wait for a person's approval.

证据还很少The evidence is still thin

只在一个模型系列、一台机器上跑过,每种配置一次。在题目互相独立、环境干净的基准上,同一个麻烦很少出现第二次,所以解出的题数只多出一点,单看任何一项都还在随机波动之内,这一点评测页写得很直白。差别最清楚的地方是反复出现的麻烦,实录页有一个完整的例子。Runs so far cover one model family and one machine, once per configuration. On benchmarks whose tasks are independent and whose environment is clean, the same trouble rarely comes twice, so the gain in tasks solved is small and each score taken alone is still within chance; the evaluation page says so plainly. Where the difference is clearest is trouble that comes back, and the trace page has one complete example.