递归自我改进,落在 Harness 这一层Recursive self-improvement, at the level of the harness
递归自我改进说的是:一个系统用自己的能力改进自己,改进之后的系统再去做下一轮改进。DSHRSI 把这个闭环接在智能体真正干活的地方,也就是 Harness 上。 Recursive self-improvement means a system uses its own abilities to improve itself, and the improved system carries out the next round. DSHRSI closes that loop where an agent does its work: in the harness.
要同时满足三件事,才配叫 RSIThree things have to hold at once
改的是自己It changes itself
不是人在外面调参数,也不是另一个系统来训练它。动手的和被改的是同一个系统。No one tunes it from outside and no other system trains it. What acts and what is changed are the same system.
改完确实更好The change is an improvement
“更好”要能检验。改动先过关,装上之后还要拿后来的运行来对账。Better has to be checkable. A change passes a gate first, and later runs are held against it afterwards.
这一轮的产出是下一轮的起点One round's output starts the next
改进不是做一次就停。改过的系统继续运行,继续发现问题,继续改,改进叠在改进上。It does not stop after one change. The changed system keeps running, keeps finding trouble and keeps changing, improvement on top of improvement.
四个环,一个套一个Four loops, each inside the next
每个环的产出,都是外面那个环的输入。What one loop produces is what the loop around it works with.
-
环一:一轮任务之内Loop one: inside a turn
同一个调用失败两次,Harness 当场让模型写一段适配,过校验关,下一次模型请求之前就挂上。这一轮剩下的部分,已经跑在改过的 Harness 上。The same call fails twice. The harness has the model write an adaptation on the spot, passes it through the gate, and mounts it before the next model request. The rest of the turn already runs on the changed harness.
-
环二:会话与会话之间Loop two: from session to session
下一次会话一开始就带着适配库里的全部内容。它遇到的新麻烦,是在已经改过的 Harness 上遇到的;它学到的东西,又成为再下一次会话的起点。以前的会话见过同一个麻烦,这次动手的门槛就更低。The next session starts with everything in the library. The new trouble it meets, it meets on a harness that has already changed, and what it learns is where the session after it starts. Trouble that earlier sessions met lowers the threshold for acting on it.
-
环三:改进本身被改进Loop three: the improvement gets improved
适配装上之后,同一个麻烦在装着它的会话里又出现两次,就触发修订。模型拿到现有源码和“它没拦住”的证据,写出更严的一版,旧版留着可以回滚。起过作用、没出过错的转正;出错三次的退回上一版,没有上一版才隔离。When the same trouble occurs twice in sessions that had the adaptation mounted, a revision starts. The model gets the current source and the evidence that it did not hold, and writes a stricter version; the old one is kept for rollback. What acts without faults is promoted. What throws three times goes back to the version before it, and is quarantined only when there is none.
-
环四:从一个项目到一台机器,再到一个团队Loop four: from a project to a machine to a team
在第二个工作区碰到同一个麻烦,适配的作用范围从工作区扩大到全机。项目层的适配转正后写进仓库的
.dsh-rsi/,同事接受之后,他们的 Harness 从这一版继续往下改。Met in a second workspace, an adaptation widens from the workspace to the whole machine. A promoted project-layer adaptation is written to.dsh-rsi/in the repository; once colleagues accept it, their harness continues from that version.
同样是“让智能体变好”,改的地方不一样Three places to make an agent better
| 上下文层:提示词、记忆、SkillContext: prompts, memory, skills | 权重层:微调Weights: fine-tuning | Harness 层:DSHRSIHarness: DSHRSI | |
|---|---|---|---|
| 改的是什么What changes | 模型读到的文字Text the model reads | 模型参数Model parameters | 围绕模型运行的代码Code that runs around the model |
| 多久生效Takes effect | 下一次请求Next request | 一次训练之后After a training run | 下一次请求,同一轮任务之内Next request, inside the same turn |
| 能不能强制Can it enforce | 不能,模型可以不照做No, the model may not follow it | 不能保证No guarantee | 能,在工具调用执行之前拒绝Yes, a call is refused before it runs |
| 谁来写Who writes it | 人,或模型自己记笔记A person, or the model taking notes | 训练流程A training pipeline | 会话里的模型,经过校验关The session's model, through a gate |
| 看得懂、撤得回吗Readable, reversible | 看得懂,撤得回Both | 都很难Neither, in practice | 一个文件一个适配,可关、可回滚、可删除One file per adaptation: switch off, roll back, delete |
实测里,模型在被明确告知之后仍有 3 次会话想跑那条不存在的命令。提示词拦不住的这 3 次,都在执行前被 Harness 拒绝了。In the measured runs, the model still tried the missing command in 3 sessions after it had been told. A prompt could not stop those three; the harness refused each one before it ran.
哪些地方没有递归,以及为什么What is not recursive, and why
把话说满很容易。下面这些是 DSHRSI 现在不做的。Overclaiming would be easy. These are the things DSHRSI does not do today.
模型权重不动The model's weights do not change
没有训练,没有微调。变的只是 Harness。换一个模型,学到的适配照样生效。No training and no fine-tuning. Only the harness changes. Swap the model, and what was learned still applies.
裁判不改自己The judge does not rewrite itself
判断点、触发策略和校验关是固定的代码,适配改不了它们。这是有意的:能自我修改的部分,必须由不能被它修改的部分来把关。Detectors, the trigger policy and the validation gate are fixed code that no adaptation can change. That is deliberate: the part that modifies itself has to be checked by a part it cannot modify.
只能收紧,不能放行It can tighten, never loosen
适配可以补充信息、拒绝、询问、要求验证。没有任何一种能力可以放行 Harness 本来会拒绝的操作。改写结果、新增工具、执行命令要等人批准。An adaptation can add information, refuse, ask, or demand verification. No capability allows what the harness would refuse. Rewriting results, adding tools and running commands wait for a person's approval.
证据还很少The evidence is still thin
只在一个模型系列、一台机器上跑过,每种配置一次。在题目互相独立、环境干净的基准上,同一个麻烦很少出现第二次,所以解出的题数只多出一点,单看任何一项都还在随机波动之内,这一点评测页写得很直白。差别最清楚的地方是反复出现的麻烦,实录页有一个完整的例子。Runs so far cover one model family and one machine, once per configuration. On benchmarks whose tasks are independent and whose environment is clean, the same trouble rarely comes twice, so the gain in tasks solved is small and each score taken alone is still within chance; the evaluation page says so plainly. Where the difference is clearest is trouble that comes back, and the trace page has one complete example.