From RSI taxonomy and textual gradients to Agent Swarms, RSI foundation models, and durable improvement infrastructure

Most so-called “automated research” systems start with a verifier that has already been defined. That is only half of RSI. The other half is figuring out who—or what—creates and evolves the verifier itself.

RSI Has Two Distinct Tracks

When people discuss Recursive Self-Improvement (RSI), they often group several different problems under the same label. Separating RSI into two tracks may be more useful.

Track 1: Evolving environments and verifiers. The system does not only improve the answer. It also improves the tasks, environments, evaluation criteria, rubrics, and reward signals—in other words, it continuously refines what “better” should mean.
Track 2: Optimizing under an existing verifier. The objective is already fixed. The system modifies data, code, prompts, model architectures, or training strategies in order to improve the score.

Most current RSI methods and autoresearch harnesses primarily focus on the second track.

Karpathy’s autoresearch is a clean minimal example. An agent modifies the language-model training code, runs experiments under a fixed budget, reads the validation metric, decides whether to keep or revert the change, and then starts the next iteration.

The value of this setup is not the code itself. It is that the core RSI loop becomes extremely clear:

Form a hypothesis → modify an artifact → run an experiment → read the evidence → modify it again.

Once You Have a Verifier, RSI Becomes Search in Artifact Space

You can think of this process as a form of optimization where the “gradient” is expressed through language.

This is only an analogy. It is not literal gradient descent.

In standard deep learning, we compute a loss and update model parameters through backpropagation:

Standard gradient descent:

θₜ₊₁ = θₜ − η∇θ L(θₜ)

In verifier-driven RSI, the verifier returns a score, while the experiment system provides logs, trajectories, failure cases, and behavioral evidence. The agent reads this evidence in context and proposes the next modification.

A rough abstraction is:

Verifier-driven RSI:

aₜ₊₁ = aₜ + Δaₜ

where:

Δaₜ = π_ICL(sₜ, τ≤ₜ, fₜ)

Here, aₜ is the current artifact; sₜ is the verifier score; τ≤ₜ is the experiment trajectory up to the current round; fₜ is the observed failure feedback; and π_ICL is the update policy implemented through in-context learning.

In this analogy, ICL plays a role similar to a textual form of backpropagation. It does not calculate parameter gradients. Instead, it reads experimental feedback and determines how the artifact should be changed next.

But the analogy has important limits. The RSI update is discrete, semantic, and history-dependent. It is not differentiable, and there is no guarantee that every update will improve the score.

The code, data, or training strategy being modified is the artifact being optimized;
the verifier score is the objective signal;
failure trajectories and logs provide local evidence;
in-context learning explains the feedback and generates the next update direction.

From a search perspective, RSI also resembles a smarter form of tree search. The system explores different modification paths, observes their outcomes, and decides whether to continue, branch, or backtrack.

Unlike traditional DFS, the backtracking and branching logic is not hard-coded in software. It is dynamically determined by the model from the available context.

Adding sub-agents or multiple agents makes the search tree wider. Adding longer context gives the search a richer history. Improving the underlying model improves the quality of the hypotheses, implementations, and postmortems.

What Actually Determines Performance Under a Fixed Verifier?

Once the verifier is reliable, system performance is mainly determined by four factors:

Model capability: Can the agent form accurate hypotheses and implement clean, verifiable changes?
Context and memory: Can it retain enough experimental history to avoid repeating mistakes?
Search width: Can multiple agents explore genuinely different directions?
Evaluation throughput: How many trustworthy experiments can be run within a fixed time and cost budget?

Kimi’s Agent Swarm primarily expands the third dimension. A complex task is dynamically decomposed, multiple sub-agents explore in parallel, and their results are aggregated.

Compared with simply modifying a prompt or workflow, Agent Swarm is more orthogonal to the capability of a single model because it directly increases test-time compute and search width.

However, parallelism is not automatically useful. Different agents may explore the same direction, produce highly correlated errors, or introduce coordination overhead. Increasing search width only helps when the branches are sufficiently diverse, evaluation is reliable, and the system can preserve the best result.

Why Most Harness-Level Work May Not Be a Long-Term Moat

Harnesses are clearly useful today.

Current models forget previous experiments, revisit failed paths, overreact to noisy scores, and often lose track of the best checkpoint discovered earlier in a run. Explicit memory, branch management, rollback, budget allocation, and result selection can compensate for these weaknesses.

Our concern is the durability of these techniques.

If extremely long context windows become practical and models become better at managing long contexts, many manually designed mechanisms for summarization, reflection, role decomposition, and state management may eventually be absorbed into the model itself.

Techniques that mainly compensate for a model’s inability to remember, compare, or plan may have a limited half-life.

What is harder for the model to absorb is real parallel compute, secure and reproducible environments, reliable verifiers, training and evaluation services, experiment lineage, strict budget management, and connections to real production systems and private data.

Search Pruning Is Useful, but It Is Not the End State

For models that are not yet strong enough, explicit search control remains important.

A system can predict the value of a candidate path, stop clearly unpromising experiments early, allocate more budget to promising directions, and preserve the best checkpoint while continuing exploration.

This should sound familiar. Let's return to the idea of using a learned value signal to decide where to spend the next unit of exploration budget.

In this sense, PPO is back—not necessarily as the exact same optimizer, but as a recurring idea: use value estimation to control exploration.

However, we should not mistake today’s pruning heuristics for the final moat. As models improve, they may gradually learn to choose promising paths themselves.

The infrastructure still needs to execute the experiment, isolate the environment, control the budget, preserve the evidence, and verify the outcome.

Do We Really Need a Dedicated RSI Foundation Model?

Another view is that we should train a foundation model specifically for RSI. I think the first question should be: what unique capabilities would such a model actually contain?

Consider an RSI system with effectively infinite context and extremely strong memory. Given enough time, it could potentially reach its goal through brute-force trial and error combined with ICL.

But this is similar to the infinite-monkey analogy: with enough attempts, almost anything may eventually happen. The real problem is efficiency.

Technically, pruning a DFS search tree means predicting the value of a path. Therefore, an RSI-capable agent would need at least two important capabilities:

World modeling: Predict what an intervention will change and what side effects it may create.
Value estimation: Predict whether a state or direction is worth pursuing before paying the full cost of evaluation.

But these capabilities are not necessarily unique to RSI models. They are already related to next-token prediction during pretraining. They are also increasingly present in post-training, where models learn from executable environments, tool interactions, verifiable tasks, and environment feedback.

Recent work from Qwen, Prime Intellect, and others is moving in this direction. These systems are not complete RSI systems, but they are training general models with many of the underlying capabilities that an RSI researcher would need.

Therefore, I am not convinced that “a dedicated RSI foundation model” automatically defines a strong independent category.

A vertical expert model may perform better in a narrow research domain, but that is specialization. A truly distinct RSI foundation model would need to demonstrate capabilities that cannot naturally emerge from stronger general world modeling, value estimation, tool use, and long-context reasoning.

The More Important Direction: Evolving the Verifier and the Environment

The first track may be more valuable to explore. The system should not only improve candidate solutions. It should also improve the mechanisms that define and measure progress.

In the current research process, most benchmarks are still designed by humans. From a research perspective, one of the central questions is therefore:

How can an agent propose useful, valid, and meaningful benchmarks by itself?

This may be the core path toward evolving the objective itself.

In real-world applications, a fixed verifier makes search executable, but real products rarely begin with a perfect verifier. The objective may be incomplete, easy to hack, expensive to evaluate, delayed, or misaligned with what users actually care about.

A practical system needs to answer questions such as:

Which real failures should become new tasks?
How should those tasks be generated, expanded, and validated?
What evidence is sufficient to determine success?
How do we detect reward hacking and verifier leakage?
When should the rubric, environment, or test distribution change?

This is harder than optimizing against a known score because the system must improve the objective without destroying its meaning.

It is also closer to real product development. A verifier must encode operational reality, not only benchmark correctness.

Final Thoughts

RSI is not a single algorithm. It is a closed-loop systems problem.

Once a verifier exists, improvement increasingly looks like scalable search in artifact space: stronger models, longer memory, wider parallelism, and higher evaluation throughput.

Many harness techniques are highly valuable for current models, but they may function more as bridges over today’s capability gaps than as durable long-term moats.

The harder and more valuable problem is to build infrastructure that can withstand continuous changes in models and algorithms, while also enabling environments and verifiers themselves to evolve.

The long-term moat may not be which agent proposes the next experiment. It may be who can reliably run, verify, learn from, and operationalize millions of experiments.

This is the direction we are currently exploring: building infrastructure for continuous experimentation, environment validation, and model improvement.

If you are working on self-improving agents, autoresearch, agentic post-training, or product-facing verifier evolution, we’d be happy to connect and have a chat.

Where Does the Real Moat in Recursive Self-Improvement Lie?