Early results toward auto-research: how an agent population learns across five open-ended tasks

The key to the next generation of AI systems is not just making a single agent stronger, but enabling a group of agents to accumulate experience, share knowledge, and continuously evolve — like an organization.

Over the past year, we've seen individual coding agents, research agents, and data agents tackle increasingly complex tasks. But if we shift the lens from "can one agent solve this single task" to "can an organization get better and better at long-running work," the question changes entirely.

The intelligence of a human organization never lives in any single mind. It lives in case files, papers, codebases, postmortems, experiment logs, runbooks, and team intuition. A doctor's case notes help those who come after; an engineering team's incident review reshapes the whole development process; a growth team's campaign experience becomes the starting point for the next round of experiments.

Agents should work the same way.

We believe the true form of a self-evolving agent is not a model that "gets smart in the moment" within a single context window, but a population of agents that forms collective memory through long-running operation: every attempt, failure, fix, and breakthrough enters a shared experience layer, where later agents can read, compress, and reuse it to reach promising regions faster in the next round of exploration.

This is the direction Evolvent is building: turning agents from one-off tools into intelligent organizations with compounding memory.

From Individual to Collective Intelligence

Most agent systems today still behave like "a smart intern you hired": they can finish the task, but once it's done, the experience rarely settles naturally. Next time they spin up, the agent still has to re-read the code, run into the same pitfalls, and re-discover which approaches work.

Real organizations don't run this way. They turn experience into reusable assets: processes, knowledge bases, benchmarks, leaderboards, best practices, failure cases, domain vocabulary, decision principles. Individuals learn, and so do teams. Individual capability sets the ceiling for a single task; collective learning capability determines long-term compounding.

So we split "evolve" into two levels:

Individual evolve makes an agent more skilled; collective evolve makes the organization smarter. What's truly valuable is not "some agent happened to get it right once," but whether the system can preserve why it got it right, so the next batch of agents gets it right faster, more reliably, and more systematically.

In practice, this means agents must share not just final answers but the exploration process: which directions were tried, why they failed, which program is currently best, which strategies stay effective across tasks, which experiences should enter long-term memory. Each agent explores in its own workspace with feedback from a grader; a shared state layer continuously stores attempts, notes, skills, and leaderboard; the system periodically triggers reflect, consolidate, and pivot to turn scattered trajectories into collective knowledge the next round of agents can use.

This is not simply making the context window longer — it's adding an "collective learning loop" to the agent population.

Why AutoResearch

To validate this mechanism, we chose a set of AutoResearch-style tasks. They are not chat Q&A, nor fixed-pipeline automation, but open-ended search problems that require repeated attempts, evaluation, correction, and retry.

Such tasks are naturally suited to testing collective intelligence:

• Feedback is grounded: every submission is scored by the environment, not judged subjectively.
• The search space is large enough: a single agent is unlikely to find a high-quality path consistently in one shot.
• Experience is transferable: success patterns, failure patterns, heuristics, and code structures can all be inherited by later agents.
• The domains differ enough: if the same mechanism works across systems, algorithms, and scientific tasks, it's not a trick tailored to a single benchmark.

We observe Evolvent's collective evolution loop across five scenarios: signal processing, low-level kernel optimization, MoE load balancing, combinatorial optimization, and molecular design. They span numerical science, systems software, and AI4Sci, yet all reduce to the same loop:

an agent generates candidate solutions, the environment scores them, the system records trajectories and accumulates collective experience, and later agents continue searching from that experience.

Here is the performance span from initial seed to evolved best across the five tasks:

Here is the same set of experiments under real evaluation, plotted separately by task, with the x-axis showing the grader's attempt index. Many gains are not linear climbs; they land at jump points after several attempts, then flatten into plateaus:

Case 1: Signal Processing

Signal processing is the experiment set where "evolution" is easiest to see directly. The task is real-time adaptive signal filtering: the input is a noisy dynamic signal, and the goal is to denoise while staying as close as possible to the clean target.

The difficulty is balance. Filter too weakly and noise remains; filter too strongly and dynamic detail is wiped out. The agent must find a more stable filtering strategy among many candidate programs, correcting continuously via score feedback.

We ran two context-pool experiments, one with Qwen and one with Claude. Both show best-so-far rising as attempts progress, with later turns starting from a higher point. This suggests the experience the system has accumulated is already influencing new exploration — rather than every agent starting from scratch.

The core here is not "a model wrote a decent piece of filtering code," but that the system begins to form task-level intuition: which smoothing strategies lose dynamics, which parameters are more stable, which structures are worth continued local search. Once these intuitions enter the shared experience layer, the exploration starting point of later agents is raised across the board.

Case 2: Low-Level Kernel Optimization

Kernel Builder is a lower-level systems problem. Given a kernel, the agent must reduce cycles through scheduling, vectorization, and memory-access optimization. The feedback here is highly engineering-driven: a seemingly tiny code change can alter the entire execution pipeline.

This kind of problem is ideal for demonstrating the value of collective intelligence. A single agent might stumble onto a local optimization — reducing certain loads/stores, or adjusting SIMD scheduling. But if those discoveries stay inside one run, they're wasted the next time. The point of collective evolve is to preserve these small but expensive discoveries: which optimization directions truly help, which introduce scheduling conflicts, which code shapes are more readily accepted by the hardware execution model.

When later agents inherit these patterns, they're no longer randomly mutating code but continuing to search atop existing engineering experience. For systems software and performance optimization this matters especially, because real progress usually comes from accumulating many small lessons rather than a single "big idea."

Case 3: ADRS / EPLB

EPLB is the expert load-balancing problem in MoE serving. In real systems, some experts become hotspots, causing uneven GPU load and rising tail latency. The agent's task is to decide which experts to replicate and how to place them across GPUs for more balanced load.

The crux is global trade-off. Fixating on local hotspots may just migrate the bottleneck to another GPU; chasing average load alone may ignore long-tail pressure in real workloads. A single attempt can only show a strategy works on one workload; layering many attempts lets the system distill more robust strategies: replicate hot experts first, don't chase local balance only, watch for cross-GPU bottleneck migration.

This is the difference between collective experience and a single result. A single result tells you "how to place it this time"; collective experience tells later agents "what to prioritize when facing similar load shapes."

Case 4: Frontier-CS / Polyominoes

Polyominoes is a combinatorial optimization task: given a set of irregular puzzle pieces, the agent must pack them as tightly as possible into a grid. Unlike numerical optimization, there's no smooth gradient — it depends more on search strategy, local repair, and constraint handling.

The experience in such tasks is usually not a single formula but a set of heuristics: place large pieces first, reduce holes, back off locally, use boundary shapes for subsequent filling, avoid early choices that create unfixable gaps. A single agent can form these skills in its own trajectory; at the collective level, effective heuristics are written into shared notes to help later agents find feasible layouts faster.

Combinatorial optimization rarely has an answer that's "obviously correct at a glance." It takes patience, trial and error, and memory. The value of collective intelligence is to keep that trial and error from being wasted.

Case 5: Drug Design

Drug Design is an AI4Sci-style molecular optimization task. The agent must search the candidate molecule space for higher-scoring structures while satisfying validity, similarity, and novelty constraints.

This task is representative because the score doesn't just rise at random. As evolution progresses, candidates gradually lean toward certain more stable scaffold families. In other words, what the system records is not only "which molecule scored high" but also "which structural edit directions are more likely to yield high scores."

For AI4Sci this is crucial. A genuinely useful system shouldn't just spit out the highest-scoring candidate; it should help researchers understand the search process: which directions have been explored, which structural families are more promising, which constraints limit further gains. Collective experience keeps this information from scattering across attempts and instead organizes it into knowledge the next round of research can reuse.

The Flywheel of Collective Intelligence

Putting these cases together, what we really want to demonstrate is not "we got a decent score in some environment," but that a more fundamental loop is starting to hold:

1. Agents explore different candidate solutions in parallel.
2. The environment and grader give comparable feedback.
3. The system saves full trajectories, best programs, failure cases, and intermediate observations.
4. Experience is organized into notes, skills, synthesis, and leaderboard.
5. Later agents read this experience and continue searching from a higher starting point.

Each turn of this loop adds a bit more experience to the organization; the more experience, the less waste in the next round of exploration; the more effective the exploration, the more reusable patterns the system discovers. What ultimately forms is not "an agent that can do one thing," but an agent organization that keeps learning.

We believe this will become an important form of enterprise agent systems. What businesses truly need is not a one-off demo, but a network of agents that, after long-running operation, understands the business, the systems, and collective preferences better and better. Ad delivery, growth operations, industry research, R&D automation, data analysis, experiment design — these are all the same kind of problem: tasks recur, environments change, experience must be retained, and the organization needs to get stronger the longer it runs.

Evolvent's goal is to bring this long-term compounding into agent systems.

Not making a single agent perform more like an expert once, but letting a group of agents gradually form a truly learning, collaborating, evolving intelligent organization through continuous work.