SIA: Self Improving AI with Harness & Weight Updates

Prior work turns one knob, the scaffold or the weights. SIA turns both in a single loop, and shows the two levers reach gains neither reaches alone.

SIA is a self-improvement loop that does in one system what prior work split into two camps: it updates both the agent's harness (the scaffold of prompts, tools, retry logic, and parsers around a model) and the model's own weights, with no human in the loop after setup.

A language-model "Feedback-Agent" reads the full execution trace of a task-specific agent and decides, step by step, whether to rewrite the scaffold or fire an RL weight update. Tested on three deliberately unrelated tasks (Chinese legal classification, GPU kernel optimisation, single-cell RNA denoising), the combined approach beats scaffold-only iteration every time and clears prior state-of-the-art on all three. The framing the paper sells: the harness changes how the agent searches; weight updates change what the model knows.

The problem it attacks

The paper opens with a blunt claim: humans are the bottleneck in improving AI. People design and post-train the models, and people scaffold, prompt, and debug the agents wrapped around them. Work on automating this has split into two camps that mostly ignore each other.

• Silo 1, harness/scaffold self-improvement. A meta-agent rewrites the task agent's scaffold (system prompt, tool dispatch, retry policy, answer extraction) across generations, weights frozen. Representatives: Darwin Gödel Machine, Meta-Harness, Hyperagents. The recurring finding is that scaffold edits tend to be software-engineering hygiene and rarely add reasoning the base model couldn't already produce.
• Silo 2, test-time post-training. A hand-written RL pipeline updates the model's weights on task feedback, harness pinned to one template. Representatives: TTRL, the Discover line of test-time training. Gains come from internal policy change, but the pipeline is human-engineered and doesn't adapt to task structure.

The gap SIA targets: harness work leaves the model fixed; test-time training leaves the harness fixed. Nobody turns both knobs in one loop.

How SIA works

SIA runs a configurable loop over three LLM components. The Meta-Agent initialises the scaffold from the task spec. The Task-Specific Agent executes against the dataset, producing a trajectory. The Feedback-Agent watches that trajectory and picks one of two actions: a harness update (evolve the scaffold, weights fixed) or a weight update (RL via a method it chooses, scaffold fixed). The decisive design choice is that the Feedback-Agent receives the full trajectory, every prompt, model response, tool call, tool result, and extracted answer, not just an aggregate metric. That lets it diagnose specific failure modes instead of reacting to a single number.

The agent it edits is itself decomposed into five named parts, and "harness" refers to everything except the weights: the LLM (weights θ), the system prompt, the tool-dispatch logic (Python that parses tool calls and routes them to handlers), the answer-extraction code (turns a model response into a benchmark-formatted prediction), and the grader (the deterministic verifier that computes per-instance reward). The two meta-agents are formalised as functions: the Meta-Agent generates the first scaffold from the task spec and any reference implementations, A₁ = M(U, R), and the Feedback-Agent synthesises the next one from the last scaffold, its trajectory, and its metrics, A_g+1 = F(A_g, τ_g, E_g, U).

Every generation runs the same three-phase protocol. Execution: the scaffold runs on the dataset inside a sandbox (read-only dataset, read/write working directory) and the trajectory is captured. Analysis: the Feedback-Agent reads the scaffold source, the trajectory, the metrics, and optionally a few sample task descriptions to discourage single-instance overfitting. Improvement: it emits two artefacts, a prose improvement report explaining the proposed changes, and the next-generation agent. The Meta-Agent is also conditioned on a diverse set of task specifications when it writes the first scaffold, which the authors call sample-task regularisation, to keep A₁ from overfitting one benchmark instance.

Across all three tasks the Feedback-Agent began with scaffold iteration and switched to weight updates once harness progress stalled. The paper stresses these are soft labels, not rigid phases: the two update types are interleaved freely, and an example run looks like H, H, H, W, H, W, W rather than a clean split.

Choosing the RL algorithm

One of the more interesting parts of the paper: the Feedback-Agent does not run a fixed RL procedure. It selects an algorithm from the reward structure it observes. The paper reports which algorithm appeared on each task, plus a broader menu of patterns seen across an unpublished task set (a fuller treatment of selection is deferred to a future version). The throughline is that each method is matched to the shape of the reward histogram and the cost of a rollout, not to a fixed schedule.

Results

Three tasks were chosen for contrast across law, systems, and biology, and because they allow direct comparison to prior published results. The "initial" score is the base model filtered through the Meta-Agent's first scaffold, before any feedback iteration. SIA-H is the harness-only best; SIA-W+H adds weight updates.

The paper also runs two off-the-shelf coding agents on the same tasks as reference points: Codex (Codex 5.5) and Claude Code (Opus 4.7). Both sit far below SIA, which is the case the authors make for a dedicated self-improvement loop over a strong general agent.

SIA-W+H beats SIA-H on every task, confirming the paper's central thesis. The gains over prior SOTA:

LawBench: 191-class Chinese criminal charge classification

Given a case summary, the model picks the correct charge from 191 fine-grained categories where random guessing is right under 1% of the time. The harness phase built a TF-IDF + LinearSVC pipeline and tuned it to 50.0% (a 36.5 pp gain), then stalled. With a clean scalar reward and cheap parallel rollouts, the Feedback-Agent applied PPO with GAE and pushed accuracy to 70.1%, another 20.1 pp on top of harness-only.

AlphaEvolve TriMul: CUDA kernel optimisation

The agent must write a custom CUDA kernel for the triangular multiplicative update from AlphaFold2's Evoformer, on an H100. It is memory-bandwidth-limited with triangular sparsity, so standard dense-matrix tricks fail. The harness phase reached 12,483 µs (1.14×) then plateaued. Because most kernels fail to compile, the Feedback-Agent used entropic advantage weighting to up-weight rare high-reward rollouts. The model internalised H100-specific patterns (shared-memory tiling, fp32 register accumulation, block-size selection), driving runtime to 1,017 µs, a 14.02× speedup and a 91.9% reduction from the harness-only peak.

MAGIC scRNA-seq denoising: single-cell RNA imputation

The agent tunes MAGIC's coupled hyperparameters on sparse single-cell data. The harness phase plateaued at mse_norm 0.241. Using GRPO, the first weight-update checkpoint introduced a structural change the scaffold loop never produced across all its iterations: a two-line post-processing step (np.clip + np.rint) that rounds imputed counts to non-negative integers, enforcing a biological invariant. That lifted mse_norm to 0.289. It is the cleanest demonstration of the paper's mechanism claim: the improvement was a code-level invariant from gradient pressure, not a hyperparameter, and it never showed up in any scaffold edit.

What each lever changes

Harness iteration produces externalised changes (new tools, tighter parsers, search procedures, retry policies) while the model checkpoint stays fixed. Across tasks the Feedback-Agent built an SVC re-ranker on LawBench, a compilation-error parser and timing harness on TriMul, and a batched configuration driver on denoising. All software-engineering improvements to how the scaffold mediates between model and environment.

Weight updates produce internalised knowledge, domain-specific patterns baked into the model's parameters that no scaffold edit reaches. On LawBench, sharper disambiguation of adjacent charge categories with no prompt hint. On TriMul, H100-specific kernel patterns the base model never produced regardless of scaffold quality. On denoising, the np.clip + np.rint invariant.

Where SIA sits among prior work

The paper classifies systems on two axes: does it edit the harness, and does it edit the weights? SIA claims to be the only entry that does both in a single self-improving loop.

The closest neighbours: Hyperagents makes the meta-mechanism itself editable but keeps weights fixed. EUREKA combines an LLM-generated reward function with RL training, but the loop is one-directional, the reward generator isn't updated by the trained policy. SIA's Feedback-Agent instead selects between scaffold and weight updates in a closed loop, each informed by trajectories from the current state of both components.

Limitations

The paper flags one sharp limitation: coupled co-evolutionary Goodhart. Both levers optimise against the same fixed verifier V, and each pass reshapes the distribution the other sees. The harness finds scaffolds easy for the current policy to exploit; the weights train on data collected through a scaffold that is about to change. The joint fixed point is a Nash equilibrium between two optimisers blind to each other's update history, not a point that maximises V on out-of-distribution scaffolds or novel policies. So the reported gains carry a robustness asterisk: they are measured against the same verifier both levers optimised against, and generalisation is not established.

Future work

The authors propose two directions. The first, meta-RL over the action-selection policy, is the one that matters for recursive self-improvement: the Feedback-Agent currently picks between harness and weight updates with a frozen LLM prior. The principled version treats the selector itself as learnable, running SIA across a distribution of tasks, treating each (trajectory, action, outcome) triple as a transition in an outer MDP, and training the selector via RL on it. That would make the improvement mechanism itself self-improving, a genuinely recursive structure with open stability questions distinct from single-level RL. The second, finer-grained interleaving, would let the Feedback-Agent trigger a weight update mid-harness-search or resume harness exploration right after a gradient step, cutting the lag between detecting a plateau and reacting to it.

Learnings

1. Two orthogonal levers beat one, because they live in different change spaces. Scaffold edits top out at software-engineering hygiene; weight edits reach domain knowledge no prompt can encode. The np.clip+np.rint example is the cleanest proof.
2. Trajectory-level feedback, not metric-level, is what makes the Feedback-Agent work. Full execution logs let it diagnose failure modes and choose between PPO, GRPO, and entropic weighting. A system conditioned only on accuracy could not.
3. Algorithm selection can be delegated to an LLM agent, keyed to reward shape. The mapping from reward structure to RL method is a reusable heuristic independent of SIA's loop.
4. The verifier is the foundation and the weakness. Full autonomy keys off a deterministic verifier, but coupled Goodhart shows the cost: both optimisers can collude against the same target. Verifier robustness bounds the trustworthiness of any gain.
5. The recursion is shallow today, but the design points deeper. SIA improves the artifact (scaffold + weights). The proposed meta-RL extension would improve the improver. That distinction is the right axis for comparing RSI architectures.

Glossary