A new arXiv preprint says training runs for terminal agents should stay matched to the harness that produced them, not get pooled into one shared dataset.
Its method, CoTrace, keeps harness search and model training as separate steps. On the Tmax promotion split, Qwen3.5-9B went from 78 to 88 solved tasks with supervised fine-tuning, and an online reinforcement variant reached 90. On Terminal-Bench 2.1 and SWE-bench Lite, the authors report that transfer to out-of-distribution tasks depends on harness compatibility.
Training runs should match their harness. A preprint argues a run's value for training depends on the harness it was generated under.
Solved tasks on the Tmax promotion split: Qwen3.5-9B before training 78 tasks, After supervised fine-tuning 88 tasks. The authors' own results on the Tmax promotion split, from a single preprint.
Capability needs weights and harness. The harness is the runtime layer that formats prompts, binds tools and handles error recovery. Terminal-agent capability depends on it and on the model weights.
Keep search and training separate. CoTrace runs harness search and model training as separate steps, each with its own promotion decisions. Recurring execution failures guide how the harness is built.
Train only on matching, verified runs. Supervised fine-tuning uses only verified runs that match the harness in use. Reinforcement learning uses fresh online interactions.
Qwen3.5-9B: 78 to 88 solved tasks. On the Tmax promotion split, supervised fine-tuning took Qwen3.5-9B from 78 to 88 solved tasks. An online reinforcement variant reached 90.
Smaller corpus, lower compute. The authors report a compact harness-matched corpus gave steady gains at lower compute than much larger corpora pooled across sibling harnesses.
Cover figure: Chen, Jixuan et al., arXiv:2610.10426, CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/), https://arxiv.org/abs/2610.10426
