Abstract
Test-time scaling (TTS) improves language model outputs by spending additional inference compute: generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing, grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation, the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only ρ̂v ≈ 0.12 with true quality, rendering selection near-random regardless of budget. Neither the generator itself nor a frontier model, used as verifier, closes the gap on medical, legal and finance benchmarks. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet recovers only ~40% of available quality. The candidate pool is not the bottleneck; choosing from it is.
Key results
Use the arrows to move through the five results.
1. Exploration works; exploitation does not. Oracle quality (full bar) is high for every method, but realised quality (hatched) falls far short. Matched compute (XHigh), Qwen3.5-35B-A3B.
2. The verifier explains the gap. Left: headroom capture tracks verifier correlation along the identity line (N = 152 points, R² = 0.66). Right: headroom capture per method at XHigh compute.
3. Tree search collapses diversity. Left: mean pairwise cosine distance across final outputs. Right: particle trajectory similarity over generation depth.
4. Synthesis beats selection. Fusion output quality versus a random held-out candidate, per benchmark. Only Fusion improves over the single-sample baseline on every benchmark for Qwen3.5.
5. Refinement helps on one benchmark of five. Oracle-position frequency (bars) and mean score (line) across refinement iterations.
What we ran
Six TTS methods, five open-ended benchmarks, three generator families, two reward models, one process reward model, and a unified judge. About 23,800 B200 GPU-hours in total.
Benchmarks
| Benchmark | Domain | Prompts | Evaluation | Rubric source | Scoring |
|---|---|---|---|---|---|
| LEXam | Law | 516 | Reference-guided | Professor solutions | Holistic, 0–1 |
| HealthBench | Medicine | 5,000 | Rubric-only | Physician rubrics | Binary per criterion |
| PRBench | Finance, law | 1,650 | Rubric-only | Expert rubrics | Binary per criterion |
| WildBench | Chat, general | 1,024 | Rubric-only | LLM-generated checklist | Holistic, 1–10 |
| WritingBench | Creative writing | 555 | Rubric-only | LLM-generated rubric | 1–10 per criterion |
None of the five admits exact-match or binary verification.
Test-time scaling methods and compute budgets
| Method | How it uses compute | Verifier | Low | Mid | High | XHigh |
|---|---|---|---|---|---|---|
| Best-of-N (BoN) | Parallel sampling, select best | ORM | N=2 | N=4 | N=8 | N=16 |
| Beam Search (BS) | Deterministic tree search | PRM | – | W=2, k=2 | W=2, k=4 | W=4, k=4 |
| Particle Filter (PF) | Stochastic resampling by PRM weight | PRM | – | N=4 | N=8 | N=16 |
| Sequential Refinement (SR) | Draft, critique, refine | Self (generator) | 1 iter | 2–3 iter | 4–6 iter | 7–12 iter |
| Budget Forcing (BF) | Extended thinking, one trajectory | None | 2× think | 4× think | 8× think | 16× think |
| Fusion | Synthesise a final answer from N candidates | None (generator synthesises) | – | 3 cand. | 6–7 cand. | 15 cand. |
Compute is normalised by total generator output tokens, excluding the (negligible) cost of reward models. Each level matches the average BoN output at N = 2, 4, 8, 16 within 25%. A dash means the method was not run at that level.
Models
| Role | Model | Notes |
|---|---|---|
| Generators | Qwen3.5-9B, Qwen3.5-35B-A3B | Main results use Qwen3.5-35B-A3B. Thinking mode on. |
| OLMo3-7B-Think, OLMo3.1-32B-Think | Second family. Fusion results are mixed here. | |
| Gemma-4-26B-A4B | Third-family check at High compute, all five benchmarks. | |
| Outcome reward models | Skywork-Reward-V2-Llama-3.1-8B | Used for Best-of-N. Chosen for RewardBench 2 performance. |
| Llama-3.1-70B-Instruct-RM-RB2 | Second ORM. Near-identical realised scores to Skywork. | |
| Process reward model | VersaPRM-8B | Guides Beam Search and Particle Filter. Chosen for multi-domain training. |
| Judges | Qwen3.5-397B-A17B | Unified judge for all benchmarks. Checked against each native judge. |
| GLM5.2 | Non-Qwen judge that re-scores Qwen3.5-35B-A3B outputs, to rule out same-family bias. | |
| Sanity checks | MATH-500, GPQA Diamond | Same reward models add only ~2 pp here. Appendix of the paper. |
Talk slides
BibTeX
@inproceedings{romano2026testtime,
title = {Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck},
author = {Romano, Davide and Raj, Kanak and Parker, Jerrod and Giofr{\`e}, Daniele},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2026},
note = {arXiv:2608.18931},
url = {https://arxiv.org/abs/2608.18931}
}