Test-Time Scaling in the Wild:
Why Exploitation, Not Exploration, Is the Bottleneck

Davide Romano*, Kanak Raj*, Jerrod Parker, Daniele Giofrè
Thomson Reuters
NeurIPS 2026

*Equal contribution
Schematic of the six test-time scaling methods compared in the paper: Best-of-N, Beam Search, Particle Filtering, Budget Forcing, Fusion and Sequential Refinement.

We compare five test-time scaling families, at matched compute, on five open-ended benchmarks. The candidate pool is not the bottleneck; choosing from it is.

ρ̂v ≈ 0.12
correlation between reward-model scores and true quality, so selection is near-random
~40%
of available headroom recovered by Fusion, the best method (Best-of-N: ~15%)
5 × 5
TTS families × open-ended benchmarks: medicine, law, finance, chat, creative writing

Abstract

Test-time scaling (TTS) improves language model outputs by spending additional inference compute: generating multiple candidates, searching over partial sequences, or iteratively refining drafts. These techniques yield large gains on mathematics and code, but have been developed and stress-tested almost exclusively on tasks where verification is straightforward. We conduct the first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing, grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation. The answer depends on which side of that decomposition you examine. Scaling exploration works: the best candidate in the pool improves steadily with compute across all settings. What breaks is exploitation, the step that converts a rich candidate pool into a final output. With state-of-the-art generators, reward models correlate at only ρ̂v ≈ 0.12 with true quality, rendering selection near-random regardless of budget. Neither the generator itself nor a frontier model, used as verifier, closes the gap on medical, legal and finance benchmarks. Tree search amplifies this failure through diversity collapse. Refinement helps on one of five benchmarks; its apparent gains elsewhere are confounded. Only synthesis across candidates (Fusion) consistently improves over single-sample baselines, yet recovers only ~40% of available quality. The candidate pool is not the bottleneck; choosing from it is.

Key results

What we ran

Six TTS methods, five open-ended benchmarks, three generator families, two reward models, one process reward model, and a unified judge. About 23,800 B200 GPU-hours in total.

Benchmarks

BenchmarkDomainPromptsEvaluationRubric sourceScoring
LEXamLaw516Reference-guidedProfessor solutionsHolistic, 0–1
HealthBenchMedicine5,000Rubric-onlyPhysician rubricsBinary per criterion
PRBenchFinance, law1,650Rubric-onlyExpert rubricsBinary per criterion
WildBenchChat, general1,024Rubric-onlyLLM-generated checklistHolistic, 1–10
WritingBenchCreative writing555Rubric-onlyLLM-generated rubric1–10 per criterion

None of the five admits exact-match or binary verification.

Test-time scaling methods and compute budgets

MethodHow it uses computeVerifierLowMidHighXHigh
Best-of-N (BoN)Parallel sampling, select bestORMN=2N=4N=8N=16
Beam Search (BS)Deterministic tree searchPRM–W=2, k=2W=2, k=4W=4, k=4
Particle Filter (PF)Stochastic resampling by PRM weightPRM–N=4N=8N=16
Sequential Refinement (SR)Draft, critique, refineSelf (generator)1 iter2–3 iter4–6 iter7–12 iter
Budget Forcing (BF)Extended thinking, one trajectoryNone2× think4× think8× think16× think
FusionSynthesise a final answer from N candidatesNone (generator synthesises)–3 cand.6–7 cand.15 cand.

Compute is normalised by total generator output tokens, excluding the (negligible) cost of reward models. Each level matches the average BoN output at N = 2, 4, 8, 16 within 25%. A dash means the method was not run at that level.

Models

RoleModelNotes
GeneratorsQwen3.5-9B, Qwen3.5-35B-A3BMain results use Qwen3.5-35B-A3B. Thinking mode on.
OLMo3-7B-Think, OLMo3.1-32B-ThinkSecond family. Fusion results are mixed here.
Gemma-4-26B-A4BThird-family check at High compute, all five benchmarks.
Outcome reward modelsSkywork-Reward-V2-Llama-3.1-8BUsed for Best-of-N. Chosen for RewardBench 2 performance.
Llama-3.1-70B-Instruct-RM-RB2Second ORM. Near-identical realised scores to Skywork.
Process reward modelVersaPRM-8BGuides Beam Search and Particle Filter. Chosen for multi-domain training.
JudgesQwen3.5-397B-A17BUnified judge for all benchmarks. Checked against each native judge.
GLM5.2Non-Qwen judge that re-scores Qwen3.5-35B-A3B outputs, to rule out same-family bias.
Sanity checksMATH-500, GPQA DiamondSame reward models add only ~2 pp here. Appendix of the paper.

BibTeX

@inproceedings{romano2026testtime,
  title     = {Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck},
  author    = {Romano, Davide and Raj, Kanak and Parker, Jerrod and Giofr{\`e}, Daniele},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  note      = {arXiv:2608.18931},
  url       = {https://arxiv.org/abs/2608.18931}
}