Post-training · MathCAMPS · nanochat-d34

GRPO Budget-Scaling Diagnostic

Pass@k alone can't tell a real gain from a model that quietly stopped trying. This adds two health probes — answer diversity and parse rate — across every General / LittleLearner / Hybrid checkpoint evaluated in the 300–1500 step budget sweep, so collapsed runs are flagged instead of just under-performing silently.

Eval: MathCAMPS no-ICL, N=128, T=0.8, top_k=50 Filter: paper eval-filter applied to pass@k only Window: 2026-05-05 → 2026-07-28

How “fried” is decided

Healthy modal-answer share < 20% on both bands — the model's answers track problem diversity, as expected. Degraded partial collapse — modal share > 30% on one band, parse rate visibly down, but still emitting varied, mostly-coherent text. modal share > 50% on ≥1 band — the model gives (near-)the same answer regardless of the question, or degenerates into repeated-token garbage.
finish_eos_frac is not the signal here. It reads 0.0 for every General/LittleLearner/Hybrid row — healthy or fried alike — because that eval script's hardcoded stop strings don't match this family's chat template; every completion runs to the 512-token cap regardless. The real tell is modal-answer share: the fraction of problems where the model's first-sample answer equals its single most common answer, computed on the raw grade-restricted eval set (N=3700 ≤K-5 / N=1200 >K-5, pre eval-filter — pass@k columns use the filtered numbers).

General · unfiltered pretrain

≤K-5 climbs cleanly through ~900–1000 steps at lr=2e-5, then flattens. 5× the LR (aggressive) breaks it at the same step count.
≤K-5 >K-5
StageVerdict p@1p@10p@128modal share p@1p@10p@128modal share
baseno post-train Healthy 4.8%17.6%37.2% 3.0%1550 distinct · parse 98% 0.2%1.0%4.5% 3.1%767 distinct · parse 97%
GRPO 300 stepslr=1e-5 (v4 baseline) Healthy 2.9%18.0%45.4% 4.6%1126 distinct · parse 100% 0.3%1.6%5.8% 3.9%597 distinct · parse 100%
GRPO 600 stepslr=2e-5 (moderate) Healthy 7.1%26.6%46.7% 5.3%676 distinct · parse 100% 0.5%2.2%6.8% 4.5%427 distinct · parse 99%
GRPO 600 stepslr=5e-5 (aggressive)5× the moderate LR at the same step count — partial collapse; >K-5 eval was never run for this one Degraded 0.5%2.7%10.1% 35.4%97 distinct · parse 66% not evaluated
GRPO 900 stepslr=2e-5 Healthy 8.4%28.1%46.1% 5.1%538 distinct · parse 99% 0.5%2.5%6.2% 5.5%330 distinct · parse 94%
GRPO 1200 stepslr=2e-5 Healthy 9.1%27.0%42.8% 12.1%406 distinct · parse 98% 0.4%2.2%5.5% 14.1%276 distinct · parse 92%
GRPO 1500 stepslr=2e-5 Healthy 9.1%27.0%44.2% 11.7%398 distinct · parse 98% 0.7%2.8%6.6% 10.3%273 distinct · parse 92%

LittleLearner · K-5-filtered pretrain

The same lr=2e-5/600-step recipe that helped General collapses this base by step 600, and continuing training doesn't recover it — it degrades further. Dropping back to the safe lr=1e-5 avoids collapse, but 600 steps still quietly erases the 300-step gain rather than building on it.
≤K-5 >K-5
StageVerdict p@1p@10p@128modal share p@1p@10p@128modal share
baseno post-train Healthy 1.1%7.6%24.2% 13.3%511 distinct · parse 94% 0.2%1.3%3.7% 11.6%268 distinct · parse 93%
GRPO 300 stepsbaseline recipe Healthy 1.5%9.5%27.7% 9.8%638 distinct · parse 90% 0.1%0.8%3.7% 19.8%392 distinct · parse 82%
GRPO 600 stepslr=1e-5 (fresh from sft_v3_filtered_base, isolation test)Doesn't mode-collapse — modal share stays near the healthy range, parse rate is actually higher than the 300-step checkpoint (99%). But every single pass@k number is now below the 300-step baseline: le_k5 p@128 is statistically indistinguishable from the untrained base (24.06% vs 24.18%), and gt_k5 p@128 drops below base (3.68% → 2.45%). 600 steps at the "safe" LR quietly erased the 300-step gain instead of building on it — a third failure mode, distinct from both General's clean improvement and the mode-collapse below. Degraded 1.4%8.8%24.1% 18.6%610 distinct · parse 99% 0.1%0.7%2.5% 20.8%353 distinct · parse 99%
GRPO 600 stepslr=2e-5 (moderate)Mode-collapsed: answers ‘1’ on ~94% of ≤K-5 problems, regardless of the question. 1.9%1.9%2.0% 94.4%5 distinct · modal '1' 0.2%0.2%0.2% 85.3%3 distinct · modal '1'
GRPO 900 stepslr=2e-5, continued from the 600-step checkpointDegenerated further into literal token-repeat garbage — sample completion: “Step matched matched matched matched…” repeated for all 512 tokens. Parse rate craters. 0.7%1.7%2.3% 56.2%4 distinct · parse 36% 0.03%0.1%0.2% 74.3%3 distinct · parse 18%

Hybrid · K-5-filtered pretrain + unfiltered SFT

Same story as LittleLearner at 600 steps — a different, but equally total, collapse. Confirmed it isn't an LR artifact: a fresh run at the original lr=1e-5 collapses too, with a third distinct failure signature. A reviewer-suggested curriculum (introduce grades 6→7→8 one at a time instead of all at once) fixes the collapse entirely and beats the healthy baseline on ≤K-5 — but >K-5 still doesn't move.
≤K-5 >K-5
StageVerdict p@1p@10p@128modal share p@1p@10p@128modal share
base= LittleLearner base (shared checkpoint) Healthy 1.1%7.6%24.2% 13.3%511 distinct · parse 94% 0.2%1.3%3.7% 11.6%268 distinct · parse 93%
GRPO 300 stepsbaseline recipe Healthy 1.4%10.2%31.1% 6.6%780 distinct · parse 93% 0.2%1.0%3.9% 6.4%483 distinct · parse 92%
GRPO 600 stepslr=2e-5 (moderate)Mode-collapsed onto a repeated-digit string (‘555555…’, 200+ digits) — not word-repetition like LittleLearner's, but the same reward-hacked-format-without-solving-anything failure. 0.01%0.04%0.18% 37.8%125 distinct · parse 100% 0.00%0.00%0.00% 46.1%74 distinct · parse 100%
GRPO 600 stepslr=1e-5 (low-LR isolation test)Fresh from sft_v3_mixed (not warm-started from the lr=2e-5 collapse above) — isolates whether the elevated LR was the cause. It wasn't. Caught via a fast 12-problem/1-sample probe instead of the full N=128 sweep (the sweep's own throughput signature already matched the known-fried pattern): 12/12 wrong, looping a generic template — “Step 1: Find the sum… Step 9: Find the final answer…” — without computing anything. A third distinct failure signature (templated step-counter babble, vs. word-repeat and digit-repeat above). full N=128 eval not run — stopped after the probe confirmed collapse, to avoid burning ~5h+ of GPU time on a checkpoint already known to be fried not evaluated
GRPO 600 stepsCURRICULUM: K-6 → K-7 → K-8, 200 steps each, lr=1e-5Reviewer hypothesis test: stage the grade introduction instead of dumping the full K-8 pool at once. Fresh from sft_v3_mixed, K-5 synth present throughout, grades 6/7/8 added one at a time. Result: collapse is fully avoided (best modal-share/diversity numbers in the whole Hybrid track) and ≤K-5 beats even the healthy 300-step baseline (32.3% vs 31.1% p@128). But >K-5 p@128 lands at 3.90% — exactly tied with the 300-step baseline's 3.90%. Staging fixes the instability and gives a real ≤K-5 bonus; it does not move the >K-5 ceiling at all. Healthy 1.7%11.7%32.3% 5.1%727 distinct · parse 99% 0.1%0.9%3.9% 5.2%429 distinct · parse 99%