Chinchilla refit
A reproducible refit of the Chinchilla scaling law that measures how much the fitted exponents depend on the procedure used to fit them.
Results
- 01Reproduces Epoch AI's published fit on the same 240 runs: E = 1.817, alpha = 0.3473, beta = 0.3672, objective within 2e-11.
- 02Only 34.0% of the 4,500 L-BFGS-B starts reach the best optimum, and 7.6% at SciPy default tolerances.
- 031,000 bootstrap refits give 95% intervals that exclude the published E, B, beta and compute-optimal exponent.
Engineering details
Scaling-law exponents guide how compute is split between parameters and data, yet the estimation choices behind them (starting points, tolerances and the loss function) usually go unreported.
Reproduce the reference fit
Fit the parametric loss in log-sum-exp form with a Huber objective on log residuals, using L-BFGS-B from every point of the standard 4,500-point grid on Epoch AI's extracted runs.
Measure the optimiser
Record where every start ends. Most stop in degenerate fits where one or both power-law terms vanish, at 11x to 21x the loss.
Quantify the uncertainty
Run 1,000 bootstrap refits, swap Huber for least squares, and test extrapolation by fitting below 1e21 FLOP and predicting the 23 largest runs.