Study the paper

Scaling Laws for Neural Language Models

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Overfitting and the Infinite Data Limit

Optimal Allocation Scaling: step by step

Interactive study view

Optimal Allocation Scaling: step by step

Illustrative teaching data

Follow the existing bounded walkthrough in its intended sequence.

Step 1 of 3

  1. Step 1

    Calculate the exponent for model size N: alpha_C_min / alpha_N = 0.042 / 0.076 = 0.5526

  2. Step 2

    Calculate the exponent for batch size B: alpha_C_min / alpha_B = 0.042 / 0.21 = 0.2000

  3. Step 3

    Calculate the exponent for training steps S: alpha_C_min / alpha_S = 0.042 / 0.17 = 0.2471