You are reading immutable version 3. The current guide may be newer.

Study the paper

Scaling Laws for Neural Language Models

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Transformer Parameter and Compute Scaling

Unified Loss Equation L(N,D): step by step

Interactive study view

Unified Loss Equation L(N,D): step by step

Illustrative teaching data

Follow the existing bounded walkthrough in its intended sequence.

Step 1 of 5

  1. Step 1

    Calculate the ratio of critical model size to model size: Nc / N = 10^9 / 10^10 = 0.1

  2. Step 2

    Raise this ratio to the power of alpha_N / alpha_D = 0.08 / 0.08 = 1.0, yielding 0.1

  3. Step 3

    Calculate the ratio of critical dataset size to dataset size: Dc / D = 10^9 / 10^9 = 1.0

  4. Step 4

    Sum the two terms: 0.1 + 1.0 = 1.1

  5. Step 5

    Raise the sum to the power of alpha_D = 0.08 to obtain the final loss: 1.1^0.08 ≈ 1.00765