You are reading immutable version 4. The current guide may be newer.

Study the paper

Scaling Laws for Neural Language Models

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Overfitting and the Infinite Data Limit

Joint Scaling Law L(N,D): step by step

Interactive study view

Joint Scaling Law L(N,D): step by step

Illustrative teaching data

Follow the existing bounded walkthrough in its intended sequence.

Step 1 of 5

  1. Step 1

    Calculate exponent ratio alpha_N / alpha_D = 0.8

  2. Step 2

    Calculate model term (N_c / N)^(alpha_N / alpha_D) = 100^0.8 ≈ 39.8107

  3. Step 3

    Calculate dataset term D_c / D = 10

  4. Step 4

    Sum the terms: 39.8107 + 10 = 49.8107

  5. Step 5

    Raise the sum to the power of alpha_D: 49.8107^0.10 ≈ 1.478