Study the paper

Kimi Linear: An Expressive, Efficient Attention Architecture

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Comprehensive Assessment on Kimi Linear

Research and implementation context

Supplementary context

Research and implementation context

Research lineage

Notable related work, not a complete survey

Predecessors

  • Gated Delta Networks: Improving Mamba2 with Delta Rule

    Songlin Yang, Jan Kautz, Ali Hatamizadeh · 2024

    This paper introduces Gated Delta Networks (Gated DeltaNet), which improves Mamba2 with the Delta Rule, establishing the foundation for subsequent linear attention and delta rule optimizations.

Successors