Chapter 3 · Attention

Chapter 3 builds attention step by step, from a simplified version to full multi-head causal self-attention.

Lessons

1. Self-attention
2. Simplified self-attention
3. Self-attention with trainable weights
4. Multi-head attention

Code

• @node-llm/core → src/model/attention.ts defines MultiHeadAttention
• Runnable: pnpm ch03