Skip to content

Chapter 4 · GPT Model

Chapter 4 starts from a single layer of neurons, then assembles the pieces (normalization, feed-forward, and attention) into a transformer block, and finally stacks blocks into the full GPT.

  1. A mini neural network layer
  2. LayerNorm & GELU feed-forward
  3. The transformer block
  4. The full GPT model
  • @node-llm/core → src/model/block.ts, src/model/gpt.ts (GPT_CONFIG_124M)
  • Runnable: pnpm ch04