Skip to content

Building an LLM in NodeJS

Select theme

Introduction
- Overview
Ch 2 · Tokenizer & Data
Ch 3 · Attention
Ch 4 · GPT Model
Ch 5 · Training

Select theme

On this page

Overview

On this page

Overview

LayerNorm & GELU feed-forward

Two building blocks used inside every transformer block: layer normalization (stabilizes activations) and the position-wise feed-forward network with a GELU activation.

Coming soon

This lesson is still being written. It will cover layer normalization and the position-wise feed-forward network with a GELU activation.

Previous
Overview Next
2 · Transformer block