Transformers

Attention mechanisms, training, alignment, and mathematical foundations.

Introduction

Overview

The Transformer (Vaswani et al., 2017) replaced recurrence with self-attention, enabling parallel training and becoming the foundation of virtually all modern LLMs.

Technical Deep Dive

Key innovation: all pairwise token interactions in O(n²) attention vs. O(n) sequential RNN steps—tradeoff favors GPUs on parallel matmul. Variants: encoder-only (BERT), decoder-only (GPT), encoder-decoder (T5).

Scaled to 100B+ parameters via data, compute, and architectural tweaks (RoPE, SwiGLU, RMSNorm).

Practical Use Case

Every enterprise copilot today runs on Transformer decoders—understanding attention is prerequisite for debugging context failures.

01Self-Attention

Overview

Self-attention computes a weighted sum of value vectors where weights come from query-key compatibility—each token builds a context-aware representation.

Technical Deep Dive

Per head: Q=XWq, K=XWk, V=XWv; Attention=softmax(QKᵀ/√d_k)V. Multi-head runs h parallel attentions; concat + project. √d_k prevents softmax saturation.

Causal masking in decoders sets future positions to -∞ before softmax.

Practical Use Case

Long doc QA fails when key evidence sits in lost-in-the-middle positions—mitigate with reranking + place critical chunks at context edges.

QKV → Softmax(QKᵀ/√d)V

RLHF & Alignment

Overview

RLHF aligns models with human preferences via reward model training on comparison data, then policy optimization (PPO) to maximize reward while staying near reference model (KL penalty).

Technical Deep Dive

Pipeline: SFT on demonstrations → train reward model R(x,y) from human rankings → RL fine-tune π to maximize R − β·KL(π||π_ref). Alternatives: DPO, KTO (no explicit RM).

Alignment targets helpfulness, harmlessness, honesty—not raw next-token prediction.

Practical Use Case

Enterprise fine-tune: SFT on approved support transcripts + DPO on thumbs-up/down logs—reduces off-brand tone without full RL infrastructure.

Scaled Dot-Product Attention
Attention(Q,K,V) = softmax(QKᵀ / √d_k) V
Compute affinity scores between queries and keys, normalize via softmax, aggregate values.

02Attention Mechanism

Overview

Queries, keys, and values are learned linear projections of hidden states; multi-head attention runs parallel attention subspaces.

Technical Deep Dive

Each head learns different relational patterns (syntax vs. coreference). Output concatenation + W_O mixes heads. In decoder-only models, causal masking ensures position i attends only to j≤i.

Practical Use Case

Debugging generation errors often traces to attention failing on long-range dependencies—switch to models with longer context or insert retrieval for distant facts.

Input Embeddings Multi-Head Attention Feed Forward Output Projection Encoder stack (×N layers)

Embeddings & Tokenization

Overview

Tokenizers (BPE, SentencePiece) segment text into subword units; embedding layer E maps token IDs to ℝ^d vectors combined with positional information.

Technical Deep Dive

Vocabulary size 32k–128k. Special tokens: pad, bos, eos. Byte-level BPE handles rare words and typos. Embedding tables are largest memory component in small models; tied weights with output lm_head reduce parameters.

Practical Use Case

Multilingual helpdesk uses SentencePiece trained on ticket corpus—OOV rate drops 40% vs. word-level tokenization on product codenames.

Positional Encoding

Overview

Transformers have no inherent order; positional encodings (sinusoidal, learned, RoPE, ALiBi) inject sequence position information.

Technical Deep Dive

RoPE (Rotary Position Embedding) rotates Q/K vectors by position-dependent angles—enables relative position generalization and extrapolation beyond training length in some settings.

Practical Use Case

Long-context contracts use RoPE-based models at 128k; legal teams validate performance on clause cross-references at document end.

Softmax
σ(z_i) = exp(z_i) / Σ_j exp(z_j)
Normalizes attention logits into a probability distribution over positions.
Cross-Entropy Loss
L = -Σ y_i log(ŷ_i)
Next-token prediction minimizes cross-entropy between predicted and true token distributions.
Input EmbedAttend Generate