← Theory library

LLM Engineering theory

Model providers, tokens, context windows, structured output, and evaluation.

Practise in the Code Lab →

Easy

1. What a large language model actually isAn LLM is a next-token predictor trained on enormous text corpora.
2. Tokenization and why it bites youModels see subword tokens, not characters or words.
3. The transformer and self-attentionAttention lets every token weigh every other token.
4. Positional encoding and RoPEAttention is order-blind, so position must be injected.
5. Sampling parameters: temperature, top-p, penaltiesDecoding settings control creativity, determinism, and repetition.
6. Context windows and the lost-in-the-middle effectLong windows still have uneven attention and real cost.

Medium

Hard