5. Sampling parameters: temperature, top-p, penalties

The model gives you a distribution; the sampler decides what you actually get.

| Parameter | Effect | Typical use | | --- | --- | --- | | `temperature` | Scales logits before softmax. 0 ≈ greedy, >1 flattens | 0-0.3 extraction, 0.7-1.0 drafting | | `top_p` | Nucleus sampling: keep smallest set of tokens with cumulative prob ≥ p | 0.9-1.0 | | `top_k` | Keep only the k most likely tokens | 40-100 | | `presence_penalty` | Discourages tokens already used at all | Reduce topic looping | | `frequency_penalty` | Discourages tokens by how often used | Reduce verbatim repetition | | `max_tokens` | Hard stop on output length | Cost control |

ts
const res = await client.chat.completions.create({
  model: "gpt-5-mini",
  temperature: 0,           // deterministic-ish for structured extraction
  top_p: 1,
  max_tokens: 512,
  messages: [{ role: "user", content: prompt }],
});

Gotcha: temperature 0 is *near*-deterministic, not guaranteed. Batching, GPU kernel nondeterminism, and mixture-of-experts routing can still shift outputs. Don't build exact-match tests on live model calls.

Read in another language

How to study this topic

Scoring

Check your understanding

Badges