10. Fine-tuning vs RAG vs prompting

Diagnose before you adapt:

| Failure | Right fix | | --- | --- | | Model lacks your private/current facts | Retrieval (RAG) | | Model knows facts but formats or behaves wrongly | Prompting / few-shot | | Model consistently misses your domain style, taxonomy, or terse format at scale | Fine-tuning (LoRA) | | Needs multi-step actions in your systems | Tools / agents |

LoRA (low-rank adaptation) trains small adapter matrices instead of all weights — minutes to hours, cheap to host, easy to roll back. QLoRA adds 4-bit base quantisation so it fits on a single GPU.

Gotcha: fine-tuning does not reliably inject new facts — it shapes behaviour. Teaching facts by fine-tune usually produces confident, stale answers; keep facts in retrieval where you can update and cite them.