← All courses
Advanced · 7 lessons
LLM fine-tuning: LoRA, QLoRA, and when not to do it
When fine-tuning pays off and when RAG or a prompt is enough: a decision tree, dataset curation as 80% of the work, LoRA and QLoRA on a single GPU, DPO for preference alignment, honest evaluation against the base model, and serving with quantization — up to the capstone: a small model turned code-review specialist.
- 01Fine-tune, RAG, or prompting: the decision treeThree ways to adapt an LLM, what each one actually changes, what fine-tuning fundamentally does not solve — and a step-by-step decision tree with the economics of the question.45 min
- 02Dataset curation: 80% of the workWhere examples come from, why a thousand curated ones beat a hundred thousand random ones, chat templates and loss on responses only — and a filtering pipeline with a held-out split before training starts.50 min
- 03LoRA and QLoRA: tuning on a single GPUWhy a full 7B fine-tune needs over 100 GB, how LoRA reduces the job to a pair of small matrices, what 4-bit NF4 and double quantization buy — and a working peft + trl script for a single card.60 min
- 04Instruction tuning and DPO: from imitation to preferencesHow an instruct model differs from a base one, why SFT teaches imitation but not picking the better answer, the DPO formula in plain terms — and DPOTrainer with its usual pitfalls.55 min
- 05Evaluation: tuned model versus baseAn honest baseline with the best prompt, side-by-side with an LLM judge and position-bias control, win rate with a confidence interval — and a check for catastrophic forgetting.50 min
- 06Serving: adapter merging, quantization, costMerge the adapter into the weights or serve a fleet of LoRAs on one base, GGUF versus AWQ, re-running the eval after quantization — and an honest cost calculation of your own GPU versus an API.50 min
- 07Capstone: a code-review specialist from a small modelThe full cycle: a dataset of diffs with seeded bugs, QLoRA SFT and DPO pairs of found versus missed, acceptance against pre-committed criteria — and a quantized endpoint with a model card.60 min