Fine-Tuning
Subject: Computer and Technology noun /ˌfaɪnˈtuː.nɪŋ/ 1 definition
- machine-learning
- deep-learning
- neural-networks
- transfer-learning
-
Definition (noun)
Entry #302 · UpdatedFine-Tuning is a transfer-learning process in machine learning that adapts a pretrained model—originally trained on a broad, large-scale dataset—to a narrower task, domain, or behavioral objective through continued training on targeted examples. Initializing from pretrained weights preserves general-purpose representations such as visual edges or linguistic syntax, while gradient-based optimization with a small learning rate (often paired with warmup schedules, weight decay, or selective layer freezing) adjusts parameters toward the specialized objective. By reusing features already learned during pretraining, it achieves strong task performance using orders of magnitude less labeled data and compute than training a comparable neural network from random initialization. However, because fine-tuning directly modifies model weights, it remains bounded by training data quality and stability trade-offs: aggressive updates on narrow datasets can trigger overfitting or catastrophic forgetting of base capabilities, whereas rapidly changing external facts are often handled more reliably through prompting or retrieval-augmented generation.
Common Forms
Full Fine-Tuning: Unfreezes and updates every weight across the pretrained network via backpropagation, maximizing adaptability for major domain shifts at the cost of high GPU memory overhead, full-checkpoint storage per task, and greater risk of catastrophic forgetting.
Layer-Frozen (Partial) Fine-Tuning: Locks early backbone layers that capture generic representations and trains only upper layers or a newly attached task-specific output head, reducing overfitting and compute cost on small labeled datasets.
Parameter-Efficient Fine-Tuning (PEFT): Keeps the original backbone weights frozen and optimizes only a compact set of injected parameters—such as Low-Rank Adaptation (
LoRA) matrices, quantizedQLoRAadapters, or soft prompt embeddings—cutting optimizer memory usage and enabling modular per-task adapter swapping.Supervised and Preference Fine-Tuning: Aligns foundation models with explicit instructions and behavioral constraints by combining Supervised Fine-Tuning (
SFT) on curated(prompt, response)demonstrations with preference optimization techniques such as Direct Preference Optimization (DPO) or Reinforcement Learning from Human Feedback (RLHF).
Example
To specialize an 8-billion-parameter pretrained language model for structured SQL generation, an ML engineer loads 4-bit quantized base weights (
QLoRA), attaches rank-16LoRAadapters (r=16,lora_alpha=32) to the attention projection layers, and runs 3 epochs of supervised fine-tuning on 15,000(question, sql_query)pairs using AdamW at learning rate2e-4. Because fewer than 1% of parameters are updated while the base weights remain frozen, training fits within a single 24 GB GPU and writes a portable adapter checkpoint under 100 MB instead of duplicating the full 16 GB model.