QLoRA

An extension of LoRA that stores the frozen base model in 4-bit precision using NF4 (a normal-distribution-tuned quantizationⓘquantizationweightsStoring or computing model weights in lower-precision number formats (FP8, INT8, INT4) to reduce memory and bandwidth, accepting small quality loss. Chat about this → Open full entry → scheme) and double-quantizes the quantizationⓘquantizationweightsStoring or computing model weights in lower-precision number formats (FP8, INT8, INT4) to reduce memory and bandwidth, accepting small quality loss. Chat about this → Open full entry → constants themselves to save more memory. LoRAⓘLoRAtrainingA parameter-efficient fine-tuning method that injects small low-rank adapter matrices into a frozen base model, training a tiny fraction of weights instead of the full model. Chat about this → Open full entry → adapters in full precision sit alongside the quantized base and absorb the gradient.

The combined memory footprint is small enough that 65B parameter base models fit on a single 48GB GPUⓘGPUsiliconA massively parallel processor originally designed for graphics, repurposed since the 2010s as the dominant compute substrate for both training and inference of large neural networks. Chat about this → Open full entry → and 7B models fit on a 24GB consumer card. The original paper showed QLoRA-tuned models matching full fine-tuningⓘfine-tuningtrainingContinued training of a pretrained base model on a smaller, task-specific dataset to specialize its behavior without retraining from scratch. Chat about this → Open full entry → quality on the Vicuna benchmarkⓘbenchmarkevaluationA standardized dataset and scoring rubric used to compare model capability on a defined task, the unit of model evaluation since GLUE made the format the default. Chat about this → Open full entry →.

In practice QLoRA is the default for hobbyist fine-tuningⓘfine-tuningtrainingContinued training of a pretrained base model on a smaller, task-specific dataset to specialize its behavior without retraining from scratch. Chat about this → Open full entry →. UnslothⓘUnslothtrainingAn open fine-tuning library that uses hand-written Triton kernels and a manual gradient implementation to run LoRA and QLoRA fine-tuning roughly 2x faster than the Hugging Face baseline. Chat about this → Open full entry →, AxolotlⓘAxolotltrainingAn open YAML-driven fine-tuning framework that orchestrates Hugging Face Transformers, PEFT, TRL, and DeepSpeed for one-shot LoRA, QLoRA, and full fine-tuning workflows. Chat about this → Open full entry →, and TRLⓘTRLtrainingHugging Face's library for preference and reinforcement learning on transformer models, the canonical open implementation of RLHF, DPO, KTO, ORPO, and related preference-tuning methods. Chat about this → Open full entry → all expose it as a configuration flag. The trade-off versus LoRAⓘLoRAtrainingA parameter-efficient fine-tuning method that injects small low-rank adapter matrices into a frozen base model, training a tiny fraction of weights instead of the full model. Chat about this → Open full entry → is slower training (the quantize-dequantize step adds overhead) in exchange for fitting bigger bases on smaller hardware.