Glossary
TRL
Hugging Face's library for preference and reinforcement learning on transformer models, the canonical open implementation of RLHF, DPO, KTO, ORPO, and related preference-tuning methods.
Hugging FacetrainingThe model hub, dataset hub, and open-source library suite (Transformers, Datasets, Tokenizers, Accelerate, PEFT, TRL) that anchors the open-AI ecosystem's distribution and tooling layer. Open full entry ’s transformerruntimeThe neural network architecture that combines self-attention with feed-forward layers, dominant for language modeling since 2017 and the substrate for nearly every modern LLM. Open full entry RL library. TRL implements the post- training methods that take a base or SFT model and add preference behavior: PPO-style RLHFtrainingA post-training pipeline that uses human preference rankings to train a reward model, then optimizes a base model against that reward via reinforcement learning. Open full entry , DPOtrainingA preference-tuning method that optimizes a model on pairwise human rankings directly, bypassing the reward-model and reinforcement-learning steps of RLHF. Open full entry , KTO, ORPO, and several research variants. Each method is a trainer class with consistent API surface and broad model coverage via the Transformers library.
The library lets a fine-tuner pick DPOtrainingA preference-tuning method that optimizes a model on pairwise human rankings directly, bypassing the reward-model and reinforcement-learning steps of RLHF. Open full entry , KTO, or ORPO with a one-line config change and run a preference-tuning experiment. Combined with AxolotltrainingAn open YAML-driven fine-tuning framework that orchestrates Hugging Face Transformers, PEFT, TRL, and DeepSpeed for one-shot LoRA, QLoRA, and full fine-tuning workflows. Open full entry (which wraps TRL behind a YAML interface) or UnslothtrainingAn open fine-tuning library that uses hand-written Triton kernels and a manual gradient implementation to run LoRA and QLoRA fine-tuning roughly 2x faster than the Hugging Face baseline. Open full entry (which adds speed) it covers the open-source post-trainingtrainingEverything that happens after pretraining ends: supervised fine-tuning, preference optimization, red-teaming, distillation, and safety work that turns a base into a shippable assistant. Open full entry pipeline.
Apache 2.0governanceA permissive open-source license used by most open-weight model releases (Llama from 4 onward partial, Qwen, Mistral, DeepSeek, Falcon), allowing commercial use without acceptable-use restrictions. Open full entry licensed.