What the terms mean

A purpose-built data center optimized for AI training rather than general cloud workloads, characterized by liquid-cooled high-density GPU racks, gigawatt-scale single-tenant power, and tightly-coupled networking.

A power arrangement where generation sits on the same side of the utility meter as the load, letting a data center draw directly from the plant and bypass the grid.

A protocol market matching GPU supply from many independent providers to AI demand, settled on a token rail; Akash, io.net, Bittensor compute, and Hyperbolic are canonical.

The second phase of LLM inference, generating one token at a time from the KV cache. Memory-bandwidth-bound; throughput tracks memory bandwidth more than peak compute.

A cooling architecture that pipes liquid coolant directly to a cold plate on each processor, evacuating the 700+ watts per GPU that air cooling cannot handle.

A parallelism strategy for mixture-of-experts models where different GPUs hold different experts; requires all-to-all communication on every token routing step.

An AI training facility whose power draw is measured in gigawatts rather than megawatts, the scale at which siting decisions become grid-and-permitting problems rather than real-estate ones.

The regulatory queue a new generation or load project must traverse to connect to the grid; currently the binding constraint on how fast gigawatt-class AI sites come online.

The capital expenditure that Microsoft, Google, Amazon, Meta, and Oracle are spending on AI infrastructure, totaling roughly $300B+ annually by 2025-2026 and dominating the supply-and-demand signal for the entire stack.

A specialized cloud provider focused exclusively on GPU and AI workloads, operating outside the traditional AWS/Azure/GCP hyperscaler perimeter, with CoreWeave, Lambda, and Voltage Park as the canonical examples.

A long-term contract under which an AI operator commits to buy a defined nuclear-generated power output, becoming the cornerstone financing mechanism for gigawatt-class AI buildouts.

The first phase of LLM inference, processing the input prompt and building the initial KV cache. Compute-bound and parallel across prompt tokens.

Power Usage Effectiveness, the ratio of total data center facility power to power delivered to IT equipment; lower is better, with 1.0 the floor and 1.1 a strong target.

A nuclear reactor under ~300 MW per unit, factory-fabricated rather than site-built, positioned as the firm-power option for AI data centers needing new generation faster than conventional plants deliver.

AI compute capacity owned, operated, or contractually controlled by a nation-state for the use of its own institutions and citizens, distinct from rented capacity on US-hyperscaler clouds.

A way to split a single model across multiple GPUs by sharding each layer's weight matrices and doing an all-reduce after every layer. Bandwidth-hungry but layer-by-layer fine-grained.

FLOPs performed per byte read from memory. Low intensity means an operation is memory-bound; high intensity means compute-bound. LLM decode has very low intensity.

A cryptographic protocol that lets a remote party verify which code is running inside a TEE, including which model is loaded and which build of the inference engine.

A 16-bit floating-point format with FP32's exponent range and only 7 mantissa bits. Designed for neural-network training; standard across 2026 accelerators alongside FP16.

An AI compute company built around wafer-scale chips (the WSE-3 is a single die covering most of a 300mm wafer), offering some of the lowest inference latency on the market.

The umbrella category of compute architectures where workloads run isolated from the host operator, combining hardware TEEs, attestation, and encrypted-memory protections.

NVIDIA's parallel-computing platform and proprietary toolchain, the de facto programming model for GPU-accelerated machine learning since the late 2000s.

The second phase of LLM inference, generating one token at a time from the KV cache. Memory-bandwidth-bound; throughput tracks memory bandwidth more than peak compute.

A cooling architecture that pipes liquid coolant directly to a cold plate on each processor, evacuating the 700+ watts per GPU that air cooling cannot handle.

An exact attention algorithm that reorders the computation to avoid materializing the full attention matrix in GPU HBM, giving 2 to 4 times speedup with no quality loss.

A 16-bit floating-point format used as the default precision for deep learning training and inference, halving memory versus FP32 with small quality cost on most workloads.

A 4-bit floating-point format with hardware-native multiplication on Blackwell-generation accelerators. NVFP4 and MXFP4 variants target large-model inference and post-training quantization.

An 8-bit floating-point format used for AI inference and increasingly for training, halving memory and bandwidth versus FP16 with minimal quality loss on most workloads.

The graphics memory generation on 2025-era consumer and workstation GPUs such as the RTX 5090 and RTX PRO 6000. High bandwidth per board, lower capacity than HBM.

An AI training facility whose power draw is measured in gigawatts rather than megawatts, the scale at which siting decisions become grid-and-permitting problems rather than real-estate ones.

A massively parallel processor originally designed for graphics, repurposed since the 2010s as the dominant compute substrate for both training and inference of large neural networks.

An AI inference company with custom deterministic LPU chips and a hosted inference service that achieves extremely low time-per-token (1000+ tokens/sec on 70B models).

Stacked DRAM used as the main memory of every modern AI accelerator, with bandwidth in TB/s rather than GB/s and capacity per stack in tens of GB.

Running a trained model to produce outputs (tokens, images, embeddings) from inputs at serving time, as distinct from the gradient updates of training.

A high-throughput, low-latency network fabric (Mellanox, now NVIDIA) used for inter-node communication in AI training clusters, supporting RDMA for direct GPU-to-GPU transfer across machines.

The stored key and value vectors from previously processed tokens, reused at each generation step so an autoregressive model does not recompute attention over the entire prefix.

Low-power DRAM used as unified memory in Apple Silicon, DGX Spark, and Strix Halo. High capacity and efficiency, with bandwidth below HBM and GDDR.

The rate (GB/s or TB/s) at which an accelerator reads its memory. It sets the ceiling on decode tokens/sec, since each token streams the active weights once.

Apple's open-source ML framework designed for Apple Silicon's unified memory architecture, the local-first inference engine for Mac and increasingly iPad and iPhone.

MBU is the fraction of an accelerator's peak memory bandwidth a serving stack actually reaches during decode. Real systems land around 60 to 85 percent.

MFU is the fraction of an accelerator's peak compute a workload actually achieves. The compute-bound analogue of MBU, relevant to prefill and training, not memory-bound decode.

NVIDIA's proprietary GPU-to-GPU interconnect, providing bandwidth an order of magnitude above PCIe and the basis for tightly-coupled 8-GPU server nodes (DGX, HGX).

Running model inference on the user's local hardware (phone, laptop, embedded device), enabled by smaller models, FP8 quantization, and runtimes like llama.cpp and MLX.

The first phase of LLM inference, processing the input prompt and building the initial KV cache. Compute-bound and parallel across prompt tokens.

Storing or computing model weights in lower-precision number formats (FP8, INT8, INT4) to reduce memory and bandwidth, accepting small quality loss.

A networking technique that lets a remote machine read or write local memory without involving the CPU, foundational for high-throughput distributed training over InfiniBand or RoCE.

An open instruction set architecture, royalty-free and modular, increasingly used in AI accelerator cores (Tenstorrent, SiFive Intelligence) as the open alternative to ARM and x86.

AMD's open-source GPU compute stack, the main credible alternative to CUDA, with growing coverage in PyTorch and vLLM but still trailing on kernel maturity and tooling.

A performance model that bounds throughput by either compute or memory bandwidth, whichever is the limiting resource for an operation's arithmetic intensity.

Intel's earliest mainstream trusted execution environment, the predecessor to TDX, with smaller enclave sizes and a history of side-channel vulnerabilities that limited its deployment for AI.

A hardware-isolated CPU region where code and data are protected from inspection by the host OS, used to run inference in a way the operator cannot read or modify.

NVIDIA's closed-source inference engine for NVIDIA GPUs, the fastest runtime on Hopper and Blackwell but tied to NVIDIA's proprietary kernel stack and CUDA.

An AI accelerator startup designing RISC-V-based chips (Wormhole, Blackhole, Grendel) with an open software stack, positioned as the leading open alternative to NVIDIA at the silicon layer.

The headline inference speed metric. Decode tokens/sec is what a user feels as text streams; it is bounded by memory bandwidth divided by the bytes streamed per token.

Google's custom AI accelerator family, used internally for training Gemini and externally via Google Cloud, designed around dense matrix multiplication with a systolic array architecture.

A single physical memory pool shared by CPU and GPU, so the full capacity is usable as model memory; used by Apple Silicon, Strix Halo, and DGX Spark.

The first-pass formula for whether a model fits on a GPU. VRAM ≈ parameters × (bits ÷ 8), plus 10-30 percent for KV cache, activations, and overhead.

A purpose-built data center optimized for AI training rather than general cloud workloads, characterized by liquid-cooled high-density GPU racks, gigawatt-scale single-tenant power, and tightly-coupled networking.

Grouping multiple requests or training examples into a single forward or backward pass, the lever that turns GPU compute density into throughput.

Training a model across many independently-operated nodes that are not tightly coupled, contrasted with single-cluster training; the architecture for community-owned model production.

Microsoft's open-source training optimization library, originator of the ZeRO sharding technique and a peer to Megatron for distributed transformer training at scale.

The graphics memory generation on 2025-era consumer and workstation GPUs such as the RTX 5090 and RTX PRO 6000. High bandwidth per board, lower capacity than HBM.

An AI training facility whose power draw is measured in gigawatts rather than megawatts, the scale at which siting decisions become grid-and-permitting problems rather than real-estate ones.

A massively parallel processor originally designed for graphics, repurposed since the 2010s as the dominant compute substrate for both training and inference of large neural networks.

Stacked DRAM used as the main memory of every modern AI accelerator, with bandwidth in TB/s rather than GB/s and capacity per stack in tens of GB.

A high-throughput, low-latency network fabric (Mellanox, now NVIDIA) used for inter-node communication in AI training clusters, supporting RDMA for direct GPU-to-GPU transfer across machines.

The time from request submission to response completion, broken down for LLMs into time-to-first-token and time-per-output-token, the user-facing speed metric.

Low-power DRAM used as unified memory in Apple Silicon, DGX Spark, and Strix Halo. High capacity and efficiency, with bandwidth below HBM and GDDR.

NVIDIA's distributed-training framework for large transformer models, providing the reference implementation of tensor parallelism, pipeline parallelism, and 3D parallelism used in many open and closed training runs.

The rate (GB/s or TB/s) at which an accelerator reads its memory. It sets the ceiling on decode tokens/sec, since each token streams the active weights once.

A specialized cloud provider focused exclusively on GPU and AI workloads, operating outside the traditional AWS/Azure/GCP hyperscaler perimeter, with CoreWeave, Lambda, and Voltage Park as the canonical examples.

NVIDIA's proprietary GPU-to-GPU interconnect, providing bandwidth an order of magnitude above PCIe and the basis for tightly-coupled 8-GPU server nodes (DGX, HGX).

A networking technique that lets a remote machine read or write local memory without involving the CPU, foundational for high-throughput distributed training over InfiniBand or RoCE.

A performance model that bounds throughput by either compute or memory bandwidth, whichever is the limiting resource for an operation's arithmetic intensity.

The component in a serving or training system that decides which work runs next, balancing throughput, fairness, latency targets, and resource constraints.

A distributed training pattern where parameters, gradients, and optimizer states are split across GPUs (and sometimes hosts) so the total memory footprint scales with the cluster, not with each GPU.

A discounted cloud instance that the provider can reclaim with little warning, used for fault-tolerant training and batch inference where interruption is cheaper than reservation cost.

The rate at which a model produces output tokens, usually quoted as tokens-per-second per GPU or aggregate, the headline number for serving-cost economics.

Google's custom AI accelerator family, used internally for training Gemini and externally via Google Cloud, designed around dense matrix multiplication with a systolic array architecture.

A single physical memory pool shared by CPU and GPU, so the full capacity is usable as model memory; used by Apple Silicon, Strix Halo, and DGX Spark.

A subword tokenization algorithm that iteratively merges the most-frequent byte pairs in a corpus, producing a vocabulary that balances common-word coverage with arbitrary-text fallback.

A nonprofit-run repeated crawl of the public web maintained since 2007, the upstream raw source for nearly every open web-scale pretraining corpus.

An open large-scale web text dataset from Hugging Face, the highest-quality permissively-licensed pretraining corpus by 2024 to 2026 with ~15 trillion tokens after deduplication and filtering.

The model hub, dataset hub, and open-source library suite (Transformers, Datasets, Tokenizers, Accelerate, PEFT, TRL) that anchors the open-AI ecosystem's distribution and tooling layer.

The OSI's October 2024 definition of "open source AI," requiring not just weights but enough information about data, code, and architecture for third parties to reproduce the system.

The first and most compute-expensive training phase, where a base model learns general capabilities by predicting the next token on trillions of words of web and book data.

An early open reproduction of the Llama 1 pretraining corpus from Together AI (2023), now superseded by FineWeb and Dolma but historically important as the first open frontier-scale dataset.

An 825 GB diverse-source pretraining dataset assembled by EleutherAI in 2020, the open-corpus precedent that the later RedPajama and FineWeb projects expanded on.

The process of mapping raw text into the integer-ID sequences a model consumes, governed by the model's specific tokenizer; the rate-limiting interface between text and tensor.

The component that splits raw text into discrete units (tokens) the model can process, usually using a learned subword vocabulary like Byte-Pair Encoding.

A positional encoding that adds a linear bias to attention scores based on the distance between tokens, with no learned position parameters and natural length extrapolation.

The training-and-evaluation work of shaping a model's behavior to match human intent, refuse harmful requests, and answer honestly, distinct from raw capability training.

The transformer operation where each token computes a weighted average over all earlier tokens, with weights derived from learned similarity between query and key vectors.

An open YAML-driven fine-tuning framework that orchestrates Hugging Face Transformers, PEFT, TRL, and DeepSpeed for one-shot LoRA, QLoRA, and full fine-tuning workflows.

A standardized dataset and scoring rubric used to compare model capability on a defined task, the unit of model evaluation since GLUE made the format the default.

A 16-bit floating-point format with FP32's exponent range and only 7 mantissa bits. Designed for neural-network training; standard across 2026 accelerators alongside FP16.

A subword tokenization algorithm that iteratively merges the most-frequent byte pairs in a corpus, producing a vocabulary that balances common-word coverage with arbitrary-text fallback.

Anthropic's alignment technique where a model is trained to critique and revise its own outputs against a written list of principles (the "constitution"), reducing the need for human ranking labels.

The maximum number of tokens a model can attend to in a single forward pass, set during pretraining and extended (sometimes) via fine-tuning or training-free extrapolation tricks.

NVIDIA's parallel-computing platform and proprietary toolchain, the de facto programming model for GPU-accelerated machine learning since the late 2000s.

Training a model across many independently-operated nodes that are not tightly coupled, contrasted with single-cluster training; the architecture for community-owned model production.

A Chinese open-weight family known for the V3 MoE base model and the R1 reasoning model, both released under permissive licenses and unusually transparent in their training-cost reporting.

Microsoft's open-source training optimization library, originator of the ZeRO sharding technique and a peer to Megatron for distributed transformer training at scale.

A transformer where every parameter activates on every token; the conventional architecture before mixture of experts became common at frontier scale.

A preference-tuning method that optimizes a model on pairwise human rankings directly, bypassing the reward-model and reinforcement-learning steps of RLHF.

A fixed-size vector representation of a piece of text learned so semantically similar texts land near each other in the vector space, the basis for vector search and most RAG.

A parallelism strategy for mixture-of-experts models where different GPUs hold different experts; requires all-to-all communication on every token routing step.

Continued training of a pretrained base model on a smaller, task-specific dataset to specialize its behavior without retraining from scratch.

An open large-scale web text dataset from Hugging Face, the highest-quality permissively-licensed pretraining corpus by 2024 to 2026 with ~15 trillion tokens after deduplication and filtering.

A 16-bit floating-point format used as the default precision for deep learning training and inference, halving memory versus FP32 with small quality cost on most workloads.

An 8-bit floating-point format used for AI inference and increasingly for training, halving memory and bandwidth versus FP16 with minimal quality loss on most workloads.

An attention variant where multiple query heads share the same key and value heads, reducing KV cache size with little quality cost compared to full multi-head attention.

The model hub, dataset hub, and open-source library suite (Transformers, Datasets, Tokenizers, Accelerate, PEFT, TRL) that anchors the open-AI ecosystem's distribution and tooling layer.

A training technique where a small student model learns to mimic a larger teacher model's output distributions, transferring capability into a cheaper-to-serve form.

A parameter-efficient fine-tuning method that injects small low-rank adapter matrices into a frozen base model, training a tiny fraction of weights instead of the full model.

NVIDIA's distributed-training framework for large transformer models, providing the reference implementation of tensor parallelism, pipeline parallelism, and 3D parallelism used in many open and closed training runs.

A model architecture where each token activates only a fraction of total parameters by routing through learned expert subnetworks, decoupling capacity from compute.

MFU is the fraction of an accelerator's peak compute a workload actually achieves. The compute-bound analogue of MBU, relevant to prefill and training, not memory-bound decode.

Serving many LoRA adapters concurrently on a single base model, with the runtime swapping the right adapter in per request rather than loading separate fine-tuned copies.

A 4-bit normal-float quantization format from the QLoRA paper. The 16 quantization levels are spaced to match the empirical distribution of pretrained weights.

A family of fine-tuning methods that update only a small fraction of a base model's parameters, making fine-tuning feasible on consumer hardware and storage-efficient at deployment.

A measure of how well a language model predicts a text, equal to the exponential of the per-token cross-entropy loss; lower is better, often used for training diagnostics.

Everything that happens after pretraining ends: supervised fine-tuning, preference optimization, red-teaming, distillation, and safety work that turns a base into a shippable assistant.

The first and most compute-expensive training phase, where a base model learns general capabilities by predicting the next token on trillions of words of web and book data.

A fine-tuning method that combines 4-bit quantization of the frozen base model with LoRA adapters, making large-model fine-tuning fit on a single consumer GPU.

Training or fine-tuning a model with simulated quantization in the loop, so its weights stay accurate at low precision like int4 or FP8, unlike post-hoc post-training quantization.

An early open reproduction of the Llama 1 pretraining corpus from Together AI (2023), now superseded by FineWeb and Dolma but historically important as the first open frontier-scale dataset.

A post-training pipeline that uses human preference rankings to train a reward model, then optimizes a base model against that reward via reinforcement learning.

A positional encoding that rotates query and key vectors in two-dimensional subspaces by an angle proportional to their position, making attention scores depend on relative not absolute position.

The component in a serving or training system that decides which work runs next, balancing throughput, fairness, latency targets, and resource constraints.

A distributed training pattern where parameters, gradients, and optimizer states are split across GPUs (and sometimes hosts) so the total memory footprint scales with the cluster, not with each GPU.

A way to split a single model across multiple GPUs by sharding each layer's weight matrices and doing an all-reduce after every layer. Bandwidth-hungry but layer-by-layer fine-grained.

An 825 GB diverse-source pretraining dataset assembled by EleutherAI in 2020, the open-corpus precedent that the later RedPajama and FineWeb projects expanded on.

The process of mapping raw text into the integer-ID sequences a model consumes, governed by the model's specific tokenizer; the rate-limiting interface between text and tensor.

The component that splits raw text into discrete units (tokens) the model can process, usually using a learned subword vocabulary like Byte-Pair Encoding.

The neural network architecture that combines self-attention with feed-forward layers, dominant for language modeling since 2017 and the substrate for nearly every modern LLM.

Hugging Face's library for preference and reinforcement learning on transformer models, the canonical open implementation of RLHF, DPO, KTO, ORPO, and related preference-tuning methods.

An open fine-tuning library that uses hand-written Triton kernels and a manual gradient implementation to run LoRA and QLoRA fine-tuning roughly 2x faster than the Hugging Face baseline.

A position-encoding extension technique that lets a RoPE-pretrained model handle context windows longer than its training length without quality collapse.

License or terms-of-service clauses that prohibit certain uses (weapons, surveillance, harassment, child sexual abuse material), common on open-weight licenses but rejected by the strict open-source definition.

A positional encoding that adds a linear bias to attention scores based on the distance between tokens, with no learned position parameters and natural length extrapolation.

A permissive open-source license used by most open-weight model releases (Llama from 4 onward partial, Qwen, Mistral, DeepSeek, Falcon), allowing commercial use without acceptable-use restrictions.

The transformer operation where each token computes a weighted average over all earlier tokens, with weights derived from learned similarity between query and key vectors.

A post-training quantization method that protects the small fraction of weight channels that handle the largest activations, achieving 4-bit weights with little quality loss.

A standardized dataset and scoring rubric used to compare model capability on a defined task, the unit of model evaluation since GLUE made the format the default.

A 16-bit floating-point format with FP32's exponent range and only 7 mantissa bits. Designed for neural-network training; standard across 2026 accelerators alongside FP16.

The maximum number of tokens a model can attend to in a single forward pass, set during pretraining and extended (sometimes) via fine-tuning or training-free extrapolation tricks.

A Chinese open-weight family known for the V3 MoE base model and the R1 reasoning model, both released under permissive licenses and unusually transparent in their training-cost reporting.

A transformer where every parameter activates on every token; the conventional architecture before mixture of experts became common at frontier scale.

License clauses that limit which industries or applications a model may be deployed in, restricting use to non-competitive, non-commercial, or non-government purposes.

Continued training of a pretrained base model on a smaller, task-specific dataset to specialize its behavior without retraining from scratch.

A 16-bit floating-point format used as the default precision for deep learning training and inference, halving memory versus FP32 with small quality cost on most workloads.

A 4-bit floating-point format with hardware-native multiplication on Blackwell-generation accelerators. NVFP4 and MXFP4 variants target large-model inference and post-training quantization.

An 8-bit floating-point format used for AI inference and increasingly for training, halving memory and bandwidth versus FP16 with minimal quality loss on most workloads.

The current capability envelope of AI, defined by the most capable models in deployment at any given time; an evolving label rather than a fixed threshold.

Google's open-weight model family derived from Gemini research, with source-available licensing that includes an acceptable-use clause and license-revocation hook.

A binary container format for quantized model weights used by llama.cpp and its ecosystem; the dominant on-device LLM file format since 2023.

A post-training quantization method that compresses transformer weights to 3 or 4 bits layer-by-layer with one-shot optimization against calibration data.

An attention variant where multiple query heads share the same key and value heads, reducing KV cache size with little quality cost compared to full multi-head attention.

The model hub, dataset hub, and open-source library suite (Transformers, Datasets, Tokenizers, Accelerate, PEFT, TRL) that anchors the open-AI ecosystem's distribution and tooling layer.

An attention design that interleaves different mechanisms across layers, typically global plus sliding-window, to combine quality with long-context efficiency.

A training technique where a small student model learns to mimic a larger teacher model's output distributions, transferring capability into a cheaper-to-serve form.

Meta's open-weight model family, the most widely deployed open release through 2024 to 2026, released under the source-available Community License with an MAU cap and acceptable-use clause.

Meta's open content-moderation model line, designed to classify prompts and responses against a configurable taxonomy of harms, deployable as an input/output filter.

A parameter-efficient fine-tuning method that injects small low-rank adapter matrices into a frozen base model, training a tiny fraction of weights instead of the full model.

A user-count metric used in restrictive open-weights licenses (notably Llama's Community License) to trigger a requirement to negotiate a separate commercial license at scale.

Standard transformer attention where each layer has N independent query, key, and value heads; foundational but memory-heavy as context windows grow.

A French open-weight model family from Mistral AI, released mostly under Apache 2.0 with strong performance per parameter and notable MoE variants (Mixtral, Mixtral 8x22B).

Mistral AI's MoE model line, with Mixtral 8x7B (the first widely-adopted open mixture-of-experts model) and the larger Mixtral 8x22B as its two flagship releases.

A model architecture where each token activates only a fraction of total parameters by routing through learned expert subnetworks, decoupling capacity from compute.

An attention variant introduced in DeepSeek-V2 that compresses keys and values through a learned low-rank projection, dramatically shrinking the KV cache.

An attention variant where N query heads share a single key and value head, minimizing KV cache memory at a modest quality cost compared to multi-head attention.

Serving many LoRA adapters concurrently on a single base model, with the runtime swapping the right adapter in per request rather than loading separate fine-tuned copies.

A 4-bit normal-float quantization format from the QLoRA paper. The 16 quantization levels are spaced to match the empirical distribution of pretrained weights.

A model release that publishes the trained parameters under some downloadable license, distinct from "open source" which (per OSAID) also requires data and training-code openness.

The OSI's October 2024 definition of "open source AI," requiring not just weights but enough information about data, code, and architecture for third parties to reproduce the system.

A family of fine-tuning methods that update only a small fraction of a base model's parameters, making fine-tuning feasible on consumer hardware and storage-efficient at deployment.

Everything that happens after pretraining ends: supervised fine-tuning, preference optimization, red-teaming, distillation, and safety work that turns a base into a shippable assistant.

The first and most compute-expensive training phase, where a base model learns general capabilities by predicting the next token on trillions of words of web and book data.

A fine-tuning method that combines 4-bit quantization of the frozen base model with LoRA adapters, making large-model fine-tuning fit on a single consumer GPU.

Storing or computing model weights in lower-precision number formats (FP8, INT8, INT4) to reduce memory and bandwidth, accepting small quality loss.

Training or fine-tuning a model with simulated quantization in the loop, so its weights stay accurate at low precision like int4 or FP8, unlike post-hoc post-training quantization.

Alibaba's open-weight model family, leading the multilingual and Chinese-language open-weight space, released under Apache 2.0 with sizes from 0.6B to 235B parameters.

A positional encoding that rotates query and key vectors in two-dimensional subspaces by an angle proportional to their position, making attention scores depend on relative not absolute position.

An attention pattern where each token attends only to a fixed window of recent tokens, trading global lookup for linear-cost inference at long sequence lengths.

A license category that lets users read and modify the code or weights but imposes restrictions (use limits, non-compete, MAU thresholds) that exclude it from the strict open-source definition.

An alternative to attention that processes sequences via a learned linear recurrence; scales linearly with sequence length where attention scales quadratically.

The neural network architecture that combines self-attention with feed-forward layers, dominant for language modeling since 2017 and the substrate for nearly every modern LLM.

The first-pass formula for whether a model fits on a GPU. VRAM ≈ parameters × (bits ÷ 8), plus 10-30 percent for KV cache, activations, and overhead.

A position-encoding extension technique that lets a RoPE-pretrained model handle context windows longer than its training length without quality collapse.

A positional encoding that adds a linear bias to attention scores based on the distance between tokens, with no learned position parameters and natural length extrapolation.

FLOPs performed per byte read from memory. Low intensity means an operation is memory-bound; high intensity means compute-bound. LLM decode has very low intensity.

The transformer operation where each token computes a weighted average over all earlier tokens, with weights derived from learned similarity between query and key vectors.

A post-training quantization method that protects the small fraction of weight channels that handle the largest activations, achieving 4-bit weights with little quality loss.

Grouping multiple requests or training examples into a single forward or backward pass, the lever that turns GPU compute density into throughput.

A 16-bit floating-point format with FP32's exponent range and only 7 mantissa bits. Designed for neural-network training; standard across 2026 accelerators alongside FP16.

The umbrella category of compute architectures where workloads run isolated from the host operator, combining hardware TEEs, attestation, and encrypted-memory protections.

Masking the model's next-token probabilities at each step so only tokens allowed by a grammar or schema can be sampled, forcing the output to be structurally valid.

The maximum number of tokens a model can attend to in a single forward pass, set during pretraining and extended (sometimes) via fine-tuning or training-free extrapolation tricks.

A request-scheduling pattern where the inference engine adds new requests to the running batch as soon as one finishes a token, instead of waiting for the whole batch to complete.

NVIDIA's parallel-computing platform and proprietary toolchain, the de facto programming model for GPU-accelerated machine learning since the late 2000s.

The second phase of LLM inference, generating one token at a time from the KV cache. Memory-bandwidth-bound; throughput tracks memory bandwidth more than peak compute.

A transformer where every parameter activates on every token; the conventional architecture before mixture of experts became common at frontier scale.

A parallelism strategy for mixture-of-experts models where different GPUs hold different experts; requires all-to-all communication on every token routing step.

An exact attention algorithm that reorders the computation to avoid materializing the full attention matrix in GPU HBM, giving 2 to 4 times speedup with no quality loss.

A 16-bit floating-point format used as the default precision for deep learning training and inference, halving memory versus FP32 with small quality cost on most workloads.

A 4-bit floating-point format with hardware-native multiplication on Blackwell-generation accelerators. NVFP4 and MXFP4 variants target large-model inference and post-training quantization.

An 8-bit floating-point format used for AI inference and increasingly for training, halving memory and bandwidth versus FP16 with minimal quality loss on most workloads.

A binary container format for quantized model weights used by llama.cpp and its ecosystem; the dominant on-device LLM file format since 2023.

A post-training quantization method that compresses transformer weights to 3 or 4 bits layer-by-layer with one-shot optimization against calibration data.

A massively parallel processor originally designed for graphics, repurposed since the 2010s as the dominant compute substrate for both training and inference of large neural networks.

An attention variant where multiple query heads share the same key and value heads, reducing KV cache size with little quality cost compared to full multi-head attention.

A formal description of allowed output structure (for example llama.cpp's GBNF) that a decoder is constrained to; a JSON Schema can be compiled into one.

An AI inference company with custom deterministic LPU chips and a hosted inference service that achieves extremely low time-per-token (1000+ tokens/sec on 70B models).

An attention design that interleaves different mechanisms across layers, typically global plus sliding-window, to combine quality with long-context efficiency.

Running a trained model to produce outputs (tokens, images, embeddings) from inputs at serving time, as distinct from the gradient updates of training.

A weaker predecessor to structured output: the model is constrained to emit syntactically valid JSON, but not necessarily JSON that conforms to a specific schema.

The stored key and value vectors from previously processed tokens, reused at each generation step so an autoregressive model does not recompute attention over the entire prefix.

The time from request submission to response completion, broken down for LLMs into time-to-first-token and time-per-output-token, the user-facing speed metric.

Georgi Gerganov's C++ inference engine optimized for CPUs and consumer GPUs, the on-device standard and the engine behind Ollama, LM Studio, and most local-first AI products.

A desktop application for running open-weight models locally with a GUI, model browser, and OpenAI-compatible local server, targeting users who prefer apps over command-line tools.

An architecture stance where inference (and increasingly memory and agent state) runs on the user's own device rather than a remote API, prioritizing privacy, latency, and offline operation.

The rate (GB/s or TB/s) at which an accelerator reads its memory. It sets the ceiling on decode tokens/sec, since each token streams the active weights once.

Standard transformer attention where each layer has N independent query, key, and value heads; foundational but memory-heavy as context windows grow.

Mistral AI's MoE model line, with Mixtral 8x7B (the first widely-adopted open mixture-of-experts model) and the larger Mixtral 8x22B as its two flagship releases.

A model architecture where each token activates only a fraction of total parameters by routing through learned expert subnetworks, decoupling capacity from compute.

An attention variant introduced in DeepSeek-V2 that compresses keys and values through a learned low-rank projection, dramatically shrinking the KV cache.

Apple's open-source ML framework designed for Apple Silicon's unified memory architecture, the local-first inference engine for Mac and increasingly iPad and iPhone.

MBU is the fraction of an accelerator's peak memory bandwidth a serving stack actually reaches during decode. Real systems land around 60 to 85 percent.

MFU is the fraction of an accelerator's peak compute a workload actually achieves. The compute-bound analogue of MBU, relevant to prefill and training, not memory-bound decode.

An attention variant where N query heads share a single key and value head, minimizing KV cache memory at a modest quality cost compared to multi-head attention.

Serving many LoRA adapters concurrently on a single base model, with the runtime swapping the right adapter in per request rather than loading separate fine-tuned copies.

A 4-bit normal-float quantization format from the QLoRA paper. The 16 quantization levels are spaced to match the empirical distribution of pretrained weights.

A local inference runtime that wraps llama.cpp with a Docker-style developer experience, the easiest path to running open-weight models on a personal machine.

Running model inference on the user's local hardware (phone, laptop, embedded device), enabled by smaller models, FP8 quantization, and runtimes like llama.cpp and MLX.

An open interchange format for machine learning models, designed to let a model trained in one framework run in another via a portable graph representation.

An attention implementation that manages the KV cache in fixed-size blocks like operating-system virtual memory, eliminating fragmentation and letting many concurrent requests share GPU memory efficiently.

When a model emits several independent tool calls in a single turn so the host can run them concurrently, instead of one call per round-trip.

A volunteer-pooled inference system that runs large open-weight models across many internet-connected nodes, each holding a slice of the model, with users dispatching forward passes through the swarm.

The first phase of LLM inference, processing the input prompt and building the initial KV cache. Compute-bound and parallel across prompt tokens.

A serving optimization that stores the KV cache for shared prompt prefixes (system prompts, few-shot examples) so subsequent requests reusing them skip the prefill compute.

Storing or computing model weights in lower-precision number formats (FP8, INT8, INT4) to reduce memory and bandwidth, accepting small quality loss.

Training or fine-tuning a model with simulated quantization in the loop, so its weights stay accurate at low precision like int4 or FP8, unlike post-hoc post-training quantization.

A KV cache management scheme used by SGLang that organizes shared prompt prefixes as a radix tree, letting many requests with overlapping prefixes reuse cached attention state.

AMD's open-source GPU compute stack, the main credible alternative to CUDA, with growing coverage in PyTorch and vLLM but still trailing on kernel maturity and tooling.

A performance model that bounds throughput by either compute or memory bandwidth, whichever is the limiting resource for an operation's arithmetic intensity.

A positional encoding that rotates query and key vectors in two-dimensional subspaces by an angle proportional to their position, making attention scores depend on relative not absolute position.

The component in a serving or training system that decides which work runs next, balancing throughput, fairness, latency targets, and resource constraints.

An open inference engine from the LMSYS team featuring RadixAttention for prefix sharing and a structured-generation frontend, particularly strong on agent and tool-calling workloads.

An attention pattern where each token attends only to a fixed window of recent tokens, trading global lookup for linear-cost inference at long sequence lengths.

An inference acceleration technique where a small fast draft model proposes several tokens at once and the target model verifies them in parallel, giving 2-3x speedup with no quality loss.

An alternative to attention that processes sequences via a learned linear recurrence; scales linearly with sequence length where attention scales quadratically.

Making a model emit text that conforms to a schema or type (often JSON matching a JSON Schema) rather than free-form prose, so downstream code can parse it reliably.

A way to split a single model across multiple GPUs by sharding each layer's weight matrices and doing an all-reduce after every layer. Bandwidth-hungry but layer-by-layer fine-grained.

NVIDIA's closed-source inference engine for NVIDIA GPUs, the fastest runtime on Hopper and Blackwell but tied to NVIDIA's proprietary kernel stack and CUDA.

Hugging Face's production inference server, an early peer of vLLM that ceded throughput leadership in 2024 and now sits in maintenance mode behind vLLM and SGLang.

The rate at which a model produces output tokens, usually quoted as tokens-per-second per GPU or aggregate, the headline number for serving-cost economics.

The process of mapping raw text into the integer-ID sequences a model consumes, governed by the model's specific tokenizer; the rate-limiting interface between text and tensor.

The component that splits raw text into discrete units (tokens) the model can process, usually using a learned subword vocabulary like Byte-Pair Encoding.

The headline inference speed metric. Decode tokens/sec is what a user feels as text streams; it is bounded by memory bandwidth divided by the bytes streamed per token.

Time per output token. The latency between successive tokens during decode; tracks memory bandwidth and concurrent batch size more than peak compute.

The neural network architecture that combines self-attention with feed-forward layers, dominant for language modeling since 2017 and the substrate for nearly every modern LLM.

Time to first token. The latency from request received to the first output token streamed back; dominated by prompt-prefill cost and scheduler queueing.

A single physical memory pool shared by CPU and GPU, so the full capacity is usable as model memory; used by Apple Silicon, Strix Halo, and DGX Spark.

An inference architecture that provides cryptographic proof the claimed model produced the claimed output, via TEE attestation, zero-knowledge proofs (ZKML), or proof-of-sample-correctness schemes.

An open-source inference engine introduced by UC Berkeley in 2023, built around PagedAttention to manage KV cache memory and serve tokens efficiently under load.

The first-pass formula for whether a model fits on a GPU. VRAM ≈ parameters × (bits ÷ 8), plus 10-30 percent for KV cache, activations, and overhead.

A position-encoding extension technique that lets a RoPE-pretrained model handle context windows longer than its training length without quality collapse.

Zero-knowledge proofs of correct machine-learning inference, letting a prover convince a verifier that a specific model produced a specific output without revealing model or input.

The persistent state an agent carries across turns and sessions, ranging from session-scoped scratchpads to long-term knowledge bases the agent reads and writes itself.

A classical lexical ranking function for information retrieval, based on term frequency and inverse document frequency with saturation, still the strong lexical baseline for hybrid search.

Splitting source documents into smaller passages for embedding and retrieval, where the chunk size and overlap directly affect retrieval quality and context efficiency.

A retrieval model that produces per-token embeddings for documents and queries, then ranks by summing the maximum similarity across query tokens, more accurate than single-vector retrieval.

A fixed-size vector representation of a piece of text learned so semantically similar texts land near each other in the vector space, the basis for vector search and most RAG.

The earliest widely-adopted LLM agent and RAG orchestration framework (2022), now with the LangGraph extension for stateful multi-step agent workflows.

An open-source RAG framework focused on connecting LLMs to external data, with strong document-ingestion tooling and a smaller surface area than LangChain.

A pattern where a model retrieves relevant documents from an external store at query time and conditions its answer on them, instead of relying only on parametric knowledge.

A second-pass scoring step that takes the top-k candidates from initial retrieval and rescores them with a more expensive but more accurate cross-encoder model.

Search that matches by meaning rather than literal terms, using embeddings to rank results by similarity to the query's intent rather than its surface tokens.

A datastore optimized for approximate nearest-neighbor search over high-dimensional embedding vectors, the storage substrate for most RAG and recommendation pipelines.

A Google-launched open protocol for agent-to-agent communication, letting agents from different vendors discover each other's capabilities and exchange structured messages.

The persistent state an agent carries across turns and sessions, ranging from session-scoped scratchpads to long-term knowledge bases the agent reads and writes itself.

An informal descriptor for AI systems that pursue multi-step goals via tool use, planning, and self-correction, rather than single-turn question-answering.

A Microsoft Research framework for multi-agent systems, with a conversation-pattern API for orchestrating multiple specialized agents to solve tasks collaboratively.

A widely-cited benchmark for function-calling ability that checks emitted calls against expected ones by abstract-syntax-tree match and by executing them, across simple, parallel, and multi-turn cases.

A prompting and training technique where the model emits step-by-step intermediate reasoning before its final answer, improving accuracy on multi-step problems.

Masking the model's next-token probabilities at each step so only tokens allowed by a grammar or schema can be sampled, forcing the output to be structurally valid.

A pattern where a model emits a structured call (function name plus arguments), the runtime executes it, and the result returns as input on the model's next turn.

Block's open-source coding agent, BYOK across multiple model providers, with MCP support and a permissive license; the most cited fully-open agent platform in 2026.

A formal description of allowed output structure (for example llama.cpp's GBNF) that a decoder is constrained to; a JSON Schema can be compiled into one.

Running a trained model to produce outputs (tokens, images, embeddings) from inputs at serving time, as distinct from the gradient updates of training.

A weaker predecessor to structured output: the model is constrained to emit syntactically valid JSON, but not necessarily JSON that conforms to a specific schema.

The earliest widely-adopted LLM agent and RAG orchestration framework (2022), now with the LangGraph extension for stateful multi-step agent workflows.

An open-source RAG framework focused on connecting LLMs to external data, with strong document-ingestion tooling and a smaller surface area than LangChain.

An architecture stance where inference (and increasingly memory and agent state) runs on the user's own device rather than a remote API, prioritizing privacy, latency, and offline operation.

An open protocol from Anthropic that standardizes how language models discover and call external tools, data sources, and prompts via a small JSON-RPC interface.

Architectures where multiple LLM-driven agents collaborate or compete on a task, each with its own role, prompt, or specialization, coordinated by an orchestrator or message-passing protocol.

NVIDIA's open framework for programmable safety, topic, and conversation guardrails around LLM applications, using a Colang DSL to define allowed and disallowed conversation flows.

When a model emits several independent tool calls in a single turn so the host can run them concurrently, instead of one call per round-trip.

An attack where adversarial content in a document, tool result, or web page is interpreted as instructions by the model, overriding the user or system prompt.

A pattern where a model retrieves relevant documents from an external store at query time and conditions its answer on them, instead of relying only on parametric knowledge.

An agent loop where the model alternates between reasoning steps (thought) and acting steps (tool call), explicitly interleaving free-form deliberation with structured tool use.

An open inference engine from the LMSYS team featuring RadixAttention for prefix sharing and a structured-generation frontend, particularly strong on agent and tool-calling workloads.

Making a model emit text that conforms to a schema or type (often JSON matching a JSON Schema) rather than free-form prose, so downstream code can parse it reliably.

An agentic benchmark where a model completes realistic retail and airline customer-service tasks through tool calls and simulated user turns, scored by reliability across repeated runs.

The general pattern of an LLM invoking external functions, APIs, or systems to fetch data or take action, distinct from generating an answer purely from its weights.

The host-orchestrated cycle behind tool use: the model emits a tool call, the host (not the model) executes it, and the result is fed back for the next turn.

A prompting pattern that has the model generate and evaluate multiple branching reasoning paths, then select or backtrack rather than committing to a single chain of thought.

An open-source inference engine introduced by UC Berkeley in 2023, built around PagedAttention to manage KV cache memory and serve tokens efficiently under load.

A Google-launched open protocol for agent-to-agent communication, letting agents from different vendors discover each other's capabilities and exchange structured messages.

An informal descriptor for AI systems that pursue multi-step goals via tool use, planning, and self-correction, rather than single-turn question-answering.

The class of payment flows initiated and settled by autonomous AI agents on a user's behalf, distinct from human-initiated checkout flows.

A protocol market matching GPU supply from many independent providers to AI demand, settled on a token rail; Akash, io.net, Bittensor compute, and Hyperbolic are canonical.

A pattern where a model emits a structured call (function name plus arguments), the runtime executes it, and the result returns as input on the model's next turn.

A Lightning-Labs protocol that pairs HTTP 402 Payment Required with Lightning Network invoices, enabling sub-cent metered payments for APIs and content.

An open protocol from Anthropic that standardizes how language models discover and call external tools, data sources, and prompts via a small JSON-RPC interface.

The general pattern of an LLM invoking external functions, APIs, or systems to fetch data or take action, distinct from generating an answer purely from its weights.

The host-orchestrated cycle behind tool use: the model emits a tool call, the host (not the model) executes it, and the result is fed back for the next turn.

An open protocol revived by Coinbase in 2025 that uses the long-reserved HTTP 402 "Payment Required" status to let agents and APIs settle micropayments, including in stablecoins.

A standardized dataset and scoring rubric used to compare model capability on a defined task, the unit of model evaluation since GLUE made the format the default.

A widely-cited benchmark for function-calling ability that checks emitted calls against expected ones by abstract-syntax-tree match and by executing them, across simple, parallel, and multi-turn cases.

A prompting and training technique where the model emits step-by-step intermediate reasoning before its final answer, improving accuracy on multi-step problems.

The current capability envelope of AI, defined by the most capable models in deployment at any given time; an evolving label rather than a fixed threshold.

A model output that is fluent and plausible-sounding but factually wrong, ranging from invented citations and APIs to fabricated names, dates, and quotes.

The model hub, dataset hub, and open-source library suite (Transformers, Datasets, Tokenizers, Accelerate, PEFT, TRL) that anchors the open-AI ecosystem's distribution and tooling layer.

An OpenAI benchmark of 164 Python programming problems scored by whether unit tests pass, the default LLM-coding benchmark from 2021 until saturation in 2024.

Running a trained model to produce outputs (tokens, images, embeddings) from inputs at serving time, as distinct from the gradient updates of training.

The time from request submission to response completion, broken down for LLMs into time-to-first-token and time-per-output-token, the user-facing speed metric.

A ranked listing of models scored on one benchmark or aggregate, with LMArena and SWE-Bench Verified as the main 2026 reference points and the Open LLM Leaderboard now archived.

EleutherAI's open-source evaluation framework that runs hundreds of standardized benchmarks against any Hugging Face or OpenAI-compatible model, the de facto reference harness behind the Open LLM Leaderboard.

A multiple-choice benchmark covering 57 academic and professional subjects, once the default capability score, now largely saturated by frontier models above 88% accuracy.

A measure of how well a language model predicts a text, equal to the exponential of the per-token cross-entropy loss; lower is better, often used for training diagnostics.

Making a model emit text that conforms to a schema or type (often JSON matching a JSON Schema) rather than free-form prose, so downstream code can parse it reliably.

An agentic benchmark where a model completes realistic retail and airline customer-service tasks through tool calls and simulated user turns, scored by reliability across repeated runs.

The rate at which a model produces output tokens, usually quoted as tokens-per-second per GPU or aggregate, the headline number for serving-cost economics.

The headline inference speed metric. Decode tokens/sec is what a user feels as text streams; it is bounded by memory bandwidth divided by the bytes streamed per token.

Time per output token. The latency between successive tokens during decode; tracks memory bandwidth and concurrent batch size more than peak compute.

Time to first token. The latency from request received to the first output token streamed back; dominated by prompt-prefill cost and scheduler queueing.

License or terms-of-service clauses that prohibit certain uses (weapons, surveillance, harassment, child sexual abuse material), common on open-weight licenses but rejected by the strict open-source definition.

A strong-copyleft license that extends GPL's source-distribution requirement to network-served software, the strongest open-source license to deter proprietary SaaS deployment.

The training-and-evaluation work of shaping a model's behavior to match human intent, refuse harmful requests, and answer honestly, distinct from raw capability training.

A permissive open-source license used by most open-weight model releases (Llama from 4 onward partial, Qwen, Mistral, DeepSeek, Falcon), allowing commercial use without acceptable-use restrictions.

License clauses that limit which industries or applications a model may be deployed in, restricting use to non-competitive, non-commercial, or non-government purposes.

The current capability envelope of AI, defined by the most capable models in deployment at any given time; an evolving label rather than a fixed threshold.

Google's open-weight model family derived from Gemini research, with source-available licensing that includes an acceptable-use clause and license-revocation hook.

The regulatory queue a new generation or load project must traverse to connect to the grid; currently the binding constraint on how fast gigawatt-class AI sites come online.

The capital expenditure that Microsoft, Google, Amazon, Meta, and Oracle are spending on AI infrastructure, totaling roughly $300B+ annually by 2025-2026 and dominating the supply-and-demand signal for the entire stack.

Meta's open-weight model family, the most widely deployed open release through 2024 to 2026, released under the source-available Community License with an MAU cap and acceptable-use clause.

A user-count metric used in restrictive open-weights licenses (notably Llama's Community License) to trigger a requirement to negotiate a separate commercial license at scale.

An open interchange format for machine learning models, designed to let a model trained in one framework run in another via a portable graph representation.

A model release that publishes the trained parameters under some downloadable license, distinct from "open source" which (per OSAID) also requires data and training-code openness.

The OSI's October 2024 definition of "open source AI," requiring not just weights but enough information about data, code, and architecture for third parties to reproduce the system.

The nonprofit that maintains the canonical Open Source Definition for software since 1998, and the OSAID definition for AI as of 2024.

An open instruction set architecture, royalty-free and modular, increasingly used in AI accelerator cores (Tenstorrent, SiFive Intelligence) as the open alternative to ARM and x86.

A post-training pipeline that uses human preference rankings to train a reward model, then optimizes a base model against that reward via reinforcement learning.

A license category that lets users read and modify the code or weights but imposes restrictions (use limits, non-compete, MAU thresholds) that exclude it from the strict open-source definition.

AI compute capacity owned, operated, or contractually controlled by a nation-state for the use of its own institutions and citizens, distinct from rented capacity on US-hyperscaler clouds.

A cryptographic protocol that lets a remote party verify which code is running inside a TEE, including which model is loaded and which build of the inference engine.

The umbrella category of compute architectures where workloads run isolated from the host operator, combining hardware TEEs, attestation, and encrypted-memory protections.

Intel's earliest mainstream trusted execution environment, the predecessor to TDX, with smaller enclave sizes and a history of side-channel vulnerabilities that limited its deployment for AI.

A hardware-isolated CPU region where code and data are protected from inspection by the host OS, used to run inference in a way the operator cannot read or modify.

An inference architecture that provides cryptographic proof the claimed model produced the claimed output, via TEE attestation, zero-knowledge proofs (ZKML), or proof-of-sample-correctness schemes.

Zero-knowledge proofs of correct machine-learning inference, letting a prover convince a verifier that a specific model produced a specific output without revealing model or input.

License or terms-of-service clauses that prohibit certain uses (weapons, surveillance, harassment, child sexual abuse material), common on open-weight licenses but rejected by the strict open-source definition.

The training-and-evaluation work of shaping a model's behavior to match human intent, refuse harmful requests, and answer honestly, distinct from raw capability training.

Anthropic's alignment technique where a model is trained to critique and revise its own outputs against a written list of principles (the "constitution"), reducing the need for human ranking labels.

A preference-tuning method that optimizes a model on pairwise human rankings directly, bypassing the reward-model and reinforcement-learning steps of RLHF.

A model output that is fluent and plausible-sounding but factually wrong, ranging from invented citations and APIs to fabricated names, dates, and quotes.

An adversarial input that causes a model to bypass its training-time refusal policy and produce content it would normally refuse, distinct from prompt injection's instruction-hijack.

Meta's open content-moderation model line, designed to classify prompts and responses against a configurable taxonomy of harms, deployable as an input/output filter.

NVIDIA's open framework for programmable safety, topic, and conversation guardrails around LLM applications, using a Colang DSL to define allowed and disallowed conversation flows.

Everything that happens after pretraining ends: supervised fine-tuning, preference optimization, red-teaming, distillation, and safety work that turns a base into a shippable assistant.

An attack where adversarial content in a document, tool result, or web page is interpreted as instructions by the model, overriding the user or system prompt.

A post-training pipeline that uses human preference rankings to train a reward model, then optimizes a base model against that reward via reinforcement learning.

The class of payment flows initiated and settled by autonomous AI agents on a user's behalf, distinct from human-initiated checkout flows.

A protocol market matching GPU supply from many independent providers to AI demand, settled on a token rail; Akash, io.net, Bittensor compute, and Hyperbolic are canonical.

Training a model across many independently-operated nodes that are not tightly coupled, contrasted with single-cluster training; the architecture for community-owned model production.

A Lightning-Labs protocol that pairs HTTP 402 Payment Required with Lightning Network invoices, enabling sub-cent metered payments for APIs and content.

Georgi Gerganov's C++ inference engine optimized for CPUs and consumer GPUs, the on-device standard and the engine behind Ollama, LM Studio, and most local-first AI products.

A desktop application for running open-weight models locally with a GUI, model browser, and OpenAI-compatible local server, targeting users who prefer apps over command-line tools.

An architecture stance where inference (and increasingly memory and agent state) runs on the user's own device rather than a remote API, prioritizing privacy, latency, and offline operation.

Apple's open-source ML framework designed for Apple Silicon's unified memory architecture, the local-first inference engine for Mac and increasingly iPad and iPhone.

A long-term contract under which an AI operator commits to buy a defined nuclear-generated power output, becoming the cornerstone financing mechanism for gigawatt-class AI buildouts.

A local inference runtime that wraps llama.cpp with a Docker-style developer experience, the easiest path to running open-weight models on a personal machine.

Running model inference on the user's local hardware (phone, laptop, embedded device), enabled by smaller models, FP8 quantization, and runtimes like llama.cpp and MLX.

A volunteer-pooled inference system that runs large open-weight models across many internet-connected nodes, each holding a slice of the model, with users dispatching forward passes through the swarm.

An open instruction set architecture, royalty-free and modular, increasingly used in AI accelerator cores (Tenstorrent, SiFive Intelligence) as the open alternative to ARM and x86.

AI compute capacity owned, operated, or contractually controlled by a nation-state for the use of its own institutions and citizens, distinct from rented capacity on US-hyperscaler clouds.

An AI accelerator startup designing RISC-V-based chips (Wormhole, Blackhole, Grendel) with an open software stack, positioned as the leading open alternative to NVIDIA at the silicon layer.

An open protocol revived by Coinbase in 2025 that uses the long-reserved HTTP 402 "Payment Required" status to let agents and APIs settle micropayments, including in stablecoins.

Zero-knowledge proofs of correct machine-learning inference, letting a prover convince a verifier that a specific model produced a specific output without revealing model or input.