confidential computing

The architecture pattern that lets a workload run on infrastructure the operator does not trust. The components are: a hardware-isolated runtime (a TEE on CPU, NVIDIA’s confidential-compute mode on H100/ H200, or a similar GPUⓘGPUsiliconA massively parallel processor originally designed for graphics, repurposed since the 2010s as the dominant compute substrate for both training and inference of large neural networks. Chat about this → Open full entry → isolation), remote attestationⓘattestationidentity-trustA cryptographic protocol that lets a remote party verify which code is running inside a TEE, including which model is loaded and which build of the inference engine. Chat about this → Open full entry → that proves the expected code is running, and encrypted communication channels that keep secrets out of the operator’s reach end-to-end.

For AI, confidential computing matters because hosted inferenceⓘinferenceruntimeRunning a trained model to produce outputs (tokens, images, embeddings) from inputs at serving time, as distinct from the gradient updates of training. Chat about this → Open full entry → is otherwise a trust nightmare: prompts and outputs pass through provider infrastructure that could log, train on, or leak them. Apple Private Cloud Compute (custom Apple-silicon servers), AWS Nitro Enclaves, Anthropic Confidential InferenceⓘinferenceruntimeRunning a trained model to produce outputs (tokens, images, embeddings) from inputs at serving time, as distinct from the gradient updates of training. Chat about this → Open full entry →, and projects like Maple AI all build on confidential-compute primitives to give users cryptographic instead of contractual privacy.

The remaining gaps are GPUⓘGPUsiliconA massively parallel processor originally designed for graphics, repurposed since the 2010s as the dominant compute substrate for both training and inference of large neural networks. Chat about this → Open full entry → coverage (only NVIDIA Hopper and Blackwell support confidential modes; AMD and Intel GPUs lag), performance overhead (10 to 30 percent on transformerⓘtransformerruntimeThe neural network architecture that combines self-attention with feed-forward layers, dominant for language modeling since 2017 and the substrate for nearly every modern LLM. Chat about this → Open full entry → workloads, depending on mode), and the attestationⓘattestationidentity-trustA cryptographic protocol that lets a remote party verify which code is running inside a TEE, including which model is loaded and which build of the inference engine. Chat about this → Open full entry → chain (the user needs to trust the hardware vendor’s signing keys).