AI Safety Science program

Schmidt Sciences announced the AI Safety Science program in February 2025, selecting 27 projects with $10M total, an average of approximately $370K per project. The funder framing is the "Science of Trustworthy AI," with grants targeting concrete, implementable technical methods for testing and evaluating large language models so they are less likely to cause harm or be misused.

Featured awardees include Yoshua Bengio (Mila, Quebec AI Institute) on AI risk mitigation technology, Zico Kolter (Carnegie Mellon, also an OpenAI board member), Florian Tramèr and Mrinmaya Sachan (ETH Zurich), Andrea Bajcsy (Carnegie Mellon), Arvind Narayanan (Princeton), Karthik Narasimhan, Aditi Raghunathan (CMU), Daniel Kang and Bo Li (UIUC), Huan Sun (Ohio State), and Ziang Xiao (Johns Hopkins). The roster skews toward established alignment and ML-safety faculty rather than early-career researchers.

The project portfolio organizes into themes around understanding safety in AI systems and inference-time compute. Named projects include "Tests of Compositional Generalization as an Upper Bound on AI Safety Risks," "LLM-Derived Guardrail for Frontier Models as a Step Towards a Cautious Scientist AI," "Mechanistic Interpretability to Detect Test Set Contamination," "Five Nines of Reliability for AI Agents," "OpenAgentSafety: Measuring and Mitigating Safety Harms," "Formalizing Prompt Injection Defenses," and "Tracing and Eliminating Harmful Capabilities Across Model Generations."

Program operations include computational support from the Center for AI Safety and API access from OpenAI to grantees, which lowers the marginal cost of empirical safety research that needs frontier-model queries. This structural detail (compute + API access bundled with cash) is unusual and partially substitutes for what individual labs would otherwise spend on inference budgets.

Compared with the parallel AI2050 fellowship program, AI Safety Science funds projects rather than individuals and has a tighter thematic frame. Together the two programs make Schmidt Sciences one of the largest non-government AI-safety funders of the 2025 cohort year. A 2026 follow-on Trustworthy AI RFP has been announced through Schmidt Sciences' grants pipeline.