Self-study
These curricula and reading lists enable you to dive deeper into AI safety through independent learning.
Fundamental reading
List of sequences curated by the AI Alignment Forum team, featuring work from Richard Ngo, Paul Christiano, etc.
Category: Technical alignment
Type: Reading list
- Recommended introductory course
AI Risk Fundamentals (opens in new tab)
By Lens Academy
Based heavily on the book "If Anyone Builds It, Everyone Dies" – covering the core case for why superintelligent AI could be catastrophic, and what should be done about it.
Category: General intro
Type: Curriculum
DeepMind AGI Safety Short Course (opens in new tab)
By Google DeepMind
75-minute course covering alignment problems expected as AI capabilities advance. Includes recorded talks, exercises, and an accompanying workbook.
Category: General intro
Type: Curriculum

Agent Foundations for Superintelligence-Robust Alignment (opens in new tab)
By Alignment Ecosystem Development (AED)
Guide to the cluster of thought which expects solving alignment in a way which scales to superintelligence to require well-specified proposals guided by strong theoretical understanding.
Category: Technical alignment
Type: Curriculum

AI Safety Atlas (opens in new tab)
By Centre pour la Sécurité de l'IA (CeSIA)
Textbook systematically mapping the landscape of AI safety knowledge. Ideas are laid out in a carefully structured narrative where concepts build naturally on previous ones.
Category: Technical alignment, Governance
Type: Curriculum
In-depth introduction to foundational AI alignment for people with strong backgrounds in mathematics, physics, or theoretical computer science.
Category: Technical alignment
Type: Curriculum
Intro to Transformative AI (opens in new tab)
By BlueDot Impact
Intensive 5-unit course helping people fast-track their understanding of AI’s impact on society and start taking their first steps in making a contribution to AI safety.
Category: General intro
Type: Curriculum
AI Futurism Reading List (opens in new tab)
By Redwood Research
Guide to readings which seem valuable for thinking about key future dynamics in advanced AI development – timelines, takeoff, misalignment, control, governance, and strategy.
Category: Strategy
Type: Reading list

CS 2881: AI Safety (opens in new tab)
By Boaz Barak
Graduate-level course at Harvard University examining technical challenges in AI safety, including adversarial robustness, jailbreaks, interpretability, and potential catastrophic capabilities.
Category: Technical alignment
Type: Curriculum
Deep Dive (opens in new tab)
By European Network for AI Safety (ENAIS) and AI Safety Hungary
Designed as a 201 AI policy course, helping you find your footing in the vast AI governance landscape by engaging with various dimensions.
Category: Governance
Type: Curriculum

Language Models and Intelligent Agentic Systems (opens in new tab)
By Meridian and C2D3
Series of 16 lectures explaining how language model systems are built, in order to understand and predict their behavior.
Category: Technical alignment
Type: Curriculum
Introduction to AI Safety (opens in new tab)
By Center for AI Safety (CAIS)
Video course on malicious use, competitive pressures, loss of control, and governance. Abridged and updated version of the AI Safety, Ethics, and Society (AISES) textbook.
Category: General intro
Type: Curriculum
Future of AI (opens in new tab)
By BlueDot Impact
Very short, engaging introduction to AI and its potential impacts. Good entry point to share with laypeople curious about AI.
Category: General intro
Type: Curriculum
AGI Strategy (opens in new tab)
By BlueDot Impact
The decisions being made today determine whether AI liberates humanity, destabilizes it, or worse. This course aims to prepare participants to be part of those decisions.
Category: Strategy
Type: Curriculum
Introduction to ML Safety (opens in new tab)
By Center for AI Safety (CAIS)
PhD-level survey course on technical ML safety. Covers various technical topics to reduce existential risks from AI, including robustness, monitoring, control, and systemic safety.
Category: Technical alignment
Type: Curriculum

Tracks @ XLab (opens in new tab)
By UChicago Existential Risk Laboratory (XLab)
A technical track on AI control and a governance track on verifying international AI agreements – with primary literature, interactive demos, and writing practice.
Category: Technical alignment, Governance
Type: Curriculum

Introduction to AI Safety (opens in new tab)
By Stanford University
What is safe AI, and how do we make it? This Stanford University course explores that question, focusing on the technical challenges of creating reliable, ethical, and aligned AI systems.
Category: General intro, Technical alignment
Type: Curriculum
Facilitates upskilling in machine learning engineering for the purpose of contributing directly to AI alignment in technical roles.
Category: Technical alignment
Type: Curriculum
Introduction to Cooperative AI (opens in new tab)
By Cooperative AI Foundation (CAIF)
A primer on cooperative AI, a research field focused on improving the cooperative intelligence of advanced AI for the benefit of all.
Category: General intro
Type: Curriculum

MATS AI Safety Strategy Curriculum (opens in new tab)
By ML Alignment & Theory Scholars (MATS)
As part of their program, MATS runs a series of discussion groups focused on topics relevant to prioritizing research in AI safety. This is the curriculum.
Category: Technical alignment, Governance
Type: Curriculum

Victoria Krakovna: AI Alignment Resources (opens in new tab)
By Victoria Krakovna
Intended to help people get up to speed on the main ideas in AI alignment i.e. ensuring advanced AI systems do what we want them to do.
Category: Technical alignment
Type: Reading list

CHAI: Annotated Bibliography of Recommended Materials (opens in new tab)
By Center for Human-Compatible AI (CHAI)
Reading list discussing ideas CHAI believes warrant further attention and research for the purpose of developing human-compatible AI.
Category: Technical alignment
Type: Reading list

Learning-Theoretic Agenda Reading List (opens in new tab)
By Vanessa Kosoy
Vanessa Kosoy's math-heavy reading list for people interested in studying the learning-theoretic agenda.
Category: Technical alignment
Type: Reading list

UChicago XLab AI Security Guide (opens in new tab)
By UChicago Existential Risk Laboratory (XLab)
Introduces AI security research through five sections covering adversarial examples, jailbreaking, model tampering, data poisoning, and information extraction.
Category: Technical alignment
Type: Curriculum

Reading What We Can (opens in new tab)
By Esben Kran
Collection of books and articles for a 20 day reading challenge. Covers AI safety basics, ML engineering, ML upskilling, and AI safety-relevant sci-fi.
Category: General intro
Type: Reading list

Existential Risks Introductory Course (ERIC) (opens in new tab)
By Cambridge Long View Initiative (CLVI)
Each section covers a different topic related to existential risks. The book 'The Precipice' by Toby Ord forms the majority of the core reading material.
Category: General intro
Type: Curriculum
Economics of Transformative AI (opens in new tab)
By BlueDot Impact
Course designed for economists who want to develop their understanding of transformative AI and its economic impacts.
Category: Strategy
Type: Curriculum
Key Phenomena in AI Risk (opens in new tab)
By Principles of Intelligence
Focused on introducing people to AI risk factors from misdirected optimization or consequentialist cognition, aiming to remain largely agnostic of solution paradigms.
Category: Technical alignment
Type: Curriculum
Covers key concepts and research perspectives in AI safety, split into two main streams: Technical AI Safety and AI Governance.
Category: Technical alignment, Governance
Type: Curriculum

