Courses

AI Safety, Ethics and Society

Covers a wide range of risks, including loss of control, misalignment, malicious use, accidents, and enfeeblement, while leveraging concepts and frameworks from existing research fields to analyze AI safety.

Length

8 weeks

Category

Introductory

Organizer

Center for AI Safety (CAIS)

AI Safety Atlas

Textbook used by various training programs systematically mapping the landscape of AI safety knowledge. Ideas are laid out in a carefully structured narrative where concepts build naturally on previous ones.

Length

9 weeks

Category

Technical Alignment, Governance

Organizer

Centre pour la Sécurité de l'IA (CeSIA)

Intro to Transformative AI

Intensive 5-day course from BlueDot Impact helping people fast-track their understanding of AI’s impact on society and start taking their first steps in making a contribution to AI safety.

Length

5 days

Category

Introductory

Organizer

BlueDot Impact

Agent Foundations for Superintelligence-Robust Alignment

Guide to the cluster of thought which expects solving alignment in a way which scales to superintelligence to require well-specified proposals guided by strong theoretical understanding.

Length

6 weeks

Category

Technical Alignment

Organizer

Alignment Ecosystem Development (AED)

AI Alignment Fast-Track

Intensive 5-day course from BlueDot Impact designed to accelerate your understanding of key concepts in AI alignment.

Length

5 days

Category

Introductory, Technical Alignment

Organizer

BlueDot Impact

AI Governance Fast-Track

Intensive 5-day course from BlueDot Impact designed to accelerate your understanding of key concepts in AI governance and policy.

Length

5 days

Category

Introductory, Governance

Organizer

BlueDot Impact

Language Models and Intelligent Agentic Systems

Series of 16 lectures explaining how language-model systems are built, in order to understand and predict their behaviour.

Length

16 hours

Category

Technical Alignment

Organizer

Meridian and C2D3

Future of AI

Very short, engaging introduction to AI and its potential impacts. Good entry point to share with laypeople curious about AI.

Length

2 hours

Category

Introductory

Organizer

BlueDot Impact

Introduction to ML Safety

PhD-level survey course on technical ML safety. Covers various technical topics to reduce existential risks from AI, including robustness, monitoring, control, and systemic safety.

Length

40 hours

Category

Introductory, Technical Alignment

Organizer

Center for AI Safety (CAIS)

Introduction to AI Safety

What is safe AI, and how do we make it? This Stanford University course explores that question, focusing on the technical challenges of creating reliable, ethical, and aligned AI systems.

Length

10 weeks

Category

Introductory, Technical Alignment

Organizer

Stanford University

Alignment Research Engineer Accelerator (ARENA)

Facilitates upskilling in machine learning engineering for the purpose of contributing directly to AI alignment in technical roles.

Length

4 weeks

Category

Technical Alignment

Organizer

ARENA

Economics of Transformative AI

Course from BlueDot Impact designed for economists who want to develop their understanding of transformative AI and its economic impacts.

Length

5 weeks

Category

Strategy

Organizer

BlueDot Impact

Introduction to Cooperative AI

A primer on cooperative AI, an emerging research field focused on improving the cooperative intelligence of advanced AI for the benefit of all.

Length

8 weeks

Category

Introductory

Organizer

Cooperative AI Foundation (CAIF)

MATS AI Safety Strategy Curriculum

As part of their program, MATS runs a series of discussion groups focused on topics they believe are relevant to prioritizing research into AI safety. This is the curriculum from the latest cohort.

Length

5 weeks

Category

Technical Alignment, Governance

Organizer

ML Alignment & Theory Scholars (MATS)

CHAI: Annotated Bibliography of Recommended Materials

Reading list discussing ideas CHAI believes warrant further attention and research for the purpose of developing human-compatible AI, as well as prerequisite materials for approaching them.

Length

40 weeks

Category

Technical Alignment

Organizer

Center for Human-Compatible AI (CHAI)

Levelling Up in AI Safety Research Engineering

Level-based guide for independently upskilling in AI safety research engineering that aims to give concrete objectives, goals, and resources to help anyone go from zero to hero.

Length

6 months

Category

Technical Alignment

Organizer

Stanford AI Alignment (SAIA)

Victoria Krakovna: AI Alignment Resources

Intended to help people get up to speed on the main ideas in AI alignment i.e. ensuring advanced AI systems do what we want them to do. This is a selected list that is not designed to be comprehensive.

Length

–

Category

Technical Alignment

Organizer

Victoria Krakovna

Learning-Theoretic Agenda Reading List

Vanessa Kosoy's math-heavy reading list for people interested in studying the learning-theoretic agenda.

Length

–

Category

Technical Alignment

Organizer

Association for Long-Term Existence and Resilience (ALTER)

Arkose: AI Safety Papers

Papers, blog posts, talks, newsletters, and guides relevant to improving the safety of advanced AI systems in order to reduce large-scale risks.

Length

–

Category

Technical Alignment, Governance

Organizer

Arkose

Reading What We Can

Collection of books and articles for a 20 day reading challenge. Covers AI safety basics, ML engineering, ML upskilling, and AI safety-relevant sci-fi.

Length

20 days

Category

Introductory

Organizer

Apart Research

Existential Risks Introductory Course (ERIC)

Each section covers a different topic related to existential risks. Much of the program follows the general structure of The Precipice by Toby Ord, which forms the majority of the core reading material.

Length

8 weeks

Category

Introductory

Organizer

Cambridge Long View Initiative (CLVI)

Key Phenomena in AI Risk

Curriculum focused on introducing people to AI risk factors from misdirected optimization or consequentialist cognition, aiming to remain largely agnostic of solution paradigms.

Length

6 weeks

Category

Technical Alignment

Organizer

Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS)

ML for Alignment Bootcamp (MLAB)

Teaches machine learning skills and concepts that are relevant to doing the kinds of alignment research that seem most leveraged for reducing AI existential risk. Access requires registration.

Length

3 weeks

Category

Technical Alignment

Organizer

Redwood Research

The Interpretability Toolkit

Collection of tools to get started and skill-up in interpretability, including the "Quickstart Guide for Mechanistic Interpretability" by Neel Nanda. Created in 2023.

Length

–

Category

Technical Alignment

Organizer

Apart Research