Self-study

AI Safety, Ethics and Society

Covers a wide range of risks, including loss of control, misalignment, malicious use, accidents, and enfeeblement, while leveraging concepts and frameworks from existing research fields to analyze AI safety.

Category

Introductory

Created by

Center for AI Safety (CAIS)

AI Safety Atlas

Textbook used by various training programs systematically mapping the landscape of AI safety knowledge. Ideas are laid out in a carefully structured narrative where concepts build naturally on previous ones.

Category

Technical Alignment, Governance

Created by

Centre pour la Sécurité de l'IA (CeSIA)

Intro to Transformative AI

Intensive 5-unit course helping people fast-track their understanding of AI’s impact on society and start taking their first steps in making a contribution to AI safety.

Category

Introductory

Created by

BlueDot Impact

Agent Foundations for Superintelligence-Robust Alignment

Guide to the cluster of thought which expects solving alignment in a way which scales to superintelligence to require well-specified proposals guided by strong theoretical understanding.

Category

Technical Alignment

Created by

Alignment Ecosystem Development (AED)

CS 2881: AI Safety

Graduate-level course at Harvard University examining technical challenges in AI safety, including adversarial robustness, jailbreaks, interpretability, and potential catastrophic capabilities.

Category

Technical Alignment

Created by

Boaz Barak

Deep Dive

Designed as a 201 AI policy course, helping people form the in-depth, strategic perspective needed to navigate the complexities of domestic and international AI governance.

Category

Governance

Created by

European Network for AI Safety (ENAIS) and AI Safety Hungary

AI Alignment Fast-Track

Intensive 5-unit course from BlueDot Impact designed to accelerate your understanding of key concepts in AI alignment.

Category

Introductory, Technical Alignment

Created by

BlueDot Impact

AI Governance Fast-Track

Intensive 5-unit course from BlueDot Impact designed to accelerate your understanding of key concepts in AI governance and policy.

Category

Introductory, Governance

Created by

BlueDot Impact

Language Models and Intelligent Agentic Systems

Series of 16 lectures explaining how language model systems are built, in order to understand and predict their behaviour.

Category

Technical Alignment

Created by

Meridian and C2D3

Future of AI

Very short, engaging introduction to AI and its potential impacts. Good entry point to share with laypeople curious about AI.

Category

Introductory

Created by

BlueDot Impact

AGI Strategy

The decisions being made today determine whether this technology liberates humanity, destabilises it, or worse. This course aims to prepare participants to be part of those decisions.

Category

Strategy

Created by

BlueDot Impact

Introduction to ML Safety

PhD-level survey course on technical ML safety. Covers various technical topics to reduce existential risks from AI, including robustness, monitoring, control, and systemic safety.

Category

Technical Alignment

Created by

Center for AI Safety (CAIS)

Introduction to AI Safety

What is safe AI, and how do we make it? This Stanford University course explores that question, focusing on the technical challenges of creating reliable, ethical, and aligned AI systems.

Category

Introductory, Technical Alignment

Created by

Stanford University

Alignment Research Engineer Accelerator (ARENA)

Facilitates upskilling in machine learning engineering for the purpose of contributing directly to AI alignment in technical roles.

Category

Technical Alignment

Created by

ARENA

Introduction to Cooperative AI

A primer on cooperative AI, an emerging research field focused on improving the cooperative intelligence of advanced AI for the benefit of all.

Category

Introductory

Created by

Cooperative AI Foundation (CAIF)

MATS AI Safety Strategy Curriculum

As part of their program, MATS runs a series of discussion groups focused on topics relevant to prioritizing research into AI safety. This is the curriculum from the latest cohort.

Category

Technical Alignment, Governance

Created by

ML Alignment & Theory Scholars (MATS)

CHAI: Annotated Bibliography of Recommended Materials

Reading list discussing ideas CHAI believes warrant further attention and research for the purpose of developing human-compatible AI.

Category

Technical Alignment

Created by

Center for Human-Compatible AI (CHAI)

Levelling Up in AI Safety Research Engineering

Level-based guide for independently upskilling in AI safety research engineering that aims to give concrete objectives, goals, and resources to help anyone go from zero to hero.

Category

Technical Alignment

Created by

Stanford AI Alignment (SAIA)

Victoria Krakovna: AI Alignment Resources

Intended to help people get up to speed on the main ideas in AI alignment i.e. ensuring advanced AI systems do what we want them to do.

Category

Technical Alignment

Created by

Victoria Krakovna

Learning-Theoretic Agenda Reading List

Vanessa Kosoy's math-heavy reading list for people interested in studying the learning-theoretic agenda.

Category

Technical Alignment

Created by

Association for Long-Term Existence and Resilience (ALTER)

Arkose: AI Safety Papers

Papers, blog posts, talks, newsletters, and guides relevant to improving the safety of advanced AI systems in order to reduce large-scale risks.

Category

Technical Alignment, Governance

Created by

Arkose

Reading What We Can

Collection of books and articles for a 20 day reading challenge. Covers AI safety basics, ML engineering, ML upskilling, and AI safety-relevant sci-fi.

Category

Introductory

Created by

Apart Research

Existential Risks Introductory Course (ERIC)

Each section covers a different topic related to existential risks. The book 'The Precipice' by Toby Ord forms the majority of the core reading material.

Category

Introductory

Created by

Cambridge Long View Initiative (CLVI)

Economics of Transformative AI

Course designed for economists who want to develop their understanding of transformative AI and its economic impacts.

Category

Strategy

Created by

BlueDot Impact

Key Phenomena in AI Risk

Focused on introducing people to AI risk factors from misdirected optimization or consequentialist cognition, aiming to remain largely agnostic of solution paradigms.

Category

Technical Alignment

Created by

Principles of Intelligent Behavior in Biological and Social Systems (PIBBSS)

ML for Alignment Bootcamp (MLAB)

Teaches ML skills and concepts that are relevant to doing the kinds of alignment research that seem most leveraged for reducing AI existential risk. Access requires registration.

Category

Technical Alignment

Created by

Redwood Research

The Interpretability Toolkit

Collection of tools to get started and skill-up in interpretability, including the "Quickstart Guide for Mechanistic Interpretability" by Neel Nanda. Created in 2023.

Category

Technical Alignment

Created by

Apart Research