Events & training
Field map
Communities
Self-study
Jobs
Funding
03 November 2025
Category
You can buy physical copies of the map – including shirts, mugs, stickers etc. AISafety.com makes no profit on purchases.
Suggest something to be listed on the map
Let us know of changes to a listing
Volunteer organization dedicated to raising awareness about modern AI, highlighting its benefits and risks, and letting the public know how they can help – mainly through workshops.
Advocacy
Nonprofit aiming to catalyze collective effort towards reducing existential risk, including through an extensive action list for individuals, organizations, and nations.
Informing the public debate on existential risks, on the basis that awareness is the first step to reducing those risks.
Calling on policymakers to implement a global moratorium on large AI training runs until alignment is solved.
Researches ways to use legal advocacy to make science and technology safer, informs legal professionals about how to help, and advocates in the courts and policy-setting institutions.
Uses legal advocacy in Europe to address risks from frontier technologies, working to foster responsible development and implementation practices.
Campaign group aiming to convince governments to pause AI development – through public outreach, engaging with decision-makers and organizing protests.
Non-violent civil resistance organization working to permanently ban the development of smarter-than-human AI to prevent human extinction, mass job loss, and other problems.
Communications nonprofit focused on changing the narrative air that policymakers in DC breathe with respect to AI safety.
Watchdog nonprofit monitoring tech companies, countering corporate propaganda, raising awareness about corner-cutting, and advocating for the responsible development of AI.
Steering transformative technology towards benefitting life and away from extreme large-scale risks through outreach, policy advocacy, grantmaking, and event organisation.
Advocacy, Governance, Funding
A collaborative effort to bring together Buddhist communities, technologists, and contemplative researchers worldwide to help shape the future of artificial intelligence.
Advocacy, Governance, Strategy
Platform from the Center for AI Safety (CAIS) posting articles written by experts from a wide range of fields discussing the impacts of AI.
Blog
Publishes policy-relevant perspectives on frontier AI governance, including research summaries, opinions, interviews, and explainers.
Blog by Eric Drexler on AI prospects and their surprising implications for technology, economics, environmental concerns, and military affairs.
Blog by Daniel Paleka, an AI safety researcher discussing AI safety news, with posts about every two months.
Blog covering many topics, including reasoning, science, psychiatry, medicine, ethics, genetics, AI, economics, and politics. Includes book summaries and commentary on AI safety.
Blog on AI safety by Jacob Steinhardt, a UC Berkeley statistics professor, analysing risks, forecasting future breakthroughs, and discussing alignment strategies.
Deep coverage of technology, China, and US policy, featuring original analysis alongside interviews with thinkers and policymakers. Written by Jordan Schneider.
Blog about transformative AI, futurism, research, ethics, philanthropy etc. by Holden Karnofsky. Includes the "Most Important Century" post series. Last posted January 2024.
Blog by Zvi Mowshowitz on various topics, including AI, generally with detailed analysis, personal insights, and a rationalist perspective.
Blog from ex-OpenAI (now independent) AI policy researcher Miles Brundage on the rapid evolution of AI and the urgent need for thoughtful governance.
Publication by Garrison Lovely on the intersection of capitalism, geopolitics, and AI. Posts about once or twice a month.
Blog on aligning prosaic AI by one of the leading AI safety researchers. Last posted in 2023.
Blog about building the fields of AI safety and Effective Altruism, discussing big-picture strategy and sharing personal experience from working in the space.
Top forecaster Peter Wildeford forecasts the future and discusses AI, national security, innovation, emerging technology, and the powers – real and metaphorical – that shape the world.
Approximately fortnightly blog by Anton Leicht about charting a course through the politics of rapid AI progress.
Blog by an AI safety researcher at Google DeepMind, covering alignment research, rationality, and personal productivity insights. Last AI post was 2023.
The AI safety space is changing rapidly. This directory of key information sources can help you keep up to date with the latest developments.
Blog, Newsletter, Podcast, Video
Chinese capabilities lab developing and releasing open-weights large language models. Created DeepSeek-R1.
Capabilities research
Capabilities lab led by Elon Musk with the mission of advancing our collective understanding of the universe. Created Grok.
Team of researchers housed at Astera aiming to engineer AGI by pursuing an exploratory approach heavily inspired by cognitive science and neuroscience.
Capabilities research, Conceptual research
Research lab focusing on LLM alignment, particularly interpretability. Featuring Chris Olah, Jack Clark, and Dario Amodei. Created Claude.
Capabilities research, Empirical research
A strategy for accelerating alignment research by using human-in-the-loop systems which empower human agency rather than outsource it.
London-based AI capabilities lab with a strong safety team, led by Demis Hassabis. Created AlphaGo, AlphaFold, and Gemini.
For-profit company developing tools that automatically assess the risks of AI models and developing its own AI models aiming to provide best-in-class safety and security.
San Francisco-based capabilities lab and creator of ChatGPT, led by Sam Altman. Throughout 2024, roughly half of then-employed AI safety researchers left the company.
Research lab founded by Ilya Sutskever comprised of a small team of engineers and researchers working towards building a safe superintelligence.
Regularly-updated article with motivation and advice around pursuing a career in AI safety.
Career support
Curated list of job posts around the world tackling pressing problems, including AI safety. Also has a newsletter.
Helps people navigate the AI safety space with a welcoming human touch, offering personalized guidance and fostering collaborative study and project groups.
Directory of advisors offering free guidance calls to help you discover how best to contribute to AI safety, tailored to your skills and interests.
Empowering students to use their theses as a pathway to impact. Lists research topic ideas and runs an accelerator and fellowship coaching people working on them.
Supporting working professionals to maximize their positive impact through their talent directory and Impact Accelerator Program.
A guide written for people who are familiar with the arguments for the importance of AI alignment and are considering pursuing a career working on it.
Helps those who want to have a meaningful impact with their careers brainstorm career paths, evaluate options, and plan next steps.
Helping professionals transition to high-impact work by performing market research on impactful jobs and providing career mentoring, opportunity matching, and professional training.
Applied research lab helping existential safety advocates to systematically optimize their lives and work.
Runs various projects aimed at education, skill development, and creating pathways into impactful careers.
Career support, Training and education
Research organization trying to understand how to formalize mechanistic explanations of neural network behavior.
Conceptual research
Studying questions about multi-agent systems composed of humans and advanced AI. Based at Charles University, Prague.
Wiki on AI alignment theory, mostly written by Eliezer Yudkowsky. Includes foundational concepts, open problems, and proposed solutions.
Developing the conceptual and technical wherewithal to reorient the general thrust of AI research towards provably beneficial systems. Led by Stuart Russell at UC Berkeley.
Research group studying agent foundations to give us the mathematical tools to align the objectives of an AI with human values.
Group working on foundational mathematics research that gives us an understanding of the nature of AI agents.
Professor at University of Louisville with a background in cybersecurity, and author of over 100 publications – including two books on AI safety.
Assistant professor at MIT working on agent alignment. Runs the Algorithmic Alignment Group.
Independent alignment researcher working on selection theorems, abstraction, and agency.
Conducting long-term future research on improving cooperation in competition for the development of transformative AI.
Formal alignment organization led by Tamsin Leake, focused on agent foundations. Also has a public Discord server.
Brain-inspired framework using insights from neuroscience and model-based reinforcement learning to guide the design of aligned AGI systems.
A small team of independent researchers trying to find reward functions which reliably instill certain values in agents.
Israeli research and advocacy nonprofit working to investigate, demonstrate, and foster useful ways to safeguard and improve the future of humanity.
Conceptual research, Advocacy, Governance
Research organization dedicated to developing a theory of "organic alignment" to foster adaptive, non-hierarchical cooperation between humans and digital agents.
Conceptual research, Empirical research
Large team taking a 'Neglected Approaches' approach to alignment, tackling the problem from multiple, often overlooked angles in both technical and policy domains.
Conceptual research, Empirical research, Governance
Nonprofit aiming to reduce lock-in risks by researching fundamental lock-in dynamics and power concentration.
Argentine nonprofit conducting both theoretical and empirical research to advance frontier AI safety as a sociotechnical challenge.
Working towards better conceptual understanding, algorithmic techniques, and policies to make AI safer and more socially beneficial.
Hub for researchers to discuss all ideas related to AI safety. Discussion ranges from technical models of agency to the strategic landscape, and everything in between.
Conceptual research, Empirical research, Governance, Strategy
Using category theory and agent-based simulations to predict where multi-agent AI systems break down and identify control levers to prevent collective AI behavior from being harmful.
Conceptual research, Governance
Research, grants and community-building around AI safety, focused on conflict scenarios as well as technical and philosophical aspects of cooperation.
Conceptual research, Strategy, Funding
Applied mathematics research nonprofit dedicated to advancing foundational alignment research.
Conceptual research, Training and education
Independent LLM agent safety research group launched in 2024, focused on chain-of-thought monitorability.
Empirical research
Oxford-based startup attempting to use mathematical and theoretical techniques to achieve safe off-distribution generalization.
Aiming to detect deception by designing AI model evaluations and conducting interpretability research to better understand frontier models. Also provides guidance to policymakers.
R&D nonprofit prototyping AI-powered tools to generate formal specifications. Aiming to empower engineers to verify code easily and develop software that’s built for trust.
AI safety (and pandemic prevention) research community based in a small town in Vermont, USA.
Alignment startup born out of EleutherAI, employing a “Cognitive Emulation” approach to build controllable large language models and tackle core AI safety challenges.
Developing tools that accelerate the rate human researchers can make progress on alignment, and building automated research systems that can assist in alignment work today.
AI safety engineering company that provides technical talent and infrastructure development for AI evaluations.
Ensuring advanced AI is safe and beneficial for everyone by carrying out research, hosting events, and running programs – including the FAR.Lab coworking space.
New research nonprofit in Cambridge focused on leading projects with the shortest path to impact for AI safety. Initial work is on chain-of-thought health and monitoring.
Mechanistic interpretability research lab aiming to decode neural networks in order to make AI systems more understandable, editable, and safer.
AI safety research group at the University of Cambridge, led by David Krueger. Part of the Computational and Biological Learning Lab.
Canadian nonprofit advancing research and developing technical solutions for safe-by-design AI systems based on Scientist AI, a research direction led by Yoshua Bengio.
Developing AI Control in order to increase the probability that effective AI Control systems will be deployed to mitigate catastrophic risks from frontier AI systems when they are developed.
Research organization applying their expertise in meaning and human values to AI alignment and post-AGI futures.
Researches, develops, and runs cutting-edge tests of AI capabilities, including broad autonomous capabilities and the ability of AI systems to conduct AI R&D.
Group of researchers at New York University doing empirical work with language models aiming to address longer-term concerns about the impacts of deploying highly-capable AI systems.
Product-driven research lab developing mechanisms for delegating high-quality reasoning to ML systems. Built Elicit, an AI assistant for researchers and academics.
Nonprofit building concrete demonstrations of dangerous capabilities to advise policy-makers and the public on AI risks.
Nonprofit researching interpretability and alignment. Also consults governments and AI labs on AI safety practices.
Small team of researchers and engineers aiming to bring the best of physics and computational neuroscience together in order to understand and control AGI.
Nonprofit research lab building open source, scalable, AI-driven tools to understand and analyze AI systems and steer them in the public interest.
Nonprofit led by Owain Evans, researching situational awareness, deception, and hidden reasoning in language models.
Research group using engineering approaches to understand the brain and to develop artificial learning systems.
Nonprofit research lab building AI tools to defend and enhance human agency – by researching and experimenting with novel AI capabilities.
Empirical research, Capabilities research
Open-source research lab focused on interpretability and alignment. Operates primarily through a public Discord server, where research is discussed and projects are coordinated.
New public benefit corporation working on mitigating gradual disempowerment and the intelligence curse by creating personalized models.
Blog from the Google DeepMind safety team discussing research ideas about building AI safely and responsibly.
Empirical research, Conceptual research
Research nonprofit using singular learning theory to develop the science of understanding how training data determines model behavior.
San Francisco-based nonprofit conducting safety research, building the field of AI safety researchers, and advocating for safety standards.
Empirical research, Conceptual research, Advocacy
UC Berkeley research center bridging academic research and practical policy needs in order to anticipate and address emerging cybersecurity challenges.
Empirical research, Governance, Strategy
Small research group forecasting the future of AI. Created 'AI 2027', a detailed forecast scenario projecting the development of artificial superintelligence.
Forecasting
Advancing the science of forecasting for the public good by working with policymakers and nonprofits to design practical forecasting tools, and test them in large experiments.
Interdisciplinary group aiming to identify and understand trends in computing that create opportunities for (or pose risks to) our ability to sustain economic growth.
Prediction market platform on many topics using play money (called "mana"). Includes markets on AI and AI safety.
Well-calibrated forecasting platform covering a wide range of topics, including AI and AI safety.
Advancing forecasting and epistemics to improve the long-term future of humanity. Writing research and software.
Exploring the use of underutilized foresight methods and tools in order to better anticipate societal-scale risks from AI.
Research institute investigating key trends and questions that will shape the trajectory and governance of AI.
Forecasting, Strategy
13-week program run by Fifty Years helping scientists and engineers build startups tackling world-scale problems.
Empowering innovators and scientists to increase human agency by creating the next generation of responsible AI. Providing support, resources, and open-source software.
Aiming to reduce catastrophic risks from advanced AI through grants towards technical research, policy, and training programs for new researchers.
Grant program run by Nonlinear, providing funding to those raising awareness about AI risks or advocating for a pause in AI development.
Funder housing the science-focused philanthropic efforts of Eric and Wendy Schmidt, which moves large amounts of funding towards AI safety via its AI institute.
Philanthropic initiative supporting researchers working on key opportunities and hard problems that are critical to get right for society to benefit from AI.
Regularly-updated guide on how to donate most effectively with the funding and time you have available.
Comprehensive and up-to-date directory of sources of financial support for AI safety projects, ranging from grant programs to venture capitalists.
UK government R&D funding agency built to unlock scientific and technological breakthroughs that benefit everyone.
An analysis of the main funding sources in AI safety over time, useful for gaining a better understanding of what opportunities exist in the space.
New AI safety funding initiative with $25M annual giving, aiming to unite funders, domain experts, and entrepreneurs to launch and scale ambitious initiatives.
Supports projects and individuals aiming to address worst-case suffering risks from the development and deployment of advanced AI systems.
Charity foundation backed by a large philanthropic commitment supporting research into improving cooperative intelligence of advanced AI.
Aiming to increase the impact of effective altruism projects by increasing their access to talent, capital, and knowledge.
Helping major donors find, fund, and scale the most promising solutions to the world’s most pressing problems.
Funding projects in 1) automating research and forecasting, 2) security technologies, 3) neurotech, and 4) safe multipolar human AI scenarios.
Accelerator aiming to steer transformative technology towards benefiting life and away from extreme large-scale risks.
Fellowships include PhD and postdoctoral fellowships in technical AI safety, and a PhD fellowship in US-China AI governance.
Crowdsourced charity evaluation and philanthropic networking platform that lets funders, donors, and scouts discover and support high-impact projects.
Community of donors who have pledged to donate a significant portion of their income to highly effective charities, including those in AI safety.
VC firm investing in ethical founders developing transformative technologies that have the potential to impact humanity on a meaningful scale.
Making grants addressing global catastrophic risks, promoting longtermism, and otherwise increasing the likelihood that future generations will flourish.
Devises and executes bespoke giving strategies for major donors, working with them at every stage of their giving journey.
Swiss nonprofit focused on reducing suffering risks, including that posed by catastrophic AI misuse and conflict. Previously known as Polaris Ventures.
Marketplace for new charities, including in AI safety. Find impactful projects, buy impact certificates, and weigh in on what gets funded.
Network of donors funding charitable projects that work one level removed from direct impact, often cross-cutting between cause areas.
Aiming to empower founders building a radically better world with safe AI systems by investing in ambitious teams with defensible strategies that can scale to post-AGI.
Funder network for AI existential risk reduction. Applications are shared with donors, who then reach out if they are interested.
The largest funder in the existential risk space, backed by Cari Tuna and Dustin Moskovitz, co-founder of Facebook and Asana.
Small organization with a long history of finding the people doing the best work to prevent human-created existential risks and financially supporting them.
The second largest funder in AI safety, using an algorithm and meeting procedure called “The S-process” to allocate grants.
Offers grants to high-impact organizations and projects that are taking bold action and making significant changes.
Identifies leaders from business, policy, and academia, and helps them take on new ambitious projects in AI safety.
Funding, Career support
VC firm investing in startups helping make AI safe, secure, and beneficial for humanity. Focused on working with exceptional founders at the earliest stages.
Nonpartisan nonprofit and community of people across Canada, working to ensure that advanced AI is safe and beneficial for all.
Governance
Channeling public concern into effective regulation by engaging with policymakers, media, and the public to ensure AI is developed responsibly and transparently.
Politically and geographically neutral nonprofit converting insights from the existing literature into ready-made text for AI safety standards.
Bipartisan nonprofit seeking to address a broad range of policy issues raised by AI, including current harms, national security concerns, and emerging risks.
R&D institution dedicated to developing AI safety and governance frameworks to provide a safe foundation for AI innovation and applications.
US government organization developing voluntary AI standards and conducting security evaluations of AI systems. Formerly the USAISI.
Georgetown University think tank providing decision-makers with data-driven analysis on the security implications of emerging technologies.
Brussels think tank focused on helping governments anticipate and responsibly govern the societal impacts of rapid technological change.
AI governance research group at Oxford, producing research tailored towards decision-makers and running career development programmes.
China’s self-described counterpart to the AI safety institutes of other countries. It's primary function is to represent China in international AI conversations.
Established within the European Commission as the centre of AI expertise, playing a key role in implementing the AI Act.
Australian lobbying organization focused on AI safety policy and other AI-related issues, including cybersecurity and biosecurity. Also runs Australians for AI Safety.
Think tank researching and advising on the legal challenges posed by AI, premised on the idea that sound legal analysis will promote security, welfare, and the rule of law.
Delivers rigorous and objective public policy research on the most consequential challenges to civilization and global security.
French nonprofit working to incentivize responsible AI practices through policy recommendations, research, and risk assessment tools.
Nonprofit developing and advocating for AI safety principles to be put into practice in US state and federal legislatures.
Geneva-based think tank working to foster international cooperation on governing frontier AI.
Building bipartisan support for federal policies that prepare the US for the emergence of AI systems on the path to AGI and beyond.
Helps AI insiders raise concerns about potential risks and misbehavior in AI development by providing whistleblowing services, expert guidance, and secure communication tools.
Nonprofit based in the US and Europe aiming to define, design, and deploy projects that address institutional barriers in AI governance.
UK government organisation conducting research and building infrastructure to test the safety of advanced AI and measure its impacts. Also working to shape global policy.
Promoting informed policymaking to navigate emerging challenges from AI through research, knowledge-sharing, and skill building.
Nonprofit fighting to keep humanity in control of AI by developing policy and conducting public outreach.
Governance, Advocacy
Nonprofit dedicated to establishing an independent global organization capable of effectively mitigating extinction risks from AI and fairly distributing its economic benefits to all.
The original AI safety technical research organization, co-founded by Eliezer Yudkowsky. Now focusing on policy and public outreach.
Governance, Advocacy, Conceptual research
Advisory and research organization focused on improving the way institutions make decisions on critical global challenges.
Governance, Strategy
Research and field-building organization focusing on policy and standards, compute governance, and international governance and China.
Nonprofit aiming to ensure that AI systems are guaranteed to operate safely and ethically; and to shape policy, promote research, and build understanding and community around this goal.
French AI safety nonprofit dedicated to education (including university courses and ML4Good bootcamps), advocacy (including events and publications), and research.
Governance, Strategy, Advocacy
Weekly newsletter listing newly-announced AI safety events and training programs, both online and in-person.
Newsletter
Newsletter published every few weeks discussing developments in AI and AI safety. No technical background required.
Newsletter from Concordia AI, a Beijing-based social enterprise, providing updates on AI safety developments in China.
Current-events newsletter for keeping up to date with FMxAI (formal methods and AI) with a gear toward safety, doing a mix of shallow technical reviews of papers and movement updates.
Weekly developments in AI research (including governance) written by Jack Clark, co-founder of Anthropic.
Monthly-ish newsletter aiming to inform readers about the latest research developments in ML safety, focussing on topics such as adversarial robustness, interpretability, and control.
Aims to help decision-makers understand what’s happening in AI and why it matters, through news roundups, explainers, features, and opinion pieces.
Ran research paper/essay-writing contests to advance alignment. Last round was 2023.
No longer active
Weekly newsletter from the Center for AI Policy (CAIP). Each issue explored three key AI policy developments for professionals in the field. Stopped due to lack of funding.
Connected journalists to AI safety experts and resources.
Supported individuals pursuing a career in AI safety by running AI Safety Hub Labs, where participants complete their first research project. Now run by LASR Labs.
Crowdsourced repository of possible research projects and testable hypotheses, run by Apart Research.
Listed opportunities in AI alignment. Dropped due to lack of a maintainer.
Visual overview of the major events in AI over the last decade, from cultural trends to technical advancements. No longer maintained.
AI safety field-building nonprofit. Ran support programs facilitating technical research, did outreach, and curated educational resources. Closed due to lack of funding.
Increased public understanding of AI safety and called for strong laws to stop the development of dangerous and overly powerful AI. Now merged with the Existential Risk Observatory.
Nonpartisan research organization developing policy and conducting advocacy to mitigate catastrophic risks from AI. Shut down due to lack of funding.
Minimalistic site focused on understanding donations, donors, donees, and the thoughts and discussions that go alongside donation decisions.
Blog on AI safety work by a PhD mathematician and AI safety researcher, Robert Huben. Currently semi-dormant.
Longtermist/x-risk research organization led by Nick Bostrom at the University of Oxford. Killed by university politics disallowing them from fundraising.
University of Oxford research center conducting foundational research to inform the decision-making of those seeking to do as much good as possible. Closed in 2025.
Fast funding for projects that have a chance of substantially changing humanity's future trajectory for the better. Last funding round was 2023.
Weekly podcast, YouTube and newsletter with updates on AI safety. Dropped due to organizational reprioritization.
Bootcamp aimed at teaching ML relevant to doing alignment research. Run by Redwood Research for two iterations in 2022.
Database of grants in effective altruism. No longer maintained.
Funding organization aiming to minimize the risk of AI systems. Died for reasons unknown to us.
Website communicating the risks of god-like AI to the public and offering proposals on preventing its development.
Decentralized bounty platform for existential risk reduction and other effective altruist cause areas.
The blog of janus the GPT cyborg. Last post was published in 2023.
In-depth conversations about the world’s most pressing problems (including AI safety) and what you can do to help solve them.
Podcast
Interviews with (mostly technical) AI safety researchers about their research, aiming to get a sense of why it was written and how it might reduce existential risk from AI.
Well-researched interviews with influential intellectuals going in-depth on AI, technology, and their broader societal implications. Hosted by Dwarkesh Patel.
Interviews with existential risk researchers, policy experts, philosophers, and a range of other influential thinkers.
Podcast from the Center for Strategic & International Studies (CSIS) discussing AI regulation, innovation, national security, and geopolitics.
Biweekly podcast where host Nathan Labenz interviews AI innovators and thinkers, diving into the transformative impact AI will likely have in the near future.
Building and maintaining key online resources for the AI safety community, including AISafety.com. Volunteers welcome.
Research support
Providing fiscal sponsorship to AI safety projects, saving them time and allowing them to access more philanthropic funding.
Providing flexible funding and operations support to university research groups working on existential risk, enabling projects otherwise hindered by university administration.
Incubating early-stage AI safety research organizations. The program involves co-founder matching, mentorship, and seed funding, culminating in an in-person building phase.
Free or subsidised accommodation and board in Blackpool, England, for people working on/transitioning to working on global catastrophic risks.
Center for collaborative research in AI safety, supporting promising work through fellowships, workshops, and hosting individuals and teams.
Community of researchers and policymakers from over 13 countries across Europe, united in their efforts to advance AI safety.
Provides engineering teams, compute, and infrastructure to researchers advancing neglected approaches to alignment.
Provides strategy consulting services to clients trying to advance AI safety through policy, politics, coalitions or social movements.
Marketing firm helping socially impactful projects grow, including through consulting and running full-scale campaigns.
Providing consultancy and hands-on support to help high-impact organizations upgrade their operations.
Nonprofit maintaining LessWrong, the Alignment Forum, and Lighthaven (an event space in Berkeley, USA).
Coworking space hosting organizations (including BlueDot Impact, Apollo, Leap Labs), acceleration programs (including MATS, ARENA), and independent researchers.
Incubator and coworking space in San Francisco aimed at various groups, including those working on AI safety.
Nonprofit aiming to prevent extinction and reduce suffering risks by providing funding, 1:1 coaching, and career advice.
Coworking space in Zurich, Switzerland for people working on AI safety or effective altruism. Regularly hosts events – ranging from relaxed gatherings to lightning talks and discussions.
Free accomodation and board in Blackpool, England, for individuals doing work related to pushing for a pause to AI development.
Program designed to help AI safety organizations strengthen their management and leadership practices in order to amplify their effectiveness.
Nonprofit running programs to give altruists the tools they need to improve their mental health, so they can have a greater impact on the world.
Fostering responsible governance of AI to reduce catastrophic risks through shared understanding and collaboration among key global actors.
Coworking, events, and community space for people working on AI safety. Runs regular talks, hackathons, and networking events.
AI safety coworking and events space in downtown Toronto, Canada, providing a physical space, hosting informative events, and maintaining a collaborative community.
Non-profit AI safety research lab hosting open-to-all research sprints, publishing papers, and incubating talented researchers to make AI safe and beneficial for humanity.
Research support, Career support
Coworking space and field-building nonprofit based in Cambridge, UK. Houses projects like CAISH and the ERA:AI Fellowship, as well as visiting AI safety researchers.
Research support, Training and education
The biggest real-time online community of people interested in AI safety, with channels ranging from general topics to specific fields to local groups.
Resource
Concise visual digest of important trends in AI, grounded in concrete examples of what AI models can do right now.
AI safety chatbot that excels at addressing hard questions and counterarguments about existential risk.
Flashcards for helping to learn and memorize the main organizations, projects, and programs currently operating in the AI safety space.
Field-building organization now chiefly serving as the home for an extensive resources list called Lots of Links.
Database tracking people, organisations, and “products” in the AI safety community, serving as a reference for positions, affiliations, and related data.
Hub for key resources for the AI safety community, including directories of courses, jobs, upcoming events and training programs etc.
Directory of domains freely available to be used for high-impact projects, including those contributing to AI safety.
Forum on doing good as effectively as possible, including AI safety. Also has a podcast featuring text-to-speech narrations of top posts.
Online forum dedicated to improving human reasoning, containing a lot of AI safety content. Also has a podcast featuring text-to-speech narrations of top posts.
Comprehensive living database of over 1600 AI risks, categorized by their cause and risk domain.
Collects actions for frontier Al labs to avert extreme risks from AI, then evaluates particular labs accordingly.
Strategy
Ranked and scored contributable compendium of alignment plans and their problems. Runs regular hackathons.
Conducting interdisciplinary research supporting global AI risk management. Also produces policy and technical research.
Interdisciplinary research organization based in China exploring contemporary and long-term impacts of AI on society and ecology.
Interdisciplinary research centre at the University of Cambridge dedicated to the study and mitigation of existential risks.
Small research nonprofit focused on how to navigate the transition to a world with superintelligent AI systems.
Small think tank developing solutions for reducing existential risk by leveraging both scholarship and the demands of real-world decision-making.
International initiative with 44 member countries working to implement human-centric, safe, secure, and trustworthy AI embodied in the principles of the OECD Recommendation on AI.
Interdisciplinary research centre at the University of Cambridge exploring the nature, ethics, and impact of AI.
Research nonprofit working on models of past and future progress in AI, intelligence enhancement, and sociology related to existential risks.
A series of proposals developed by ControlAI intended for action by policymakers in order for humanity to survive artifical superintelligence.
Convening academic, civil society, industry, and media organizations to create solutions so that AI advances positive outcomes for people and society.
Nonprofit researching solutions and strategies, mobilizing resources, and uncovering actionable insights to safeguard a flourishing present and future.
Living document aiming to present a coherent worldview explaining the race to AGI and extinction risks and what to do about them – in a way that is accessible to non-technical readers.
Aiming to improve institutional response to existential risk from future AI systems by conducting research and outreach, and developing educational materials.
Strategy, Advocacy
Building a foundational series of sociotechnical reports on key AI scenarios and governance recommendations, and conducting AI awareness efforts to inform the general public.
Answering decision-relevant questions about the future of AI, including through research, a wiki, and expert surveys. Run by MIRI.
Strategy, Forecasting
Think tank aiming to transform global resilience to extreme risks by improving relevant governance, processes, and decision-making.
Strategy, Governance
Workshop letting decision-makers experience the tensions and risks that can emerge in the highly competitive environment of AI development through an educational roleplay game.
Strategy, Governance, Training and education
Platform for connecting junior researchers and seasoned civil servants from Southeast Asia with senior AI safety researchers from developed countries.
Training and education
3-month part-time online research program with mentorship, aimed at helping people who want to work on AI safety team up together on concrete projects.
Supports students and professionals in contributing to the safe development of AI, including through introductory seminars.
Georgia Tech community hosting fellowships and research projects investigating open problems in AI safety, including specification, robustness, interpretability, and governance.
Course from the Center for AI Safety (CAIS) covering a wide range of risks while leveraging concepts and frameworks from existing research fields to analyze AI safety.
Comprehensive, up-to-date directory of AI safety curricula and reading lists for self-led learning at all levels.
Accessible guide to AI safety for those new to the space, in the form of a comprehensive FAQ and AI safety chatbot. Project of Rob Miles.
4–5 week ML engineering upskilling program, focusing on alignment. Aims to provide individuals with the skills, community, and confidence to contribute directly to technical AI safety.
Short hackathons and challenges, both online and in-person around the world, focused on important questions in AI safety.
10-week remote mentorship program for women looking to strengthen their research skills and network in technical Al alignment research.
Runs the standard introductory courses, each three months long and split into two tracks: Alignment and Governance. Also runs other courses.
Helping students get into AI safety research via upskilling programs and fellowships. Supports AISST and MAIA.
In-person, paid, 8-week summer research fellowship at the University of Cambridge for aspiring AI safety and governance researchers.
Designed for students and professionals who are interested in research in human-compatible AI. Interns work on a research project supervised by a mentor.
2-3 month summer research fellowship in London working on challenging research questions relevant to reducing suffering in the long-term future.
Annual program providing students and early-career professionals in AI, computer science, and related disciplines with a firm grounding in the emerging field of cooperative AI.
1-year program catalyzing collaboration among young scientists, engineers, and innovators working to advance technologies for the benefit of life.
Remote, part-time research opportunities in AI safety, policy, and philosophy. Also provide ongoing support, including coworking sessions, issue troubleshooting, and career guidance.
Intensive 3-day workshops for students to explore the foundational arguments around risks from advanced AI (and biotechnology).
3-month program from the Centre for the Governance of AI designed to help researchers transition to working on AI governance full-time.
4-day program held in Prague, Czech Republic, teaching alignment research methodology through talks, workshops, and discussions.
Fully-funded program from the Institute for AI Policy and Strategy for professionals seeking to strengthen practical policy skills for managing the challenges of advanced AI.
Fully-funded research program connecting exceptional STEM researchers with full-time placement opportunities at AI safety labs and organizations.
8-week mentored research program for London students, where teams explore a research question tackling global challenges – including AI governance and technical AI safety.
12-week technical research program aiming to assist individuals in transitioning to full-time careers in AI safety.
Research program connecting talented scholars with top mentors in AI safety. Involves 10 weeks onsite mentored research in Berkeley, and, if selected, 4 months extended research.
8-day intensive in-person bootcamps upskilling participants in technical AI safety research, held in various locations around the world.
Research program run by CAISH, connecting aspiring researchers with experienced mentors to conduct AI safety (technical or policy) research for 2–3 months.
Aiming to empower bright high schoolers to start solving the world's most pressing problems through various research programs.
London-based talent development scheme designed to equip outstanding students with the knowledge and skills to shape the future of AI governance.
Pairs fellows from disciplines studying complex and intelligent behaviour in natural and social systems with mentors from AI safety.
Fellowship run by Kairos for students organizing technical AI safety or AI policy university groups – providing mentorship, funding, and other resources.
Annual 9-week program designed to enable promising researchers to produce impactful research and accelerate their careers in AI safety (or biosecurity).
Virtual research program run by Kairos, offering early-career individuals and professionals the chance to engage in AI safety research for 3 months.
7-month program enabling ambitious graduates to launch EU policy careers reducing risks from AI. Fellows participate in one of two tracks: training or placement.
10-week summer research fellowship giving undergraduate and graduate students the opportunity to produce high impact research on various emerging threats, including AI.
Filipino nonprofit aiming to develop more AI interpretability and safety researchers, particularly in Southeast Asia.
Nonprofit supporting journalism that helps society navigate the emergence of increasingly advanced AI. Runs fellowships, grants, and residencies.
Training and education, Funding
African-led research program dedicated to building talent, generating research, and shaping policy to advance AI safety.
Training and education, Governance
Supports individuals in London interested in working on AI safety by raising awareness of the risks, especially in universities, and providing high-quality resources and support.
Training and education, Research support
YouTuber discussing the latest AI developments as they happen, offering explanations and analysis of important research and events.
Video
Channel produced by 80,000 Hours presenting thoroughly researched, cinematic stories about what's happening in AI and where the trends are taking us.
Comprehensive directory of AI safety video content, from beginner-friendly introductions to in-depth expert talks.
Channel hosted by Liron Shapira featuring in-depth debates, explainers, and live Q&A sessions, focused on AI existential risk and other implications of superintelligence.
Computer science PhD and AI research scientist talking about how AI will likely affect all of us and society as a whole, plus interviews with experts.
Talks given by AI safety experts at various events, covering topics ranging from mechanistic interpretability to evals and governance.
Animated videos aiming to foster good thinking, promote altruistic causes, and help ensure humanity's future goes well – particularly regarding AI safety.
YouTube channel featuring visually-rich explanations of AI safety concepts; highlighting potential risks, clarifying advanced topics, and advocating for responsible development.
YouTube channel explaining (mostly) AI safety concepts through entertaining stickman videos.
YouTube channel run by John Sherman, a Peabody and Emmy Award-winning former investigative journalist, aiming to make AI risk a kitchen table conversation.
Suggest entry →
Suggest correction →
Let us know of changes to something listed here
View raw data →
See the database in Airtable
We're a global team of volunteers and professionals from various disciplines who believe AI poses a grave risk of extinction to humanity.
Help us out
Partner projects
-(1).webp){kind=logo}
-(1).webp)
.svg)
-(1).png)
-(1).png)
-(1).png)
.svg)
-(1).svg)
.svg)
-(1).png)
-(1)-(1).webp)
-(1)-(1).webp)
-(1)-(1).webp)
-(1).webp){kind=logo}
.webp)
.svg)
-(1)-(1).webp)
.webp)
.webp)
.jpeg)
-(1).png)
.svg){kind=logo}
.svg)
.svg)
-(1).webp)
.svg)
.svg)
-(1).png)
-(1).webp)
-(1).webp)
.svg)
-(1).png)
.svg){kind=icon}
.webp)
.svg)
-min-(1)-(1).png)
.webp){kind=logo}
-(2).png)
-(1).webp)
-(1).jpeg)
.svg)
-(1).jpeg)
.svg)
.svg){kind=logo}
-(1).png)
-(1).png)
.svg)
-(1).png)
.webp)
.svg)
.svg)
.webp)
-(1).png)
.svg)
-(1).png)
.svg)
-(1).webp)
.svg)
-(1).png){kind=logo}
.png)
.webp)
-(1).png)
-(1).png)
-(1).png)
.svg)
-(1).png)
.svg)
.svg)
.svg)
.svg)
-(1).png)
.svg)
.svg)
-(1).webp)
-(1).png)
-(1).png)
-(1).png)
-(1).png)
.webp)
.svg)
.svg)
.svg)
-(1)-min.png)
.svg)
.svg)
-(1).png)
.svg)
-(1)-(1).webp)
-(1).png)
-(1).webp)
.svg)
-(1).png)
.png)
.svg)
.svg)
.webp)
.png)
.svg)
-min-(1).png)
-(1).png)
-(1).png)
.svg)
.svg)
-(1).webp)
.svg)
.png)
-min-(1).png)
.png)
.webp)
.svg)
-min.png)
.svg){kind=logo}
.webp)
.png)
-(1).png)
.svg)
-(1)-(1).webp)
.svg)
-(1)-(1).png)
.svg)
-(1).png){kind=logo}
-(1)-(1).png)
.svg)
-(1)-(1).webp)
-(1).svg)
.svg)
.jpeg)
-(1).webp)
-(1).webp)
.svg)
.svg)
-(1).svg)
.svg)
.svg)
.png)
.svg)
.svg)
-(1).png)
-(1).png)
-(1).webp)
-(1).png)
-(1).webp)
-(2)-(1).webp)
.png)
.svg)
-(1).webp)
-(1).webp){kind=icon}
.svg)
.webp)
.svg)
.svg)
.svg)
.svg)
.webp)
-(1).webp)
.webp)
-(1).png)
.svg)
.svg)
.svg)
-(1).webp)
.svg)
.svg)
.svg)
-(1).webp)
-(1).png)
-(1).png)
.png)
.svg)
.svg)
-(1).png)
-(1).png)
.svg)
-(1).webp)
.svg)
.png)
.svg)
.webp)
-(1).png)
.webp){kind=icon}
-(1).png)
-(1).png)
-(1).png)
.png)
.svg)
-(1).webp)
.jpeg)
.webp)
-(1).png)
.svg)
-(1).png)
.svg)
-(1)-(1).webp)
.svg)
-(1).webp)
.svg)
-(1).webp){kind=logo}
.svg)
.svg)
.svg)
-(1).webp)
.svg)
-(1).png)
.svg)
.svg)
-(1).png)
-(1).png)
-(1).png)
.webp)
.png)
-(1).png)
-(1).webp)
.svg)
.svg)
.png)
.png)
.webp)
.svg)
.svg)
.svg)
.svg)
.png)
.svg)
.svg)
.svg)
.webp)
-(1).webp)
.webp){kind=logo}
.svg)
.svg)
-(1).png)
-(1).png)
-(2).png)
-(1).webp)
.svg)
-(1).png)
.svg)
.svg)
.svg)
.svg)
.svg)
-(1).png)
.svg)
-(1).webp){kind=logo}
-(1).webp)
.webp)
.webp)
.svg)
-(1).png)
.webp)
.webp){kind=logo}
.png)
-(1).webp)
.svg)
-(1).png)
-(1).png)
-(1).webp)
.png)
.png)
.png)
-(1).webp)
-(1).png)
-(1).png)
.svg)
.svg)
.svg)
.svg)
.svg)
.png)
-(1).png)
-(1).webp)
-(1).webp)
-(1).webp)
.webp)
.svg)
.png)
-(1).png)
-(1).png)
-(1).png)
.svg)
.svg)
.svg)
.svg)
.webp)
-(1).png)
-(1).png)
.webp)
-(2)-(1).webp)
.svg)
-(1).png)
-(1).png)
-(1).png)
-(1).webp)
-(1).webp)
.svg)
.png)
-(1).png)
-(1).png)
-(1).jpeg)
.svg)
-(1).png)
-(1).png)
-(1).png)
.webp)
-(1).webp)
.webp)
-(2)-(1).webp)
.svg)
-(1).png)
-(1).webp)
-(1).webp)
-(1).png)
.svg)
