Funders

Open Philanthropy (OP)

The largest funder in existential risk reduction. Most funding is done via proactive research, but there are also frequent requests for proposals in certain areas.

Type

Fund

Accepting applications

Yes

Manifund

Regranting platform where individual regranters have funding pools to direct towards listed projects. Also offers impact certificates.

Type

Platform

Accepting applications

Yes

UK AI Security Institute (AISI): Challenge Fund

Awarding grants of up to £200,000 per project to address pressing, unresolved questions in AI safety. Researchers worldwide can access grants for innovative research in fields such as cyber-attacks and AI misuse.

Type

Grant program

Accepting applications

Yes

Future of Life Foundation (FLF)

Incubator helping start new organizations that steer transformative technology towards benefiting life and away from extreme large-scale risks.

Type

Incubator

Accepting applications

Yes

CSET: Foundational Research Grants

Supports the exploration of foundational technical topics that relate to the potential national security implications of AI over the long term, shedding light on questions of interest from a strategic or policy perspective.

Type

Grant program

Accepting applications

No

Thiel Fellowship

Two-year program for young people who want to build new things. Fellows skip or drop out of college to receive a US$200,000 grant and support from the Thiel Foundation’s network of founders, investors, and scientists.

Type

Grant-based fellowship

Accepting applications

Yes

Foresight Institute

Funding projects in 1) research and forecasting automation, 2) neurotech to integrate with or compete against AGI, 3) security technologies for AI-relevant systems, and 4) safe multi-agent scenarios.

Type

Grant program

Accepting applications

Yes

AI Risk Mitigation (ARM) Fund

Aiming to reduce catastrophic risks from advanced AI through grants towards technical research, policy, and training programs for new researchers.

Type

Fund

Accepting applications

No

Effective Altruism Infrastructure Fund (EAIF)

Aiming to increase the impact of projects that use the principles of effective altruism by increasing their access to talent, capital, and knowledge. Application processing is currently paused until 31 July 2025.

Type

Fund

Accepting applications

Yes

Longview Philanthropy

Devises and executes bespoke giving strategies for major donors. Grants focus on reducing the risks posed by AI, nuclear war, and engineered pandemics. Currently seeking proposals on hardware-enabled mechanisms.

Type

Fund

Accepting applications

Yes

AI Safety Science

Funder housing the science-focused philanthropic efforts of Eric and Wendy Schmidt, which moves large amounts of funding towards AI safety via its AI institute./

Type

Grant program

Accepting applications

No

ARIA

UK government R&D funding agency built to unlock scientific and technological breakthroughs that benefit everyone.

Type

Fund

Accepting applications

Yes

GiveWiki

Crowd-sourced charity evaluator and one-stop shop for AI safety funding applications. All applications are public and charities that receive the most support from donors with the best track records are ranked highest.

Type

Platform

Accepting applications

Yes

Catalyze Impact

Facilitates faster progress in solving the AI alignment problem by supporting individuals in setting up AI safety research organizations.

Type

Incubator

Accepting applications

No

Cooperative AI Foundation (CAIF)

Charity foundation backed by a large philanthropic commitment supporting research into improving cooperative intelligence of advanced AI for the benefit of all.

Type

Fund

Accepting applications

No

AI Safety Fund (AISF)

Aiming to promote the responsible development and deployment of frontier models, minimize risks, and enable independent, standardized evaluations of capabilities and safety.

Type

Fund

Accepting applications

No

Future of Life Institute (FLI)

Makes grants for studying the risks from powerful technologies and developing strategies for reducing them. The goal is to win the race between the growing power of our technology and the wisdom with which we manage it.

Type

Grant-based fellowship

Accepting applications

No

Center on Long-Term Risk (CLR) Fund

Supports projects and individuals aiming to address worst-case suffering risks from the development and deployment of advanced AI systems.

Type

Fund

Accepting applications

Yes

Ergo Impact

Helps major donors find, fund, and scale the most promising solutions to the world’s most pressing problems.

Type

Incubator

Accepting applications

No

Vista Institute for AI Policy

Sponsors students and recent graduates to undertake independent research with mentor guidance or to serve as research assistants with law professors and other AI policy experts.

Type

Grant-based fellowship

Accepting applications

Yes

Tarbell Grants

Journalism will likely play a crucial role in helping the public understand AI – and in holding companies and policymakers to account. Tarbell Grants offers awards of $1–15k to support journalism on AI and its impacts.

Type

Grant program

Accepting applications

No

Constellation Residency

Year-long salaried position for experienced researchers, engineers, entrepreneurs, and other professionals to pursue self-directed work on AI safety.

Type

Grant-based fellowship

Accepting applications

Yes

AE Studio Research

Empowering innovators and scientists to increase human agency by creating the next generation of responsible AI. Provides support, resources, and open-source software.

Type

Grant program

Accepting applications

Yes

Saving Humanity from Homo Sapiens (SHfHS)

Small organization with a long history of finding people doing impactful work to prevent human-created existential risks and financially supporting them.

Type

Fund

Accepting applications

No

Macroscopic Ventures

Swiss nonprofit focused on reducing suffering risks, including those posed by catastrophic AI misuse and conflict. Makes both grants to nonprofits and investments in socially beneficial companies.

Type

Fund

Accepting applications

No

Emergent Ventures

Run by the Mercatus Center, this program seeks to support entrepreneurs and brilliant minds with highly scalable, “zero to one” ideas for meaningfully improving society.

Type

Grant-based fellowship

Accepting applications

Yes

Seldon

12-week program in San Francisco, USA, for pre-seed companies composed of top-tier talent aiming to make an impact with AI safety and assurance technologies.

Type

Incubator

Accepting applications

No

Good Ventures

Foundation run by Dustin Moskovitz and Cari Tuna providing funding in a range of cause areas, including minimizing risks from potentially world-changing science and technology.

Type

Fund

Accepting applications

No

Def/acc at Entrepreneur First

Incubation program focusing on "defensive" tech – the idea that the most powerful solution to technological risk is often more technology.

Type

Incubator

Accepting applications

No

AI2050

Program from Schmidt Sciences supporting exceptional people working on key opportunities and hard problems that are critical to get right for society to benefit from AI.

Type

Grant-based fellowship

Accepting applications

No

Founders Pledge

Helping entrepreneurs direct their charitable giving where it will do the most good by researching the world’s biggest problems and identifying the most effective solutions.

Type

Fund

Accepting applications

No

Safe AI Fund (SAIF)

Early-stage venture fund dedicated to supporting startups developing tools to enhance AI safety, security, and responsible deployment. Provides both financial investment and mentorship.

Type

Venture capitalist

Accepting applications

Yes

Economics of AI Fellowship

Fellowship run by Stripe for graduate students and early-career researchers who are interested in pursuing foundational academic research around the economics of AI.

Type

Grant-based fellowship

Accepting applications

No

Lionheart Ventures

VC firm investing in ethical founders developing transformative technologies that have the potential to impact humanity on a meaningful scale.

Type

Venture capitalist

Accepting applications

No

Juniper Ventures

VC firm investing in companies helping make AI safe. Particularly interested in security, compliance, evals & audits, insurance & risk management, trust & authentication, hardware & networking, governance, and interpretability.

Type

Venture capitalist

Accepting applications

No

Nonlinear Network

Platform taking grant applications and presenting them to a donor circle, who then reach out if they are interested.

Type

Platform

Accepting applications

No

The Audacious Project

Funding initiative housed at TED helping entrepreneurs shape impactful ideas into viable multi-year plans and launching them to the world alongside visionary philanthropists.

Type

Incubator

Accepting applications

Yes

Interact Fellowship

For technologists with an enthusiasm for making intentional impact. Fellows attend two summer retreats and receive access to grants programs for their personal projects, and accountability and support mechanisms for founders.

Type

Grant-based fellowship

Accepting applications

No

Mythos Ventures

Aiming to empower founders building a radically better world with safe AI systems by investing in ambitious teams with defensible strategies that can scale to post-AGI.

Type

Venture capitalist

Accepting applications

No

The Navigation Fund

Offers grants to high-impact organizations and projects that are taking bold action and making significant changes.

Type

Fund

Accepting applications

No

Superlinear Prizes

Decentralized bounty platform offering prize money for delivering on specific AI safety projects.

Type

Platform

Accepting applications

No

Lightspeed Grants

Funds projects that have a chance of substantially changing humanity's future trajectory for the better. Disbursed 8 million USD in the 2023 round but has been inactive since.

Type

Fund

Accepting applications

No