Funding

Long-Term Future Fund (LTFF)

Active large grantmaker funding individuals and small projects aiming to positively influence the long-term trajectory of civilization. Relatively straightforward application process.

Type

Fund

Accepting applications

Yes

BlueDot AGI Strategy Fund

$5–50k grants for individuals building high-impact AI safety projects. Applicants must have completed their AGI Strategy course.

Type

Grant program

Accepting applications

Yes

Manifund

Regranting platform where individual regranters have funding pools to direct towards listed projects. Also offers impact certificates.

Type

Platform

Accepting applications

Yes

EU AI Safety Tenders

The EU is funding important projects to research AI safety, including AI risk modelling, AI accountability, driverless transport, robotics, information security, and many other topics. Tendery.ai offers free support in applying.

Type

Grant program

Accepting applications

Yes

CSET: Foundational Research Grants

Supports the exploration of foundational technical topics that relate to the potential national security implications of AI over the long term, shedding light on questions of interest from a strategic or policy perspective.

Type

Grant program

Accepting applications

Yes

Future of Life Institute (FLI)

Operates 3 fellowships to provide financial support to researchers working on AI safety: US-China AI Governance PhD, Technical PhD, and Technical Postdoctoral.

Type

Grant-based fellowship

Accepting applications

Yes

Thiel Fellowship

Two-year program for young people who want to build new things. Fellows skip or drop out of college to receive a US$200,000 grant and support from the Thiel Foundation’s network of founders, investors, and scientists.

Type

Grant-based fellowship

Accepting applications

Yes

Foresight Institute

Funding projects in 1) research and forecasting automation, 2) neurotech to integrate with or compete against AGI, 3) security technologies for AI-relevant systems, and 4) safe multi-agent scenarios.

Type

Grant program

Accepting applications

Yes

Cooperative AI PhD Fellowship

Provides financial support to future and current PhD students in the field of cooperative AI. Run by the Cooperative AI Foundation (CAIF).

Type

Grant-based fellowship

Accepting applications

Yes

5050

13-week program run by Fifty Years helping scientists and engineers build startups tackling world-scale problems.

Type

Incubator

Accepting applications

No

AI Risk Mitigation (ARM) Fund

Aiming to reduce catastrophic risks from advanced AI through grants towards technical research, policy, and training programs for new researchers.

Type

Fund

Accepting applications

No

Effective Altruism Infrastructure Fund (EAIF)

Aiming to increase the impact of projects that use the principles of effective altruism by increasing their access to talent, capital, and knowledge.

Type

Fund

Accepting applications

Yes

Astralis Foundation

New AI safety funding initiative with $25M annual giving, aiming to unite funders, domain experts, and entrepreneurs to launch and scale ambitious initiatives.

Type

Fund

Accepting applications

No

Future of Life Foundation (FLF)

Incubator helping start new organizations that steer transformative technology towards benefiting life and away from extreme large-scale risks.

Type

Incubator

Accepting applications

No

Longview Philanthropy

Devises and executes bespoke giving strategies for major donors. Grants focus on reducing the risks posed by AI, nuclear war, and engineered pandemics.

Type

Fund

Accepting applications

No

AI Safety Science

Funder housing the science-focused philanthropic efforts of Eric and Wendy Schmidt, which moves large amounts of funding towards AI safety via its AI institute.

Type

Grant program

Accepting applications

No

UK AI Security Institute (AISI): The Alignment Project

An international, cross-sector coalition offering grants of £50k to £1m to alignment researchers. Funding is available across a range of categories (compute, salaries, etc.) and project types (orgs, teams, independents).

Type

Grant program

Accepting applications

No

ARIA

UK government R&D funding agency built to unlock scientific and technological breakthroughs that benefit everyone.

Type

Fund

Accepting applications

No

GiveWiki

Crowd-sourced charity evaluator and one-stop shop for AI safety funding applications. All applications are public and charities that receive the most support from donors with the best track records are ranked highest.

Type

Platform

Accepting applications

Yes

Founders Pledge

Helping entrepreneurs direct their charitable giving where it will do the most good by researching the world’s biggest problems and identifying the most effective solutions.

Type

Fund

Accepting applications

No

Catalyze Impact

Facilitates faster progress in solving the AI alignment problem by supporting individuals in setting up AI safety research organizations.

Type

Incubator

Accepting applications

No

Cooperative AI Foundation (CAIF)

Charity foundation backed by a large philanthropic commitment supporting research into improving cooperative intelligence of advanced AI for the benefit of all.

Type

Fund

Accepting applications

No

AI Safety Fund (AISF)

Aiming to promote the responsible development and deployment of frontier models, minimize risks, and enable independent, standardized evaluations of capabilities and safety.

Type

Fund

Accepting applications

No

Center on Long-Term Risk (CLR) Fund

Supports projects and individuals aiming to address worst-case suffering risks from the development and deployment of advanced AI systems.

Type

Fund

Accepting applications

Yes

Ergo Impact

Helps major donors find, fund, and scale the most promising solutions to the world’s most pressing problems.

Type

Incubator

Accepting applications

No

Vista Institute for AI Policy

Sponsors students and recent graduates to undertake independent research with mentor guidance or to serve as research assistants with law professors and other AI policy experts.

Type

Grant-based fellowship

Accepting applications

Yes

Tarbell Grants

Journalism will likely play a crucial role in helping the public understand AI – and in holding companies and policymakers to account. Tarbell Grants offers awards of $1–15k to support journalism on AI and its impacts.

Type

Grant program

Accepting applications

No

Constellation Residency

Year-long salaried position for experienced researchers, engineers, entrepreneurs, and other professionals to pursue self-directed work on AI safety.

Type

Grant-based fellowship

Accepting applications

No

AE Studio Research

Empowering innovators and scientists to increase human agency by creating the next generation of responsible AI. Provides support, resources, and open-source software.

Type

Grant program

Accepting applications

Yes

Saving Humanity from Homo Sapiens (SHfHS)

Small organization with a long history of finding people doing impactful work to prevent human-created existential risks and financially supporting them.

Type

Fund

Accepting applications

No

Macroscopic Ventures

Swiss nonprofit focused on reducing suffering risks, including those posed by catastrophic AI misuse and conflict. Makes both grants to nonprofits and investments in socially beneficial companies.

Type

Fund

Accepting applications

Yes

Emergent Ventures

Run by the Mercatus Center, this program seeks to support entrepreneurs and brilliant minds with highly scalable, “zero to one” ideas for meaningfully improving society.

Type

Grant-based fellowship

Accepting applications

Yes

Seldon Lab

12-week program in San Francisco, USA, for pre-seed companies composed of top-tier talent aiming to make an impact with AI safety and assurance technologies.

Type

Incubator

Accepting applications

Yes

Lionheart Ventures

VC firm investing in ethical founders developing transformative technologies that have the potential to impact humanity on a meaningful scale.

Type

Venture capitalist

Accepting applications

No

Good Ventures

Foundation run by Dustin Moskovitz and Cari Tuna providing funding in a range of cause areas, including minimizing risks from potentially world-changing science and technology.

Type

Fund

Accepting applications

No

AI2050

Program from Schmidt Sciences supporting exceptional people working on key opportunities and hard problems that are critical to get right for society to benefit from AI.

Type

Grant-based fellowship

Accepting applications

No

Safe AI Fund (SAIF)

Early-stage venture fund dedicated to supporting startups developing tools to enhance AI safety, security, and responsible deployment. Provides both financial investment and mentorship.

Type

Venture capitalist

Accepting applications

Yes

Juniper Ventures

VC firm investing in companies helping make AI safe. Particularly interested in security, compliance, evals & audits, insurance & risk management, trust & authentication, hardware & networking, governance, and interpretability.

Type

Venture capitalist

Accepting applications

Yes

Nonlinear Network

Platform taking grant applications and presenting them to a donor circle, who then reach out if they are interested.

Type

Platform

Accepting applications

No

Def/acc at Entrepreneur First

Incubation program focusing on "defensive" tech – the idea that the most powerful solution to technological risk is often more technology.

Type

Incubator

Accepting applications

No

The Audacious Project

Funding initiative housed at TED helping entrepreneurs shape impactful ideas into viable multi-year plans and launching them to the world alongside visionary philanthropists.

Type

Incubator

Accepting applications

Yes

Interact Fellowship

For technologists with an enthusiasm for making intentional impact. Fellows attend two summer retreats and receive access to grants programs for their personal projects, and accountability and support mechanisms for founders.

Type

Grant-based fellowship

Accepting applications

No

Mythos Ventures

Aiming to empower founders building a radically better world with safe AI systems by investing in ambitious teams with defensible strategies that can scale to post-AGI.

Type

Venture capitalist

Accepting applications

No

The Navigation Fund

Offers grants to high-impact organizations and projects that are taking bold action and making significant changes.

Type

Fund

Accepting applications

No

Economics of AI Fellowship

Fellowship run by Stripe for graduate students and early-career researchers who are interested in pursuing foundational academic research around the economics of AI.

Type

Grant-based fellowship

Accepting applications

No

Superlinear Prizes

Decentralized bounty platform offering prize money for delivering on specific AI safety projects.

Type

Platform

Accepting applications

No

Lightspeed Grants

Funds projects that have a chance of substantially changing humanity's future trajectory for the better. Disbursed 8 million USD in the 2023 round but has been inactive since.

Type

Fund

Accepting applications

No