Projects

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Alignment Transcription Service

Some approaches to solving alignment go through teaching ML systems about alignment and getting research assistance from them. Training ML systems needs data, but we might not have enough alignment research to sufficiently fine tune our models, and we might miss out on many concepts which have not been written up. Furthermore, training on the final outputs (AF posts, papers, etc) might be less good at capturing the thought processes which go into hashing out an idea or poking holes in proposals which would be the most useful for a research assistant to be skilled at. It might be significantly beneficial to capture many of the conversations between researchers, and use them to expand our dataset of alignment content to train models on. Additionally, some researchers may be fine with having their some of their conversations available to the public, in case people want to do a deep dive into their models and research approaches. The two parts of the system which I'm currently imagining addressing this are: An email where audio files can be sent, automatically run through Whsiper, and added to the alignment dataset github. Clear instructions for setting up a tool which captures audio from calls automatically (either a general tool or platform-specific advice), and makes it as easy as possible to send the right calls to the dataset platform.

Contact

plex, michael trazzi would be a good person to talk to (he's already hiring ppl to edit otter docs), and daniel filan

Volunteer with AI-plans.com

Currently, newcomers to the field of AI Alignment often struggle to understand the ongoing work and individuals involved, as well as the assumptions, strengths, and weaknesses of each plan.   We believe AI-plans.com will be an easy, centralized way to discover and learn more about the most promising alignment plans.   The site is currently in Stage 1, functioning purely as a compendium. We are in the process of adding up to 1000 plans and the criticisms made against them so far. Further plans and criticisms can be added by users.   Projected benefits of Stage 1: - Easy discovery of proposed plans and better understanding of their prevalent challenges.  (This is already showing promise, with one researcher letting us know they found useful papers on the site and multiple researchers interested- including Jonathan Ng who has been helping us.) Next, in Stage 2, we will introduce a scoring system for criticisms and a ranking system for plans. Plans will be ranked based on the cumulative scores of their criticisms. Criticism votes will be weighted, giving more influence to users who have submitted higher-scoring criticisms. Alignment researchers will have the option to link their AI-Plans account to accounts on research-relevant platforms (such as arXiv, OpenReview or the AI Alignment Forum) in order to start out with a slightly weighted vote (with mod approval).   Each new plan will start with 0 bounty, and lower bounty plans will give the most points. That way, each new plan will have a lot of opportunity and incentive for criticism. More details here [https://www.lesswrong.com/posts/hcTiw9xKNZAi7qcy6/an-overview-of-the-points-system].   Projected benefits of Stage 2: - Incentivizes users to write high-quality criticisms. - Facilitates identification of plans with significant weaknesses, supporting arguments against problematic plans. - Allows newcomers to the field(including talented and untapped scientists and engineers) to see which companies have the least problematic plans.  After all, who would want to work for the lowest-ranked company on the leaderboard?  (I have spoken with the creator of aisafety.careers, who intends to integrate with our site.)   At Stage 3, in addition to everything from Stage 1 and 2, we plan to introduce monthly cash prizes for the highest ranking plan and for the users with the most criticism points that month.  Projected benefits of Stage 3: - Supercharges the impact of Stage 2, attracting talented individuals who require a non-committal monetary incentive to engage with alignment research. - Provides a heuristic argument for the difficulty of the problem: "There is money on the table if anyone can come up with a plan with fewer problems, yet no one has done so!"

Contact

Kabir- kabir03999@gmail.com

Make It Easier for AGI Safety Endeavors to Get Funding from Non-EA Sources

Our primary funding sources have suffered last year, and there are numerous foundations and investors out there happy to invest into potentially world-saving and/or profitable projects. Especially now, it might be high-leverage to collect knowledge and build infrastructure for tapping into these funds. I lack the local knowledge to give recommendations for how to tap into funding sources within academia. However, here are four potential routes for tapping into non-academic funding sources: 1. Offer a service to proofread grant applications and give feedback. That can be extremely valuable for relatively little effort. Many people don't want to send their application to a random stranger, but maybe people know you from the EA forum? Or you can just offer giving feedback to people who already know you. 2. Identify more relevant funding sources and spread knowledge about them. https://www.futurefundinglist.com/ [https://www.futurefundinglist.com/] is a great example: It's a list of dozens of longtermist-adjacent funds, both in and outside the community.  Governments, political parties, and philanthropists often have nation-specific funds happy to subsidize projects. Expanding the Future Funding List further and finding/building similar national lists might be extremely valuable. For example, there is a whole book [https://web.fundraiser-magazin.de/buch-foerdertoepfe-fuer-vereine] with funding sources for charity work in German. 3. Become a professional grant writer. A version of this that is affordable for new/small orgs and creates decent incentives and direct feedback for grantwriters might be a prize-based arrangement: Application writers get paid if and only if a grant gets through. If you are interested in this and already bring an exceptional level of written communication skills, a reasonable starting point may be grant writing courses like https://philanthropyma.org/events/introduction-grant-writing-7 [https://philanthropyma.org/events/introduction-grant-writing-7]. 4. Teach EAs the skills to communicate their ideas to grantmakers. Different grantmakers have different values and lingos. If you want to convince them to give you money, you have to convince them in their world. This is something many AGI safety field builders didn’t have to learn so far. Accordingly, a useful second step after becoming a grant writer yourself might be figuring out how to teach grant writing as effectively as possible to the relevant people. (A LessWrong/EA Forum sequence? Short rainings in pitching and grantwriting?)

Contact

Facilitate people’s transition from AI capabilities research to AI safety research

Increasing the ratio of AI safety to AI capabilities researchers (which is currently plausibly as skewed as 1:300[1] [https://forum.effectivealtruism.org/posts/AJwuMw7ddcKQNFLcR/20-concrete-projects-for-reducing-existential-risk#fnh3bpfb1q9h]) seems like a core way to reduce existential risk from AI. Helping capabilities researchers switch to AI safety could be especially effective since their experience working on frontier models would allow uniquely useful insights in alignment research, and their existing skills would likely transfer well and shorten any training time needed. While there are many existing projects providing career advice and upskilling programs for people who want to enter AI safety, we’re not currently aware of transition programs that specifically target people who already have a strong frontier ML background. We think this could be a valuable gap to fill since this group likely needs to be approached differently (e.g., ML expertise should be assumed) and will have different barriers to switching into AI safety compared to, say, recent graduates. The project would need to be better scoped, but some directions we imagine it taking include identifying and setting up shovel-ready projects that can absorb additional talent, offering high salaries to top ML talent transitioning to safety research, and providing information, support and career advice for people with existing expertise. We think this is a relatively challenging and potentially sensitive project. In order to prevent inadvertently causing harm, we think the founding team would likely need to have a deep understanding of alignment research, including the blurry distinction between alignment and capabilities, as well as the relationship between the capabilities and safety communities.  The organization could do several kinds of outreach: booths at conferences, lectures to ML grad students, targeted ads online, content marketing, AI safety career coaching, etc. It could provide explanations of the rationale, one-on-one coaching to find a good research path and get into the industry, provide introductory materials, etc. 

Contact

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

(ɔ) 2024 · This site is released under a CC BY-SA license