Великобритания
СШАResearch Scientist/Engineer (Fundamental Science)
1 нед. назад
ВеликобританияСШАOnsite
Research Scientist/Engineer role focused on fundamental science to improve embedded evaluations and develop new scientific approaches.
О компании
- Application deadline: We are conducting interviews actively and aim to fill this role as soon as we find someone suitable. ABOUT THE OPPORTUNITY The OpenAI/Hugging Face incident may be the biggest warning shot we get before catastrophic loss-of-control. In its aftermath, Anthropic and OpenAI committed to giving embedded evaluators an unprecedented level of access. Embedded evaluations are the best existing method to find out whether alignment is on track. With OpenAI predicting an automated researcher by March 2028, we don't have time to start with an MVP embedded evaluation and iteratively improve it. The first embedded evaluations will set the standard and we have to get it right. As a fundamental researcher, your ground-breaking discoveries will make our embedded evaluations better. For example, our fundamental science team's paper “Measuring Reward-Seeking via Contrastive Belief Updates” can be adapted by embedded evaluators. We want more breakthroughs like this. We are also developing a “Science of Scheming” . The goal is ambitious and we’re looking for Research Scientists and Research Engineers who are excited to build a new hard science from the ground up.
Обязанности
- Collaborate with leading AI developers. We partner with multiple labs, giving you access to a breadth of models that no single AI lab could offer. Through long-term research collaborations, your work directly impacts how the most capable AI systems are built and deployed.
- Deeply study the RL dynamics that lead to the emergence of reward-seeking, evaluation awareness or misaligned preferences. Design and train model organisms, and scale your insights to frontier systems.
- Work towards “Scaling laws of scheming”. Build the empirical foundations to predict how scheming risks evolve as models scale in capability.
- Develop novel and ambitious evaluation techniques that have a chance of scaling to highly evaluation aware models.
- Deep dive into AI cognition. Discover patterns in the reasoning processes of frontier AI systems that no one else has ever observed before.
Требования
- Fast-paced empirical research: You can design and execute experiments. You always strive to speed up iteration cycles and relentlessly drive progress towards the next empirical milestone.
- AI power-user: You’re capable of using AI to accelerate your work, technical or otherwise. You have experience using different models, know which ones to use for which tasks, when not to use AI, and always experiment with new AI workflows.
- Conceptual thinking and writing : You are familiar with prior work on AI scheming, from classic literature like Is Power-Seeking AI an Existential Risk? to modern research. You have done enough deep thinking about the problem to generate some original ideas. Given vague and underspecified concepts, you come up with clarifying insights, clever ideas for experiments, and detailed plans. To explain your work, you can produce high-quality scientific writing, figures, blog posts, microsites, and tweet threads.
- Intense interest in AI progress: You always stay up to date on the latest model releases, and continuously tinker with new and creative AI workflows to speed up your work. You are fascinated by AI cognition and actively spend time trying to understand how they think.
- Experience RL-training LLMs: You have hands-on experience in training LLMs via reinforcement learning. You have encountered and resolved countless painful issues from GPU failures to debugging learning instabilities.
- Strong analytical skills: You bring rigorous quantitative chops from working on fields such as scaling laws in LLMs, statistical physics, dynamical systems, applied statistics etc. You're comfortable building mathematical models of empirical phenomena and know how to extract signal from noisy data.
- Mission commitment: Apollo estimates that RSI will happen soon, and we are extremely concerned that AGI will catastrophically not be aligned with human values. Hence we have an “intense, but in a good way” culture . We will ask a lot from you, but in return you will personally have a large impact on loss-of-control risk.
- Software engineering skills: Strong software engineering skills correlate highly with effective execution, even in an era of AI agents. Our entire stack uses Python.
- Project management experience: We are rapidly growing while executing high-quality research projects as fast as possible. You have previously managed strong ICs and can keep research projects on schedule despite unexpected delays or surprising experiment results.
Условия
- This role offers a market-competitive salary, equity, and competitive benefits.
- Depending on your experience, you may be offered a Lead or individual contributor position.
- Salary:
- Recognised experts who can lead projects of significant scope: USD 350,000 - USD 590,000 or £230,000 - £395,000
- Strong individual contributors: USD 245,000 - USD 405,000 or £160,000 - £270,000
- Unlimited vacation
- Unlimited sick leave
- Up to 6 months of paid parental leave
- Comprehensive health, dental and vision insurance
- Retirement savings with competitive employer matching (e.g. 401(k) for US employees)
- Lunch, dinner, and snacks are provided for all employees on workdays
- Paid work trips, including staff retreats, business trips, and relevant conferences
- A yearly $1,000 (USD) professional development budget
- Relocation support and visa fees (if applicable). We are able to sponsor visas to the UK and US.
Другое
- Time Allocation : Full-time
- Location : This is an in-person role working out of our London or San Francisco office. We offer flexible working hours and wfh arrangements. Remote candidates will only be considered if they are exceptional and can travel at least 1 week/month to either office.
- Visa sponsorship: We sponsor visas in both the UK and US. Sponsorship isn't guaranteed for every role or candidate, but if we make you an offer, we'll work with you to find the right visa route.