Safety Operations Lead

3 нед. назад

USALeadHybrid

policy designcase triagecontent moderationautomation toolingabuse detection

Lead safety operations to ensure product safety and integrity in human-AI collaboration tools by reviewing flagged content, applying policies, and building moderation tooling.

Обязанности

We’re looking for a Safety Operations Lead who will focus on making our products safe by default while supporting fast product iteration and ambitious ideas.
You’ll work closely with product engineers, security, researchers, and designers to bake safety and integrity into the way we design, build, and ship human-AI collaboration tools. Day to day, you'll be in the moderation queue while building the tooling and policy that make the next round of moderation faster and more accurate.
You have thoughtful opinions about what safe, trustworthy, frictionless user experiences look like, and you test those opinions against real cases in the queue, not just in the abstract.
You can translate safety and technical constraints into clear product trade-offs and feature requirements, but you see that as something that grows out of daily casework, not a substitute for it.
You bias toward speed and learning, and you measure that bias by how much faster or better the queue runs this month, not by how many new risk categories you've personally discovered.

Требования

7+ years in an operational trust & safety, content moderation, or fraud/abuse ops role with direct, recurring responsibility for a case queue.
Experience owning policy definition, operationalization, and enforcement end to end, evidenced by specific policies or enforcement programs you've built or run.
Direct case experience with at least one of: cybersecurity abuse, CBRN-relevant risk, youth safety, or prompt injection, in a production environment.
Working familiarity with model safety and abuse risk categories (jailbreaks, prompt injection, scaled abuse) and how to identify and mitigate them in a live product, evidenced by specific cases you've handled.
Practical experience using AI tools (Claude, Codex, or similar) to build or accelerate operational workflows, not just as a general user of these tools.

Будет плюсом

We encourage you to apply even if you don’t meet all preferred qualifications.
Experience with safety and integrity operations specifically on AI-powered products or LLM APIs, and their unique abuse patterns.
Track record of turning recurring case patterns into reusable tooling, workflows, or process improvements, while still owning the underlying queue rather than handing it off once the interesting part is solved.
Experience training, onboarding, or setting the quality bar for other moderators or reviewers, showing you scale a team's output rather than just your own.

Другое

The mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.
Review flagged content, safety escalations, and account-level abuse signals daily. This is a standing responsibility, not a rotation: you'll triage cases, apply policy judgment, and take action (content flags, account review, bans/recovery) on an ongoing basis.
Use patterns from that casework to design and refine safety policy across the product stack, working with engineering, legal, safety research, and security stakeholders.
Build and maintain tooling and automation that make your own casework faster and more consistent: triage agents, ban/recovery workflows, abuse detection frameworks and templates.
Partner with product teams to embed safety into the product experience: model refusals, content flagging, account review, and safety protections, informed by what you're seeing in the queue.
Improve observability and detection for safety-relevant events (model safety trends, abuse patterns, malicious behavior in production), so the next round of cases surfaces faster and with better signal.
Location: This role is based in San Francisco, California.
Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $190,000 - $300,000.
Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.
As set forth in Thinking Machines' Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.