Software Engineer, ML Infra

вчера

USASeniorOnsite

kernelnetworkscheduler

Software Engineer role focused on ML infrastructure, bridging research and infrastructure to support and debug ML systems.

We're hiring a Software Engineer to sit at the day-to-day interface between research and infrastructure. This is a generalist role with broad scope: you'll be one of the people always in the room for infra decisions on the ML systems side, with a clear enough view of upcoming compute needs to see support burden coming before it arrives.
You'll also be one of the people who babysits hero runs — the ones at 2am when a 4k-GPU job hits a weird Xid and someone needs to decide, quickly and correctly, whether to drain the node, restart the job, or escalate to NVIDIA before the run loses checkpoints. That kind of judgment, built across the kernel, the network, the scheduler, and the application layer, is the core of the job.
Debug across the full stack — kernel, NCCL, scheduler, application, and telemetry — often in the same afternoon, to find root causes that don't show up in any single layer
Provide embedded, hands-on support during hero runs and major incidents, staying with a problem until it's genuinely resolved
Serve as the front door for researchers when something's broken and it isn't obvious who owns it
Lead postmortems and build the tooling that prevents the next incident — acting as a force multiplier, not just a responder
Mentor other engineers into this kind of cross-stack breadth

Credible, hands-on competence in 4 or more of the following: Linux kernel, networking, GPUs / CUDA, distributed systems runtimes, storage, compilers / language runtimes, observability internals — at this level, this required range is the qualifying signal
Comfort operating without a clearly defined owner, and the judgment to know when to dig in yourself versus when to escalate

Track record of being the person other engineers escalate to, across 2+ companies
Shipped meaningful contributions in 3+ distinct technical stacks
Track record of leading major incidents where the root cause was non-obvious
Researcher-facing comfort: can talk to a researcher about their workload without making them feel dumb, and can tell them no when the right answer is no
Experience operating at the scale of frontier training or inference clusters, and appetite for owning the hardest, least-defined problems in that stack

The mission of Thinking Machines is to build AI that extends human will and judgment. We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication. We believe the future worth building is human, and we're hiring people who want to build it.
Location: This role is based in San Francisco, CA.
Compensation: Depending on background, skills and experience, the expected annual salary range for this position is $350,000 – $475,000 USD, plus equity.
Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.
Benefits: Thinking Machines offers generous health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.