СШАMember of Technical Staff, GPU Compiler
2 нед. назад
285k–315k USD / годСШАLeadOnsite
gpucompilerc++performance optimization
Developing high-performance GPU compiler technologies to optimize and accelerate compute for AI applications.
Обязанности
- We build the fastest GPU compiler in the world. Most compilers have to preserve correctness at every transform, constraining how far they can search, while we prove correctness at the end instead, allowing us to search a far wider space, with agents, with RL, with anything that works and still guarantee the result. It's why we hold #1 on NVIDIA's own kernel benchmark across hundreds of production kernels.
- We're hiring a Member of Technical Staff for GPU Compiler Engineering to build the machine that does the searching.
- You'll be working on the entire pipeline end to end (StableHLO ingestion, MLIR dialects & passes, backend code generation, executable GPU & host binaries). The pipeline already does things worth seeing. Given only a naive attention spec, the search found FlashAttention v4-level performance on a B300 in about 25 minutes and your job will be to make sure that next time, we beat FlashAttention with ease.
- You'll have access to better tooling than anywhere else because we own the stack all the way down to the ISA, allowing us to create kernels that others can't even express. For example, our custom LLVM backend emits cubins directly, without ptxas, letting us work on instruction selection, scheduling, register allocation and stall counts. Our team understands the hardware better than anyone else, to this extent we've built a bit-exact software model of Blackwell's tcgen05 .
- You'll be writing kernels that ship immediately on runs such as pre-training AlphaFold v3 at 3.4× the throughput, post-training robotics models on Trainium or running our custom RL rollout engine on TPU at multi-100B parameter scale.
- You'll build and extend MLIR dialects and passes to optimize training and inference workloads
- You'll work in our LLVM backend, below ptxas, on instruction selection, scheduling, register allocation and direct cubin emission and work on standing up equivalent depth on AMD, TPU and Trainium targets
- You'll expand our search-based compiler infrastructure, including agent- and RL-driven program search and the formal correctness proofs that make aggressive search safe
- You'll implement classic compiler optimizations tuned for large-scale training
- You'll create hybrid codegen paths for cases where direct MLIR lowering isn't practical
- You'll own testing, benchmarking and performance regression systems, including bit-identical hardware models
- You'll work closely with our research team and with customer workloads to find the optimization opportunities that matter
Требования
- Someone with deep experience in compiler infrastructure (LLVM, MLIR or similar)
- Someone with a strong background in GPU architecture and low-level optimization (CUDA, ROCm or similar)
- Someone with hands-on experience with at least one of: PTX/SASS, GCN/RDNA assembly or other GPU ISAs
- Someone familiar with ML compiler stacks (XLA, TVM, Triton, torch.compiler or similar)
- Someone with solid systems programming skills in C++ and/or Rust
- Someone with a proven track record of building production-grade compiler infrastructure
Будет плюсом
- Someone with experience in autotuning or search-based optimization
- Someone with a background in formal verification, proof assistants or SMT solvers
- Someone with experience writing or maintaining an LLVM backend
- Someone with a background in distributed systems or multi-device compilation
- Someone who's contributed to open-source compiler projects
- Someone familiar with large-scale training infrastructure
- Someone with experience in (Stable)HLO
Условия
- Most compiler engineers spend their career inside a constraint they didn't choose, forced to ensure that every transform is correct on it's own. If you've ever wanted to put real search inside a compiler and never had the architectural freedom or the hardware budget to try it, this is the job for you. This is a chance to work at the very bottom of the stack on a compiler that is already the fastest in the world, with a direct and measurable effect on the cost of every large training run we touch.
- We're a small team operating at frontier scale. We pre-trained foundation models on 4,000 AMD GPUs as a team of three, designed and brought up GB300 NVL72 clusters and designed a TOP500 supercomputer.
- We believe that hard problems get solved in person and most of our work happens at our office in San Francisco. We offer relocation assistance and, where possible, we'd like you here as often as possible.
- The base salary range for this full-time position is $285,000-$315,000, plus meaningful equity and benefits.
Другое
- We firmly believe that the future of AI depends on the unglamorous: rethinking and rebuilding the stack, all the way down. From the hardware underneath it to the compiler targeting it and the cloud running it. Right now those three things fight each other and that friction shows up as a tax on every researcher trying to build something ambitious. We're here to axe that tax and make compute faster, cheaper and more available. When we succeed, compute will be portable enough that "which cloud, which chip" stop being something you worry about.
- To achieve this, we are building our Kernel Optimizer, which takes code and finds its fastest possible form for whatever vendor and cluster topology you point it at, automatically, as well as the Model Foundry which manages the runs, makes research easier and moves workloads across clouds and chips as prices and availability ship, instead of leaving you locked into whatever vendor you signed with first.
- We're backed by Susa Ventures, Y Combinator, along with some great funds and angels including Max Mullen and Paul Graham, as well as founders and executives at Neuralink, Notion and AMD. We're looking for researchers, engineers and organizations who agree with the basic premise: you don't get the next leap in AI without a leap in compute first.