БразилияMember of Technical Staff | Inference Platform
3 нед. назад
БразилияLeadRemote
kubernetesautoscalinggputelemetryvectorized operations
Responsible for owning and optimizing inference platform systems, running model inference efficiently on cloud and Kubernetes environments.
Обязанности
- At , every technical IC is a Member of Technical Staff (MTS). The title doesn't put anyone in a silo: you own systems and outcomes, not steps in a function, and you keep building depth in your area. Seniority shows up in your scope, level, and compensation, not in titles.
- In this role, you'll join the Platform team to own where our models execute. Customers consume our models through large batches of millions of records and through real-time APIs, and they make business decisions on every response. You'll run governed model releases reliably and efficiently — in our cloud and on customer-hosted Kubernetes — and make inference fast, predictable, and cheap enough to serve both enterprise and mid-market customers.
- Evolve Sophos, our online and batch inference runtime, built on Kubernetes.
- Run large batch inference on ephemeral jobs, with multi-dimensional admission control (CPU, memory, GPU) through Kueue.
- Build and extend the Sophos controller and its Kubernetes custom resources.
- Optimize each model's inference engine and feature processing, using vectorized, columnar operations.
- Serve graphs and data efficiently from Lance-based storage.
- Own execution of training, post-training, and fine-tuning jobs, in our cloud and in customer dataplanes.
- Drive autoscaling, GPU serving, performance, and cost optimization, with telemetry for every model we run.
- Solve open problems such as deterministic job sizing, checkpointing and recovery for batch runs, per-customer encryption and isolation, resilience to difficult input files, and automatic profiling when a new model is accepted.
Требования
- Experience running model serving or large-scale batch compute on Kubernetes.
- Experience building Kubernetes controllers or operators.
- Skill at profiling and optimizing data-heavy Python pipelines.
- A clear sense of cost: you treat compute efficiency as a product feature.
- Production-quality code and reviews, and a willingness to operate what you build.
Будет плюсом
- Ray, Ray Serve, or KubeRay in production.
- Kueue or other batch scheduling and admission-control systems.
- GPU serving and performance optimization.
- Arrow, Parquet, Lance, or other columnar formats.
- Shipping software to customer-hosted Kubernetes.
- GCP/AWS and GKE/EKS, and financial services or regulated environments.
Другое
- 99.9% serving availability.
- p95/p99 latency for online inference and throughput for batch.
- Cost per prediction and per training job.
- GPU utilization: paid capacity versus capacity actually used.
- Training and batch jobs that finish on time and succeed without manual retries.