СШАSoftware Engineer - Ingestion Execution Platform
вчера
160k–230k USD / годСШАSenior
distributed systemscloudpipeline orchestrationmulti-tenant scheduling
Develop and maintain the ingestion execution platform powering data ingestion into 's AI Data Cloud at scale.
Обязанности
- Design and ship core platform capabilities. Work on areas such as multi-tenant isolation, fair scheduling, ownership and placement, durable queues, source monitoring, or compute allocation. You own the work from design through production.
- Strengthen isolation and fairness. Build and evolve the primitives that let many products and pipelines share one platform safely: quotas, backpressure, resource accounting, and intelligent placement.
- Raise reliability and observability. Improve availability and latency for pipeline health, ingest APIs, and compute allocation. Instrument the system so behavior is visible and actionable.
- Build a clean platform surface. Give product teams a consistent way to express pipelines and processing steps, so they can ship on shared primitives rather than special-casing the stack.
- Scale the control plane. Design for more concurrent pipelines, higher ingest volume, and efficient use of compute and metadata as customer usage grows.
- Operate what you ship. This platform sits on the critical path for production ingest. You will participate in production support, improve observability, and automate operational work. Mentorship of junior engineers is part of the job. You will also work closely with engineers on larger platform bets.
Требования
- 3–6+ years building and operating large-scale distributed systems or cloud infrastructure in production. You have shipped something hard that other teams depend on, not only application features.
- Track record of end-to-end ownership: you took an ambiguous, cross-cutting problem, wrote the design, landed the code, and lived with it in production. Comfortable operating above your level, influencing adjacent teams and helping sequence larger bets without waiting to be told what to do.
- Strong proficiency in Java (this codebase is Java-first). C++ or systems-level experience is a plus. You write concurrent, failure-aware code and you care about the invariants.
- Experience in at least one of: multi-tenant control planes, durable queues and at-least-once processing, distributed locking / leases / membership, cluster scheduling, or streaming/ingest execution engines.
- Instinct for correctness under failure. You think carefully about crashes, retries, redelivery, and exclusive ownership.
- Clear written and verbal communication. You can write a clear design for engineers on neighboring product teams, and you can hold a technical argument without ego.
- BS, MS, or PhD in Computer Science, Computer Engineering, or equivalent practical experience.
Будет плюсом
- Experience building internal platforms used by multiple product teams (orchestration, job/runtime frameworks, multi-tenant schedulers).
- Background in multi-tenant isolation and fairness: resource accounting, request throttling, quota systems, or load-aware placement.
- Hands-on with transactional or consensus-backed metadata stores (for example etcd, ZooKeeper, or Spanner-style systems).
- Familiarity with open table formats (Apache Iceberg, Delta Lake), Kafka-style streaming, or warehouse scheduling.
- Performance work at scale: tail latency, memory and thread accounting, I/O batching, or cost-per-unit efficiency.
- You have used AI as a serious engineering collaborator (design review, debugging, test generation) and can show how it made you faster without lowering the bar.
- is growing fast, and we’re scaling our team to help enable and accelerate our growth. We are looking for people who share our values, challenge ordinary thinking, and push the pace of innovation while building a future for themselves and .
- How do you want to make your impact?
- For jobs located in the United States, please visit the job posting on the Careers Site for salary and benefits information: careers..com
Другое
- At , we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At , your role isn't just to execute a function, but to help redefine the future of how work gets done.
- ’s ingestion products land customer data into the AI Data Cloud at petabytes scale, on a path to much larger scale. Those products run on a shared, always-on ingestion execution platform : a multi-tenant control plane that owns pipeline lifecycle, durable work queues, single-owner execution, step orchestration across control plane and compute, multi-tenant scheduling, source monitoring, and cluster topology.
- You will build the substrate other ingestion teams ship on. Reliability, fairness, and efficiency at this layer multiply across ingestion products.
- As a Software Engineer , you will own substantial platform work end to end: take a hard distributed-systems problem from design through production, make neighboring teams more successful, and raise the bar on correctness, isolation, and operability.