Senior Software Engineer, Cortex Quality

2 нед. назад

200k–270k USD / yearUSASenior

pythondata engineeringaibenchmarking

Senior software engineer responsible for end-to-end quality and efficiency of Cortex Code, 's coding agent for data.

Обязанности

This is a measurement-first role that owns the quality and efficiency of Cortex Code end to end: how good the agent is, how much it costs to run, and how reliably it behaves in production. You will take agents from research capability to real, measurable user value — turning fuzzy “the agent feels worse” signals into hard metrics, running the experiments that move them, and shipping the changes that stick. You will work on a small, high-powered modeling and infrastructure team where your work reaches every developer building on .
Take agents from prototype to production: design and refine agent behaviors for real coding and data-engineering workflows, and make them reliable enough to depend on.
Own agent quality end to end: build eval harnesses, diagnose failure modes from real agent trajectories, and run experiments that hillclimb the metrics that matter.
Drive down cost and latency without regressing quality: prompt caching, context compaction, tool-result offloading, and cheaper model routing for sub-tasks. At scale, efficiency is user value.
Debug production failures and systematically increase robustness: close the loop from a customer’s broken run back to a fix and a regression test.
Build the data pipelines that feed real-world task insights back into evals and modeling.
Onboard and bake off frontier models: measure their strengths and failure modes, and decide where each belongs in the product.
Partner with product and infra: shape user-facing agent behavior, and set the metrics and standards that define a successfully completed complex task.

Требования

Bachelor’s degree in Computer Science, Engineering, Statistics, or a related field. Master’s or higher preferred but not a requirement.
6+ years of experience shipping software in production, including AI/LLM features.
Fluency in at least one of Python, TypeScript, or Go, and willingness to work across all three
A measurement-first, systems-thinking instinct: you optimize for user outcomes over isolated metrics, and you can design an eval that is not fooling you (sampling, ground-truth quality, leakage, noise).
Comfort debugging complex, unpredictable, real-world failures.
Strong communication skills: you can make a quality or cost result legible to engineers, product, and leadership, and collaborate effectively in a team environment.

Будет плюсом

Deep hands-on experience with agentic coding tools and real intuition for model strengths, failure modes, and prompting limits.
Prior work on eval harnesses, LLM observability, or safety/guardrails in production.
Background in data engineering, data modeling, analytics, retrieval/RAG, or semantic layers, which is highly relevant for data-centric coding agents.
Experience working with large-scale datasets or production system logs.

Другое

At , we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At , your role isn't just to execute a function, but to help redefine the future of how work gets done.
Cortex Code is ’s coding agent for building with data. It ships inside the platform that thousands of the world’s largest enterprises — including a large share of the Forbes Global 2000 — run their data on, which means the quality of this agent is felt by the data teams behind a meaningful slice of the global economy. We are taking coding agents from impressive demos to tools Data Science and Engineering teams depend on every day, and we hold them to a rigorous, public bar: see our data engineering agent benchmark .
Are a power user of modern coding agents and want to turn that intuition into systematic measurement and improvement.
Have built and owned complex systems — pipelines, orchestration, or software with substantial state, branching logic, and operational requirements.
Thrive in high-intensity environments with short feedback loops and high standards for rigor.
Take problems to completion independently: you don’t stop at a prototype; you care about production reliability and clear metrics.
Are genuinely bothered by numbers that do not reconcile.
is growing fast, and we’re scaling our team to help enable and accelerate our growth. We are looking for people who share our values, challenge ordinary thinking, and push the pace of innovation while building a future for themselves and .
How do you want to make your impact?
For jobs located in the United States, please visit the job posting on the Careers Site for salary and benefits information: careers..com