СШАHarness Lead
6 дн. назад
140k–220k USD / годСШАLead
data-driven featuresdesignproduction deploymentbenchmarking
Lead design and implementation of research agent harness architecture and post-training cycles to improve model quality, speed, and cost.
Требования
- At , we strongly believe that successful people possess two traits: (1) agency, (2) ruthless prioritization. You must be able to will things into existence. This requires an incredible drive to make things happen quickly. You must also know the difference between what is urgent and what is important, and when to prioritize what.
- For this role specifically, we are looking for the right taste for building a harness and training a model within it. Taste comes from experience; we would love to see your work that demonstrates this. We are also looking for work that demonstrates obsession; have you spent an insane amount of time or effort building or improving something?
Другое
- We build and maintain our own harness. Today, it delivers quality through skills, automatic data lineage, and a curated SDK of internal tools. The harness needs to help the model find evidence, use it correctly, and search for information strategically. The design of harness features is guided by production traces with a single goal: quality.
- We think of this infrastructure as scaffolding. Our long-term goal is to post-train recurring harness capabilities into the coding model, then remove scaffolding where evaluations show it is no longer needed. This cycle of adding features to the harness and then removing them after “training” them into the model lets us continually improve the agent. We run this cycle monthly.
- Your goal is to help us move faster and reduce bloat. You'll help decide what belongs in the harness, when to add tools, what the model should learn, and how we can tell when a change actually improves the research. Concretely:
- Harness architecture and execution. You will own the design for our research agent harness. This involves coming up with data-driven features to improve quality and shipping them into production for users.
- The post-training cycle. You are responsible for the design of post-training experiments to teach the model to search better while retiring unnecessary parts of the harness.
- Benchmarks. You will design evals from production traces to measure our agents’ performance across: (1) quality, (2) speed, (3) cost.
- Co-optimization for search. This involves working closely with the Search Lead to distinguish regressions/gaps from poor retrieval or poor tool use, and improve the system at the layer where it matters.
- All ambitious ideas come with a clock. We are in a rush to build a generational company. If you don’t feel this rush, this may not be the best fit for you.