СШАPIPELINE - L5 Principal Software Engineer
1 нед. назад
244k–366k USD / годСШАLead
data platformstreamingbatch computestoragelakehousedata governance
Lead the technical direction and delivery of 's data platform including streaming, batch compute, storage, and governance supporting autonomous, agent-driven experiences.
Требования
- Experience: 10+ years building and operating distributed data systems (e.g., Kafka/PubSub, Flink/Spark/Beam, Airflow/Dagster, Iceberg/Delta/Hudi; Snowflake/BigQuery; object storage) with multi‑tenant reliability.
- Technical expertise: Data ingestion/CDC, stream processing, batch orchestration, lakehouse patterns, catalog/lineage, governance, and access controls, measured by freshness, availability, and correctness SLOs.
- AI tools & automation: You apply ML/GenAI to data platforms - semantic catalog search, auto‑docs, data‑quality anomaly detection, SQL/pipeline generation - with human‑in‑the‑loop review and privacy controls.
- Influence & enablement: You land data contracts, connectors, and templates that speed delivery for producers and consumers; you mentor via design docs and pairing.
- AI fluency ( default): You experiment, learn fast, and share AI wins responsibly.
Будет плюсом
- Regional isolation/replication strategies, privacy‑by‑design, and data governance in regulated contexts.
- Adopted paved roads: Producers/consumers are on standard ingestion, processing, and storage paths; schema governance and contracts reduce breakage.
- SLOs & efficiency: ≥99.9% freshness for key domains; measurable cost/TB reductions; production debugging is faster with defined readiness reviews and incident learning.
- AI‑augmented data operations: Semantic discovery and auto‑documentation cover the majority of high‑value datasets; AI‑assisted DQ monitors reduce data incidents on top pipelines by 25–40%; pipeline authoring and review times drop 15–25% with AI in the loop.
Условия
- Massachusetts Applicants: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
- Our salary range reflects the cost of labor across various U.S. geographic markets. The range displayed below reflects the minimum and maximum target salaries for the position across all our US locations. The base salary offered for this position is determined by several factors, including the applicant’s job-related skills, relevant experience, education or training, and work location.
- In addition to base salary, our total compensation package may include participation in the company’s annual cash bonus plan, variable compensation (OTE) for sales and customer success roles, equity, sign-on payments, and a comprehensive range of health, welfare, and wellbeing benefits based on eligibility.
- Your recruiter can provide more details about the specific salary/OTE range for your preferred location during the hiring process.
- Base Pay Range For US Locations:
- $244,000 — $366,000 USD
- This role may require up to 10% travel for purposes such as new hire onboarding, client or partner work if applicable, team meetings, and industry events. Travel is coordinated in advance.
Другое
- At , we value the unique backgrounds, experiences and perspectives each (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying. Want to learn more about life at ? Visit .com/careers to see how we empower creators to own their own destiny.
- Own the technical direction and delivery for 's data platform - streaming, batch compute, storage/lakehouse, and governance - now the foundation for autonomous, agent-driven experiences. 's data platform processes billions of events daily across billions of consumer profiles for hundreds of thousands of brands, and increasingly that data is consumed not just by people but by a growing number of AI agents acting on customers' behalf. The footprint grows on two axes at once, ever‑larger data volumes and a steady stream of new data scenarios (new sources, domains, and consumption patterns) and your systems have to scale on both. You'll design and ship systems that are fast, reliable, cost‑efficient, and safe for autonomous access, creating paved roads for data producers and consumers, human and agent alike. This is an individual‑contributor role (no direct reports); you lead through architecture, code, and influence. Expectations align to Lead/Principal IC behaviors: establishing SLOs, driving technical evolution, and acting as the interface across teams. What You'll Do
- Design and implement core data platform capabilities (e.g., event ingestion/CDC, stream processing, batch orchestration, data lake/warehouse patterns, catalog/lineage, governance, access, and compliance).
- Define and uphold SLOs for data freshness, availability, and correctness; author/run readiness reviews, incident response, and post‑incident learning for your domain.
- Author ADRs/RFCs, land data contracts and schema governance, and standardize connectors and templates that accelerate developer velocity.
- Profile, tune, and right‑size systems for performance and cost; partner with FinOps on unit‑economics guardrails.
- Pair with product teams and analytics/ML to expose the right abstractions and unblock customer value quickly.
- Contribute high‑quality code and reviews; mentor Staff/ Sr. engineers across pillars through example and enablement (not line management).
- Use AI to streamline data workflows, from authoring and testing pipelines to catalog/search and DQ, so analysts, ML, and product teams move faster with confidence.
- Adopted “paved roads” for producers/consumers; ≥99.9% freshness SLO for key domains; measurable cost/TB reductions.
- One or more step‑change improvements (e.g., 2× faster ingest → analytics latency) demonstrated with metrics and post‑incident trend‑down.
- We’re (pronounced clay-vee-oh). We empower creators to own their destiny by making first-party data accessible and actionable like never before. We see limitless potential for the technology we’re developing to nurture personalized experiences in ecommerce and beyond. To reach our goals, we need our own crew of remarkable creators—ambitious and collaborative teammates who stay focused on our north star: delighting our customers. If you’re ready to do the best work of your career, where you’ll be welcomed as your whole self from day one and supported with generous benefits, we hope you’ll join us.
- AI fluency at includes responsible use of AI (including privacy, security, bias awareness, and human-in-the-loop). We provide accommodations as needed.
- By participating in ’s interview process, you acknowledge that you have read, understood, and will adhere to our Guidelines for using AI in the interview Process . For more information about how we process your personal data, see our Job Applicant Privacy Notice .
- By clicking "Submit Application" you consent to processing your Personal Data in accordance with our Job Applicant Privacy Notice. If you do not wish for to process your Personal Data, please do not submit an application. You can find our Job Applicant Privacy Notice here and here (FR).