USAPrincipal Site Reliability Engineer, Machine Learning
1 мес. назад
USALeadOnsite
awseksdatabrickscloudwatchdatadogautoscalinggpu schedulingcapacity planning
Principal Site Reliability Engineer focusing on managing and optimizing machine learning workloads infrastructure.
Обязанности
- Use independent judgment and discretion to own SLOs, error budgets, and the operational health of Ray clusters running on AWS EKS and Databricks workloads on AWS EC2 across multiple accounts and regions
- Maintain the observability of uptime, availability, and scalability of EKS Ray and Databricks workloads using CloudWatch and Datadog, including defining alerting that maps to SLOs
- Operate and tune EKS Ray workloads at scale including autoscaling, GPU scheduling, and automated failure recovery
- Manage Databricks on AWS including workspace administration, cluster policies, Unity Catalog, job orchestration, and IAM Roles and Policies
- Maintain ongoing cost visibility, cost optimization, and capacity planning across EC2 and EKS workloads, including through the use of On Demand Capacity Reservations and Spot lifecycle
- Perform ongoing maintenance of the underlying EC2 and EKS infrastructure, including regular security updates and operating system upgrades
- Codify everything as infrastructure-as-code using Terraform and CI/CD pipelines, enabling updates through Pull Requests with approval workflows, while also automating maintenance tasks to reduce toil
- Lead incident response for Data Science and Machine Learning platform outages, run blameless postmortems, and drive systemic remediation, including participating in an on-call rotation
- Complete any additional tasks as they arise
Требования
- Bachelor’s degree or equivalent years of experience and/or certification in a related field
- 7+ years working in Site Reliability Engineering or Information Technology
- Design and document systems, including writing and reviewing code, to automate away problems within your team’s domain
- Intermediate to expert experience deploying and maintaining AWS services such as EC2, ECS, EKS, SQS, Lambda, Dynamo, RDS/Aurora, S3, and IAM
- Intermediate to expert experience monitoring services and applications using tools such as CloudWatch Metrics, CloudWatch Logs, and Datadog, including defining and configuring alerts and SLO reports
- Intermediate to expert experience maintaining the uptime and scalability of AWS compute services used for Machine Learning and Data Science workloads, specifically EC2 and EKS
- Intermediate to expert coding skills in at least one programming language; we work primarily in Python
- Intermediate to expert experience using Infrastructure as Code platforms and CI/CD pipelines, specifically Terraform, to manage AWS infrastructure and services
- In-depth knowledge & experience with Linux operating systems (Amazon Linux, Ubuntu) on EC2 and Docker / Kubernetes
- Experience with leading projects in system design, architecture changes, and technology selection
Условия
- Fair and competitive salary based on skills and experience, and annual performance bonus
- Equity may be awarded in the form of Restricted Stock Units (RSUs)
- Medical, Dental, Vision and Life Insurance, matching 401k, short-term & long-term disability and parental leave
- Unlimited Paid Time Off including vacation, sick days & public holidays
- Flexible scheduling and work from home policy depending on role and responsibilities
- The base salary range for this position is: $142,000 to $177,600. This range is specifically for Cambridge, MA
- Work on a mission with real impact: crashes prevented, injuries avoided, lives protected around the world
- Join an industry leader — 65 million drivers protected, powering 140+ programs across 25 countries
- Recognized innovator in mobility AI, earning top honors including the TIME Industry Leader in AI, a Gold Edison Award, and the Artificial Intelligence Excellence Award for AI for Social Good. CMT is also Great Place to Work Certified
- Be part of the team inventing the future of mobility and road safety
- Move fast, own outcomes, do work that matters
- High ownership, small teams, and direct access to leadership — no layers between your work and its impact
- Unlimited PTO, flexible scheduling, competitive salary, annual performance bonus, RSUs, and full benefits including medical, dental, vision, and 401k match
- Summer Fridays provide team members with half days to recharge
- Join one of our employee resource groups: Black, AAPI, LGBTQIA+, Women, Book Club, and Health & Wellness
- Comprehensive wellness, education, and employee assistance programs
Другое
- CMT is looking for a Principal Site Reliability Engineer I, Machine Learning to help us change the world. CMT has helped protect over 65 million drivers and prevent over 126,000 crashes worldwide. We build AI to solve some of the most difficult challenges in mobility — understanding and reducing risk, detecting crashes, and getting people life-saving help. The problems are hard. The impact is real. No matter your role, your work will matter at CMT.
- CMT is looking for a collaborative, customer-committed, and creative SRE team member who wants to join us in making roads safer by making drivers better!
- At CMT, we believe the best ideas come from a mix of backgrounds and perspectives.
- We are an equal-opportunity employer committed to creating a workplace and culture where everyone feels valued, respected, and empowered to bring their unique talents and perspectives. Diversity is essential to our success, and we actively seek candidates from all backgrounds to join our growing team.
- We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status or disability state. CMT is headquartered in Cambridge, MA. To learn more, visit and follow us on Instagram @
- Cambridge Mobile Telematics (CMT) is the world’s largest telematics and AI company for safer mobility. Its mission is to make the world’s roads and drivers safer. The company’s AI-driven platform, DriveWell Fusion®, proactively identifies and reduces driving risk, leading to fewer crashes and injuries. To date, CMT’s technology has helped prevent over 126,000 crashes worldwide. CMT enables partners to measure risk, detect crashes, provide life-saving assistance, and streamline claims. Headquartered in Cambridge, MA, CMT operates globally with offices in Budapest, Hungary; Chennai, India; Seattle, Washington; Tokyo, Japan; and Zagreb, Croatia. Learn more at