Friendliai
Solutions Architect - AI Inference Specialist
3mo ago
USASeniorRemotedockerkubernetesterraformhelmci/cdaws
Architect and implement scalable AI inference deployment solutions and collaborate with customers to integrate and optimize AI workloads.
About the company
- FriendliAI is building the next-generation AI inference platform that accelerates the deployment of large language and multimodal models with unmatched performance and efficiency. Our infrastructure powers high-throughput, low-latency workloads for global organizations and integrates directly with Hugging Face, providing instant access to over 500,000 open-source models. We are on a mission to deliver the world’s best platform for AI inference.
Responsibilities
- FriendliAI is seeking a Solution Architect to assist enterprises in deploying, scaling, and operating generative and agentic AI workloads on FriendliAI infrastructure. You will work directly with customers to solve and implement production-grade applications using our products, such as Serverless Endpoints, Dedicated Endpoints, or Container.
- Friendli Container is our service that allows customers to download our inference engine as Docker images and deploy it in their chosen environment, such as private clouds or on-premises. Our Friendli Container can be adopted directly to AWS EKS clusters using our EKS add-on product.
- You will work directly on our customers’ projects, collaborating with their engineering teams to solve AI inference challenges like scaling, orchestration, and monitoring. This is a hands-on, customer-embedded role. If you have worked in DevOps, platform engineering, or SRE for AI applications, this is your ideal position.
- Design and implement large-scale deployment architectures for LLM and multimodal inference
- Deploy and manage containerized workloads across Kubernetes clusters
- Diagnose production issues, such as performance bottlenecks, and implement temporary fixes as needed
- Collaborate with customers’ DevOps teams to integrate FriendliAI’s infrastructure into their CI/CD workflows
- Develop scripts, Helm charts, and Terraform modules that simplify repeated deployments
- Contribute field insights to shape our platform reliability, observability, and scaling strategies
- Lead workshops, technical sessions, or webinars to help customers master infrastructure best practices.
Requirements
- 3+ years of experience in cloud infrastructure, DevOps, or reliability engineering
- Bachelor’s or Master's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent
- Proficiency with Kubernetes, Docker, Terraform, and Helm
- Strong foundation in distributed systems, networking, and performance tuning
- Experience with GPU-based computing and generative AI model serving workloads
- Strong technical background in backend systems or AI tooling
- Experience operating workloads on AWS, GCP, or OCI
- Excellent problem-solving and debugging skills in real-world environments
Nice to have
- Experience deploying large models (LLMs, diffusion models) on GPUs or clusters
- Familiarity with inference frameworks (Triton, vLLM, TensorRT, DeepSpeed-Inference)
- Familiarity with observability stacks (Prometheus, Grafana, Loki, ELK, OTEL)
- Understanding of networking security and compliance frameworks (e.g., SOC 2)
- Experience supporting on-prem or hybrid-cloud deployments
Conditions
- A front-row seat to the generative AI infrastructure revolution
- Competitive compensation and benefits package
- Daily lunch and dinner provided; unlimited snacks and beverages
- Health check-up and top-tier hardware support
- Flexible working hours and a highly collaborative environment