КанадаSDET Contractor
3 нед. назад
51.61–77.42 CAD / часКанадаSenior
test automation
Experienced SDET for automated testing strategies of Agentic AI systems and backend services.
О компании
- isn’t just the Best Place to Play — it’s also the Best Place to Work. (SIE) is the company behind the PlayStation brand. As a subsidiary of Sony Group Corporation, we’re part of a proud legacy of innovation and excellence. SIE is a dynamic technology company, delivering cutting-edge hardware and network services to more than 100 million people and an entertainment leader, home to some of the most beloved and recognizable intellectual properties (IP) in the world. Our role at SIE is to create and nurture the experiences under the PlayStation brand, a name synonymous with entertainment excellence and creativity.
- SDET Contractor
- We are looking for an experienced Software Development Engineer in Test (SDET) to help ensure the quality, reliability, and resilience of Agentic AI systems and backend services. This role will design and implement automated testing strategies across AI/LLM-driven applications, APIs, microservices, and distributed systems.
- The ideal candidate brings strong Quality Engineering and test automation fundamentals, with experience testing Generative AI, LLM-based, or Agentic AI systems. They should be comfortable testing complex systems where behavior may be non-deterministic and traditional pass/fail testing alone may not be sufficient.
Обязанности
- Develop and execute comprehensive QE strategies and test plans for Agentic AI systems, backend services, APIs, and service integrations.
- Design and maintain scalable automated test frameworks across API, functional, integration, contract, regression, and end-to-end testing.
- Develop testing approaches for non-deterministic AI behavior and establish meaningful evaluation criteria.
- Identify quality risks early and apply risk-based, shift-left testing practices throughout the development lifecycle.
- Integrate automated testing into CI/CD pipelines and establish effective quality gates.
- Partner with Software Engineers, Product Managers, and other QE teams to improve testability, observability, and overall product quality.
- Analyze defects and production issues to identify root causes, testing gaps, and opportunities for continuous QE improvement.
- Establish quality metrics and provide visibility into test coverage, reliability, defects, and release readiness.
- Test Agentic AI workflows, including multi-step execution, tool selection and invocation, task completion, and failure handling.
- Validate AI-generated responses using appropriate measures for accuracy, relevance, groundedness, consistency, and adherence to expected behavior.
- Test AI guardrails, unexpected inputs, and failure scenarios, including incorrect tool usage and prompt-based attacks.
- Validate interactions between AI agents, APIs, tools, data sources, and downstream services.
- Develop reusable automation, test data, and evaluation scenarios for repeatable AI and services testing.
- Validate API contracts, schemas, authentication, business rules, error handling, and service integrations.
- Investigate failures across distributed systems and distinguish application, integration, environmental, automation, and AI-related issues.
- Participate in defect triage, root-cause analysis, regression testing, and release-readiness activities.
Требования
- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
- 4+ years of experience in Software Development Engineering in Test, Quality Engineering, Software Engineering, or a related discipline.
- Strong experience testing REST APIs, backend services, microservices, or distributed systems.
- Hands-on experience developing automated tests using Python, Java, JavaScript, TypeScript, or a similar programming language.
- Experience with functional, integration, regression, contract, and end-to-end testing.
- Experience integrating automated tests into CI/CD pipelines.
- Strong analytical, debugging, and root-cause-analysis skills.
- Experience working in an Agile development environment and collaborating across engineering teams.
Будет плюсом
- Experience testing Agentic AI workflows, including planning, multi-step execution, tool calling, memory, or state management.
- Experience with AI evaluation techniques such as evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, or hallucination detection.
- Experience with AI/LLM evaluation or observability tools such as LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar technologies.
- Experience testing RAG systems or AI agents interacting with APIs, databases, or external tools.
- Experience with contract testing tools such as Pact and API specifications such as OpenAPI/Swagger.
- Experience with event-driven architectures, Kafka, Docker, Kubernetes, or cloud platforms such as AWS.
- Experience with performance, scalability, reliability, or resiliency testing.
- Experience establishing QE metrics, quality gates, or cross-team testing standards.
- We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond.
- We sincerely appreciate the time and effort you spent in contacting us and we thank you for your interest in PlayStation.
Условия
- At SIE, we consider several factors when setting each role’s base pay range, including the competitive benchmarking data for the market and geographic location. Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job-related factors which may include knowledge, skills, experience, and location.
- The estimated Ontario base pay range for this role is listed below.
- $51.61 — $77.42 CAD
- Please note, conducts background checks at the offer stage for all new employees (which may include criminal background checks for some roles) and will need to process personal information to support these checks.
- Please refer to our Candidate Privacy Notice for more information about what personal information we collect, how we use it, who we share it with, and your data protection rights.
Другое
- Strong understanding of software testing methodologies and Quality Engineering practices.
- Strong API and services testing experience, including REST, JSON, schemas, authentication, and error handling.
- Experience designing and maintaining automated test frameworks.
- Experience with Git and CI/CD development workflows.
- Ability to develop risk-based test strategies and identify appropriate testing layers.
- Experience testing distributed systems and service-to-service integrations.
- Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems.
- Understanding of testing non-deterministic AI behavior and evaluating outputs beyond exact-match assertions.