Kake
Senior Software Engineer - LLM Trainer
1 мес. назад
Страна неизвестнаSeniorRemoteC1
pythonjavascripttypescriptgojavac++sqlgit
Senior Software Engineer at Kake contributing to AI training data development and evaluation for AI agents and LLMs, working remotely in a fast-paced environment.
About the company
- Kake is a technology company building modern web applications for consumer products.
Responsibilities
- Create and review coding tasks based on real-world software engineering scenarios, including debugging, refactoring, code generation, API usage, automated tests, performance, security, and edge cases.
- Write high-quality reference solutions that are correct, clear, testable, and aligned with task requirements.
- Evaluate AI-generated code and responses using structured rubrics, assessing correctness, clarity, security, performance, maintainability, and instruction-following.
- Compare multiple model responses, select the strongest answer, and justify your decision with clear technical reasoning.
- Identify bugs, hallucinated APIs, missing edge cases, weak explanations, and poor engineering decisions in AI-generated outputs.
- Work with terminal-based development workflows when needed, including running tests, debugging issues, managing dependencies, and navigating repositories.
- Follow detailed guidelines consistently and participate in calibration activities to ensure high-quality, reliable evaluations.
Requirements
- 5+ years of professional software engineering experience in a backend, fullstack, or systems role.
- Strong proficiency in at least one core programming language, ideally python, javascript/typescript, go, java, c++, or sql.
- Hands-on experience with Terminal-Bench, with the ability to evaluate AI agent performance on terminal-based tasks including compiling code, running tests, managing environments, and completing multi-step software engineering workflows.
- Comfortable working with git, command line/terminal, and common development workflows.
- Ability to evaluate code critically - not only whether it works, but whether it is well-designed, secure, and maintainable.
- Prior experience in AI data production, RLHF, data annotation, or LLM evaluation projects.
- Excellent written and verbal communication skills in English.
- Ability to work independently in a remote, asynchronous, fast-paced environment.
- High attention to detail and the ability to follow complex, rubric-based guidelines consistently
Nice to have
- Experience with python-heavy workflows, automated testing frameworks, docker, linux, bash, or containerized environments.
- Experience with repo-level code reasoning, large codebases, or open-source contributions.
- Background in backend systems, data engineering, devops, infrastructure, security, or large codebase.
Conditions
- Remote work with US Timezone Overlap: PST (GMT -8).
- Due to the high volume of applications, only shortlisted candidates will be contacted.