MEDvidi
AI Product Analyst
2d ago
USAMiddleRemoteB2sqlpythonllmai quality metricsmonitoringdashboardconversational ai
MEDvidi is a US-based AI-powered mental healthcare platform. The AI Product Analyst will evaluate, monitor, and improve Voice AI Agents that interact with patients to automate parts of the patient journey.
About the company
- MEDvidi is a US-based AI-powered mental healthcare platform helping patients across 30+ states receive treatment for ADHD, anxiety, depression, and other mental health conditions.
About the product
- Voice AI Agents and conversational AI systems that interact directly with patients and automate critical parts of the patient journey.
Responsibilities
- Analytics for Voice AI Agents and conversational AI systems.
- Investigation of hallucinations, inconsistencies, silent failures, and quality degradation.
- Analysis of call transcripts, conversation logs, AI outputs, and operational datasets.
- AI quality evaluation and reliability measurement.
- Building monitoring frameworks and dashboards for AI performance.
- Detecting quality shifts, anomalies, and emerging failure modes.
- Measuring AI impact on patient experience, operations, and business outcomes.
- Partnering with Product, Engineering, QA, and Operations teams.
Requirements
- 3+ years of experience in Product Analytics, Data Analytics, or Operational Analytics.
- Strong SQL skills.
- Python for analytical investigations and working with large-scale text datasets.
- Understanding of LLM limitations, hallucinations, prompt sensitivity, and AI failure modes.
- Experience building monitoring systems, analytical frameworks, or dashboards.
- Strong communication skills and ability to translate insights into product decisions.
- Fluent Russian and English B2+.
- CET-overlapping working hours.
Nice to have
- Experience with conversational AI or AI agents.
- Experience evaluating LLM outputs.
- Experience with AI quality metrics and evaluation frameworks.
- Experience working with ML pipelines or AI-powered products.
Conditions
- Work on one of the most visible AI products inside the company.
- Analyze real Voice AI systems used by patients every day.
- Help shape the future of AI in healthcare.
- High ownership and direct impact on product decisions.
- Fully remote long-term B2B collaboration.
- Health insurance, wellness compensation, paid vacation & sick leave.
- Personalized English lessons via Preply.
How to apply
- Apply here: https://medvidi.com/careers/co/poland-b2b/B3.D63/ai-product-analyst-ai-receptionist-team-voice-ai-agents/all?coref=1.10.s93_B66
Found in@montenegro_it_jobs