ML Model Development & Deployment Project Manager Simulation — Netflix

The scenario

Netflix — the world's largest streaming entertainment service with 230M+ paid subscribers across 190 countries — is building a production churn prediction model to identify subscribers at risk of cancellation before they leave. The company loses approximately 2-3% of subscribers per month to voluntary churn, representing $500M+ in annual revenue leakage. Every 0.1% improvement in churn prevention is worth approximately $50M annually. Netflix already has world-class recommendation algorithms and personalization systems. But churn prediction is a different problem: it requires modeling not just what people watch, but why they stop watching — and more importantly, why they decide that stopping is worth $15.49/month. The signals are behavioral (declining engagement, fewer sessions, shorter watch times), contextual (competitive launches, price increases, account sharing crackdowns), and personal (life events, financial pressure, household changes). You have been brought in as the PM for a small, elite data science team tasked with building, validating, and deploying a production churn model. The challenge isn't the technology — Netflix has world-class ML infrastructure. The challenge is the problem itself: churn is a human decision influenced by factors that no model can fully capture. Your job is to deliver a model that is accurate enough to act on, explainable enough for business teams to trust, and fair enough to pass an ethics review — while managing executives who expect a crystal ball.

What you'll do as the project manager

• →Build and deploy a production churn prediction model scoring Netflix's 230M+ subscriber base daily with a minimum AUC-ROC of 0.82 and precision at 80% recall of 0.65+
• →Deliver subscriber-level churn risk scores to Marketing (retention campaigns), Product (in-app interventions), and Content (personalized recommendation adjustments) within 24 hours of score generation
• →Provide explainable churn drivers for each at-risk subscriber — not just a score, but the top 3 reasons the model predicts churn — enabling targeted intervention strategies
• →Pass Netflix's internal AI Ethics review with no demographic bias exceeding 5% disparity in false positive rates across subscriber segments
• →Reduce monthly voluntary churn rate by 0.3% within 90 days of deployment through model-informed retention interventions — representing approximately $150M in annual retained revenue

Project management skills you'll build

The challenges you'll navigate

• •Churn definition ambiguity — three different historical definitions mean training labels may be inconsistent, leading to a model that predicts a moving target
• •Executive accuracy expectations — the Product VP promised the CMO '95% churn prediction accuracy' in a board deck. No churn model in the industry achieves this. Managing the gap between promise and reality is the project's central challenge
• •Small team, no buffer — with 5 people, losing any team member (illness, departure, competing priority) critically impacts delivery. There is no bench
• •ML serving pipeline backlog — the ML engineering team has a 6-week backlog. If the serving pipeline isn't prioritized, the model can be built but not deployed by September 15
• •Fairness risk — churn patterns may correlate with demographic factors (income, geography, age) that create bias in model predictions. The ethics review could require model redesign

Technology & stakeholders

You'll manage 7 stakeholders, including Eunice Kim (VP Data Science & Analytics), Raj Chandrasekaran (VP Product — Subscriber Growth & Retention), Sanjay Gupta (Senior Data Scientist), and more.

What you'll walk away with

A verified, shareable record of a completed enterprise project — plus the PMO deliverables you produced along the way (charter, project plan, SteerCo deck, closure document). It's real, demonstrable project management experience you can put on your resume and speak to in interviews.

Frequently asked questions

Related simulations