Not All Steam Negative Reviews Are Equal
Predicting Player Engagement and Identifying At-Risk Players on Steam
Negative reviews do not necessarily mean players will stop playing. We combined player behavior with NLP-derived complaint signals to predict observed post-review engagement and identify which negative reviewers may require different product or support responses.
Final Presentation
Key Results
- 2023 Test ROC-AUC
- 0.824
- Observed engagement in the top predicted-engagement decile
- 88.7%
- Lift vs. random selection
- 2.38×
LightGBM with structured and NLP features performed best. Complaint signals added predictive value beyond structured player features, while the final model maintained strong ranking performance on newer 2023 reviews.
The engagement target is derived from historical playtime snapshots and should not be interpreted as confirmed activity exactly 30 days after a review.
The model also showed calibration drift on newer data, indicating that periodic recalibration would be necessary in deployment.
Four Player Profiles
Hardcore 36.06%
High prior playtime · High predicted engagement
Dropout 6.59%
High prior playtime · Low predicted engagement
New Blood 8.94%
Low prior playtime · High predicted engagement
Nobody 48.41%
Low prior playtime · Low predicted engagement
These profiles help distinguish players who may require different product or support responses. They are relative engagement profiles, not confirmed churn labels.