A/B test performed on Udacity's student database to improve student experiment i.e. Reduce the number of students who cancel out of their first month free-trial without losing net-conversion(Revenue related metric).
The Process involved :
- Identifying and choosing invariants
- Identifying and choosing evaluation metrics.
- Calculating Expected , empirical standard deviation in the dataset.
- Choosing size.
- Choosing power.
- Duration vs Exposure trade-offs.
- Experiment Analysis :
-
- Sanity checks
-
- Result analysis
- Recommendation
Decrease the number of students who cancel after they enroll in their nanodegree without reducing net-conversion. (Improve retention).
Full report in Udacity Report.pdf
Project summary and instructions : Instruction