Tomi Mester – A/B Test like a Pro! (online course)
Understand the science and the business logic behind A/B testing — and achieve real, significant, long-term increases in conversion!
Course Curriculum
- Welcome to the course (0:55)
- About me (2:42)
- What’s in the course? How to get the most out of it. (4:15)
- What is A/B testing exactly? (3:01)
- The correlation vs. causation issue (4:47)
- Why do you need A/B testing? (a story and the real reason) (4:23)
- What can you A/B test? (8:52)
- EXTRA: Hand-picked A/B Testing case studies
- ACTION: The A/B tests you’ve run so far — and the A/B tests you want to run in the future
- Important A/B Testing mindset: Conversion Rate Optimization or Research? (3:42)
- Two key questions before ing an A/B Test. (And when is A/B testing a really bad idea?) (6:45)
- When you don’t need to run experiments (6:32)
- When you need to run experiments (5:30)
- A/B test only when you need it (2:09)
- EXTRA: the “can I run this A/B test” checklist
- ACTION: Filter out the bad A/B test ideas from your backlog!
- Statistics is important (1:30)
- Randomization (we all are terrible “random-generators”) (3:01)
- Statistical significance and the certainty of your results (4:37)
- An intuitive way of interpreting the importance of statistical significance (2:02)
- 80%? 95%? 99%? What’s the right confidence level? (6:45)
- Calculating the required sample size before the test (4:57)
- EXTRA: online calculators (statistical significance and sample size)
- ACTION: Review your A/B testing backlog!
- Your questions answered.
- The four steps of executing an A/B test (1:39)
- The Research Framework (14:14)
- Your success metric(s) to evaluate your test (7:36)
- A few words about setting a hypothesis (5:05)
- EXTRA: A/B Testing Hypothesis Form
- ACTION: Run your first research round!
- My favorite A/B testing tools (6:06)
- Tool demo (Google Optimize) (16:01)
- Avoiding website flickering in an A/B test (4:38)
- The change-one-thing-at-a-time myth, multivariate tests, A/B/n tests and unusual audience splits (7:04)
- A few typical mistakes at the implementation phase (3:13)
- EXTRA: The “can I the A/B test?” checklist
- ACTION: Try out the simplest ever A/B test!
- How long should an A/B test run? (2:42)
- Which part of the year/week shouldn’t you run an A/B test? (1:53)
- Typical mistakes while running an A/B test (5:57)
- Don’t lie to yourself! (4:36)
- What to do when a test didn’t perform the way you expected (2:28)
- What to do after evaluating your experiment (6:29)
- ACTION: your own knowledge base!
- The limitations of A/B testing (2:26)
- The art and science of A/B testing (3:15)
- Segmentation (2:37)
- EXTRA: Further reads and videos
- Summary & The right mindset to be successful with A/B testing (2:29)
- Where to go next
- Your questions answered
- Why?
- 1) Traffic allocation: sending visitors to the old/new landing pages (with JavaScript)
- 2) Data collection and visualization (Bash + SQL + Google Data Studio)
- 3) Evaluation: significance calculations (using Python)
- Conclusion
Your Instructor
Tomi Mester is a data analyst and researcher. He worked for Prezi, iZettle and several smaller companies as an analyst/consultant. He’s the author of the Data36 blog where he writes posts and tutorials on a weekly basis about data science, AB-testing, online research and data coding. He’s an O’Reilly author and presenter on conferences like TEDxYouth, Barcelona E-commerce Summit or Data Conf.
Proof Content
Sale Page: https://courses.data36.com/p/ab-test-like-a-data-scientist
Archive: https://archive.ph/wip/Zj0oj
Reviews
There are no reviews yet.