Search Results for: a/b test

Determine optimal sample sizes for business value in A/B testing, by Chris Said

Determine optimal sample sizes for business value in A/B testing, by Chris Said

A/B testing is a method of comparing two versions of some thing against each other to determine which is better. A/B tests are often mentioned in e-commerce contexts, where the things we are comparing are web pages. Business leaders and data scientists alike face a difficult trade-off when running A/B tests: How big should the … Continue reading Determine optimal sample sizes for business value in A/B testing, by Chris Said

Helpful resources for A/B testing

Helpful resources for A/B testing

Brandon Rohrer — (former) data scientist at Microsoft, iRobot, and Facebook — asked his network on Twitter and LinkedIn to share their favorite resources on A/B testing. It produced a nice list, which I summarized below. The order is somewhat arbitrary, and somewhat based on my personal appreciation of the resources. Course: A/B-testing by Google … Continue reading Helpful resources for A/B testing

A/B Testing a New Look

A/B Testing a New Look

This WordPress blogger I came across — let’s call him “John” for now — has a very peculiar way of testing out his looks. Using dating-apps like Tinder, John conducted A/B-tests to find out whether people would prefer him romantically with or without a beard.  Via a proper experimental setup, John found out that bearded John receives much … Continue reading A/B Testing a New Look

12 Guidelines for Effective A/B Testing

12 Guidelines for Effective A/B Testing

I wrote about Emily Robinson and her A/B testing activities at Etsy before, but now she’s back with a great new blog full of practical advice: Emily provides 12 guidelines for A/B testing that help to setup effective experiments and mitigate data-driven but erroneous conclusions: Have one key metric for your experiment. Use that key … Continue reading 12 Guidelines for Effective A/B Testing

A/B testing and Statistics at Etsy, by Emily Robinson

A/B testing and Statistics at Etsy, by Emily Robinson

Generating numbers is easy; generating numbers you should trust is hard! Emily Robinson is a data scientist at Etsy, an e-commerce website for handmade and vintage products. In the #rstats community, Emily is nearly as famous as her brother David Robinson, whom we know from the tidytext R-package. Like any large tech company, Etsy relies heavily … Continue reading A/B testing and Statistics at Etsy, by Emily Robinson

Multi-Armed Bandits: The Smart Alternative for A/B Testing

Just as humans, computers learn by experience.The purpose of A/B testing is often to collect data to decide whether intervention A or B is better. As such, we provide one group with intervention A whereas another group receives intervention B. With the data of these two groups coming in, the computer can statistically estimate which … Continue reading Multi-Armed Bandits: The Smart Alternative for A/B Testing