A/B Testing is the art of learning about your users with experiments: show half of them "A", show the other half "B", and measure the response in each group.
How many users do you need to run an A/B Test? If you have too few users in the experiment, you'll never see a difference between the two groups. If you have too many enrolled, the experiment will never end.
That's where A/B Buddy comes in: A/B Buddy helps you perform a statistically rigorous sample size calculation, with the slide of a finger, on your phone or on your iPad.
Let's say your current conversion rate is 10%, and you want to see if a proposed change from the design team can boost that number up to 12% -- that is, a 20% lift in the conversion rate. Simply pop open A/B Buddy, set the Baseline conversion rate to 10%, and the Minimum Detectable Effect to 20%, and your answer is waiting for you at the top of the screen.
A/B Buddy includes a Share button, which links to a web page displaying your sample size calculation. Your numbers, however, remain 100% private -- all the information is stored in the URL fragment, and is never transmitted to any servers.
By default, A/B Buddy uses the most common statistical thresholds: 5% for the significance level (false-positive rate), and 80% for the power level (ability to detect the Minimum Detectable Effect). Power users can adjust these values after unlocking the Pro Features in-app purchase. But most users will find the defaults adequate for their needs.
Questions? Suggestions? Ideas? Feedback is always welcome at firstname.lastname@example.org
- Evan Miller
- 8.8 MB
- Requires iOS 10.0 or later. Compatible with iPhone 5s, iPhone 6, iPhone 6 Plus, iPhone 6s, iPhone 6s Plus, iPhone SE, iPhone 7, iPhone 7 Plus, iPhone 8, iPhone 8 Plus, iPhone X, iPhone XS, iPhone XS Max, iPhone XR, iPad Air, iPad Air Wi-Fi + Cellular, iPad mini 2, iPad mini 2 Wi-Fi + Cellular, iPad Air 2, iPad Air 2 Wi-Fi + Cellular, iPad mini 3, iPad mini 3 Wi-Fi + Cellular, iPad mini 4, iPad mini 4 Wi-Fi + Cellular, 12.9-inch iPad Pro, 12.9-inch iPad Pro Wi-Fi + Cellular, 9.7-inch iPad Pro, 9.7-inch iPad Pro Wi-Fi + Cellular, iPad (5th generation), iPad Wi-Fi + Cellular (5th generation), 12.9-inch iPad Pro (2nd generation), 12.9-inch iPad Pro Wi‑Fi + Cellular (2nd generation), 10.5-inch iPad Pro, 10.5-inch iPad Pro Wi‑Fi + Cellular, iPad (6th generation), iPad Wi-Fi + Cellular (6th generation), iPad Pro (11-inch), iPad Pro (11-inch), iPad Pro (12.9-inch) (3rd generation), iPad Pro (12.9-inch) (3rd generation), and iPod touch.
- Age Rating
- Rated 4+
- © 2019 Evan Miller
- In-App Purchases
- Pro Features $8.99
With Family Sharing set up, up to six family members can use this app.