Digital Optimization and the Future of A/B Testing

Posted by Matt Brown / June 17

They say that if you were to put infinite monkeys to work banging away at infinite pianos, a few of them will inadvertently bang out a perfect rendition of Beethoven’s 9th Symphony. That’s the power of statistics when applied to infinity. But what if we didn’t have to wade through billions or trillions of samples to find the diamonds in the rough? What if we could be more selective, test in the right general areas, and find the optimal option more quickly?

Some extraordinarily clever individuals are starting to harness the power of statistics to optimize their products and services for the ultimate payoff and ROI. With the popularity of Moneyball’s application to sports and Nate Silver’s application to (mostly) politics, data science and marketing is starting to become more mainstream and is also finding its way into web development more and more. You may have heard about the Obama campaign case study that dramatically increased their donation conversion rates by implementing a few small experiments to determine which headline people would best respond to, which layout design they would best respond to, which Obama family photos they would best respond to, et cetera. This approach is akin to ramping up the evolutionary cycle to optimize for a certain goal – in this case, increasing grassroots campaign donations.

Legacy recently harnessed this approach for an eCRM email marketing program that we executed for one of our clients. Basically, we implemented a series of tests on each email blast and we learned some fascinating things about how people receive branded content. We implemented subject line tests, distribution time-of-day tests, email body content tests, call-to-action tests, database quality tests, design tests… and subsequently watched as our open/click rates improved! This concept can just as easily be applied to website, application, and prototype development.

In my previous blog post on the Top 5 Tech Trends at SXSW, I briefly mentioned the growing popularity of data science and marketing which seems to be popping up in lots of different industries and areas. And the more I’ve seen of this approach, the more confident I am in its power. I feel strongly that this approach will be a central component of development for the top digital and experiential agencies moving forward into the future.