While at GDC last week, I was able to speak to over 300 developers. It’s exciting to see all the ideas and the passion that fuels them - I love being around people that are so creative and productive. Often our conversations would turn to how to take their great idea and turn it into a great business. We’d talk about how to expand their distribution channels, best practices for monetization, opportunities for promotion and, most importantly, how to continuously improve the experience of their game.
Most of the people I spoke to understood the basic premise of AB Testing — to provide more than one experience and see which performs the best. For example, it’s common in advertising to deploy several ads with different messaging or art work to a small number of customers and then expand to more with the one that performs the best. It’s similar in a game, where you could test several experiences like the difficulty of a level, the impact of ad placement, or the effectiveness of your viral mechanic, etc.
Still, most of the developers I spoke to were surprised and excited by the following 4 aspects of Amazon’s AB Testing service:
- FREE. Anyone with a (free) developer account can use the free AB Testing service for an unlimited number of tests.
- Cross Platform. iOS, Android and, Kindle Fire are all supported. You can use AB Testing in any or all of these platforms.
- Deploy Instantly. Part of the power of AB Testing is that your producers, level designers, analysts (the non-coders) can instantly deploy the winning experience to all players without a re-submit to any app store.
- Multiple goals. It’s easy to imagine testing which ad placement results in the best click-through. But, how does it impact 3 day retention or level completion or time to first purchase? Amazon’s AB Testing can help you measure an experiment against multiple goals.
You can give it a try right now. It’s available from the developer portal at https://developer.amazon.com/public/apis/manage/ab-testing.