Anyone who has ever developed a skill for Alexa, whether hobbyist, experienced programmer or professional agency, has already used it: the submit for review button. But what exactly happens in the review? What should you pay attention to and how can you avoid unnecessary resubmissions of your skill?
As a developer, you sometimes ask yourself "Is my skill even being tested by real people?" or "Are the testers using physical Echo devices for certification?" We want to get to the bottom of these questions today. So this is what happens, when you click on the submit for review button:
Right after the click, your skill will be received by the certification team. This team will perform your requested review by testing different categories of your current submission:
Depending on which countries and languages are selected for publication, different certification teams in several countries around the world work on the review of a single skill.
The testers consciously act like end user and will certify your skill on different device types by using their own voice to simulate a realistic testing environment. If a developer decides to offer display support for his skill as well, screen devices will also be involved during the testing.
A test plan defines the skill-testing framework for different scenarios in order to keep the certification consistent at a global level. The test plan criteria have to be adapted on a regular basis, for example due to API launches or due to new use cases developers implemented. These scenarios have to be evaluated case by case, which might lead to extended processing time within the certification.
As soon as the testers of the individual countries and languages have completed the review, they are consolidated at a central point, cross-checked and sent to the developer.
If the skill passes certification, it will be available in the Alexa Skill Store. If a publication is not possible, required changes will be listed in a feedback email for the developer to support the next submission. Was this feedback helpful or is there something you’re missing? The certification team appreciate your feedback, so ask for your opinion. If you are interested in sharing your experience, just check the bottom of your feedback mail.
Still got questions about your feedback? Our developer advocate team will be happy to help! Feel free to contact us for additional information or other questions regarding your skill. Keep in mind that you should receive at least one piece of feedback from the certification team.
So behind the submit for review button there are not as many secrets as you might think! If you take the time to look at the Submission Checklist, you will also find many of the test scenarios and the required results for a successful certification in advance. If you work through this list conscientiously, the chance to publish your skill in the first submission will increase drastically.
Functional Testing for a Custom Skill
Security Testing for an Alexa Skill
Voice Interface and User Experience Testing for a Custom Skill
Certification Requirements - Submission Checklist