Five common errors why Alexa skills don’t get certified—and how to fix them

To ensure that customers have the best possible experience when using an Alexa skill, developers are required to certify each new skill before introducing it to the world. Skill certification is the process by which a new skill is tested by Amazon’s internal certification team.

In this article, Shibashis Sahu, program manager of Alexa Skills Insights, provides tips to developers on the five most common pitfalls that come in the way of their skills being certified. Alexa Skills Insights is a global team that works to identify, test, and provide recommendations to Alexa skill developers.

The certification process

The team evaluates skills based on a number of criteria that include:

• Functionality. Alexa skills need to pass functional tests to verify that the skill works as intended. “Functional tests include checking to see whether the skill is usable,” says Sahu. 
• Policy. The skill must meet Alexa policy guidelines so that it is suitable for all customers. “For example, the skill should not contain graphic depictions or descriptions of unsettling content and excessive violence.” says Sahu. 
• Security. The skill and hosting method must meet Alexa security requirements designed to protect customer data. “For example, skills hosted as web services on your own endpoint must be secure,” says Sahu. 
• Skill interaction. Skills need to have a robust voice user interface that customers can use to interact with it effectively. “We want the customer experience to be great all the time,” says Sahu.

The skill certification process begins after a developer has designed, built, and tested the skill. Developers submit the skill through the Alexa developer portal, where it must pass automated checks before entering the certification queue. Next, the certification team checks the skill for the four requirements listed above. If the skill passes all checks, it is approved and can go live.

If the skill doesn’t pass the certification requirements, developers have the opportunity to fix the issues in question and resubmit the skill. “We let them know why the skill didn’t pass, and the exact changes that they need to make to correct that particular failure in the next version,” says Sahu.

Understanding the five reasons Alexa skills don’t get certified

The most common reasons that skills don’t pass certification on the first try include:

1. Problems with customer-facing information
Customer-facing information helps the customer understand what a skill does, and how to use it. Developers often spend a lot of time building a skill. However, at times, they don’t clarify what a customer can expect from the skill. 

The remedy: The skill’s description should include overview of the skill’s functionality, examples of how and why the skill is used, and answers to common questions about the skill.

2. Intent response errors
When customers interact with a skill, they expect to receive relevant responses to their requests. When utterances don’t work as expected, customers will receive a response that is not pertinent or is error-prone. This is due to some or all the intents present in the skill not working as per the details provided in the skill’s description. For example, a user tries to launch Little Cat Facts skill, but the responses are related to dogs:

User: “Alexa, open Little Cat Facts.”
Skill response: “Dogs are the most common pets at home.”

The remedy: There could be several reasons for an utterance to not work as expected. It could be that an utterance has not been modeled exactly as mentioned in the skill description in the intent schema. In addition, there may be issues with the source code. To provide a smooth user experience, developers should test all sample utterances for each intent in the skill to make sure that responses are relevant and work as intended. 

3. Malfunctioning example phrases
The speech output for Alexa is accomplished through text-to-speech technology that converts sequences of words into audio responses. However, there are occasions when the skill may not respond appropriately to example phrases: these are the phrases provided to customers to help them interact successfully with the skill. 

For example, a user tries to open a radio skill:

User: “Alexa, open radio Shining Star.”
Skill: “Sorry, there was a problem with the requested skill’s response.”

The remedy: Prior to submitting your skill for certification, developers should conduct a test to check if all defined example phrases work properly. They can also confirm the example phrase by voice, and check the utterance history in the Alexa App by logging into the Alexa app, and tapping on ‘Settings.’

4. Missing or incorrect test account credentials
During the skill certification process, the certification team may need to perform account linking (which allows them to get user’s permission to obtain an access token”) using test credentials that are provided by the developers. In many cases, such as when a skill is intended for use with a car or smart home device, the test account also needs to have a device associated with it. Until the appropriate credentials are provided, the certification team can’t complete a full review.

The remedy: Developers should make sure to include working test account credentials when submitting a skill for certification. These can be provided using the testing instructions field under “Distribution, Privacy & Compliance” tab in the Alexa Developer portal.

5. Session management
When a user’s request is complete, the skill should either close the session or prompt the user to engage further. A skill’s session management can make or break the customer experience. Ideally, conversing with a skill should be similar to talking to a person. An example of a bad customer experience is as follows:

User: Alexa, open Space Facts.
Alexa: A year on Mercury is just 88 days long. <session remains open>

The remedy: To ensure that the session is closed when there is no user input, the developer should check if “ShouldEndSession” attribute is marked to “true”. This can be done when adding intent responses while building the skill. A session should close after completing the task when the user is not prompted for inputs within eight seconds.

Resources for improving the quality of your skill

Developers have access to a variety of resources to help them build and certify successful skills. To get started, simply log on to the Alexa Developer Console. You can visit the “test” tab to test your skill prior to submitting it for certification.

The Alexa Certification team also provides a checklist of certification requirements to assist with testing. Along with the extensive technical documentation that is available in the Alexa developer portal, skill creators have access to a knowledge base of articles to help them build and certify their skills.

Finally, Alexa offers developer support in the event that a skill fails to pass certification. 

“The support team can answer developer queries with regards to skill certification and provide inputs that developers need to update their skills,” says Sahu. “We have the same objective as you – to help you develop great skills, and grow your business by building with Alexa.”