Alexa Haus landing page

Design your skill for common certification failures

Published: December 1, 2023

Key takeaways

Among the common reasons skills might fail any of the tests performed to certify your skill, you can prevent many of them early in your design process.

 

Need quick advice?

Read the Checklist for common design-related certification failures

 

In this article:

line-break

Introduction

When you submit your skill to the Alexa skill store, Amazon performs the following tests to certify it. These test include:

  1. Functional tests verify that the skill works as intended
  2. Policy tests verify the skill follows Alexa policy guidelines and is rated appropriately
  3. Security: Verifies the skill’s hosting method meets Alexa security requirements
  4. Experience. Skills need to have a robust voice user interface that customers can use to interact with it effectively

Your skill design will need to accommodate especially (2) the need to meet policy requirements and (4) creating a robust conversation design. Some of the most common certification failures can be prevented when you design your skill aware of these requirements first.

There are a few design-related reasons that skills often fail certification, and they are easy to prevent in the early stages while you’re still creating a skill. This article will cover these common failures and design solutions for them.

Learn more about the Trustbusters that can cause certification failures.

Learn more about Natural Speech to prepare your design for UX certification (and your customers!)

First, determine whether specific policies apply to your skill, such as if it deals with financial information, is created for children, handles information about health, and more.

line-break

Checklist for preventing common design-related certification failures:

▢ Check that your interaction model contains a robust set of customer utterances for each intent

▢ Check that your interaction model does not contain intents with the same utterances

▢ Include a “welcome message“ when customers invoke your skill without a request for the first time

▢ Include a “welcome message” for customers who have used your skill repeatedly

▢ Your skill should offer help when the customer reaches an error

▢ Your skill should respond when the customer asks directly for “help”

▢ Include proactive messaging about account linking in your skill

▢ Send an account linking card to the customer’s Alexa app when appropriate

▢ Test your account linking experience with a large sample size of customer testers

▢ Indicate whether the skill session stays open or closes after each response

▢ Check your skill’s dialog for spelling, punctuation, and grammar mistakes

▢ Test and listen to all your skill’s dialog for TTS errors on an Alexa device

line-break

Common design-related reasons              skills fail certification

Problem: Not enough utterances for each of your intents

Result: The customer can’t speak naturally to your skill

Solution: When you create your Interaction Model (IM) for your skill, you’ll need to include a robust list of things the customer can say to reach each intent and to fill each slot. (Learn more about Intents, Utterances, and Slots.) When customers say something you didn’t anticipate and add to your IM, they will encounter an error instead of the response you intended. When you submit your skill for certification, also remember to thoroughly test the sample utterances you include.

Bad:

Better:

Best:

line-break

Problem: Missing or unhelpful welcome message

Result: The customer can’t invoke the skill without a request; or when they invoke a skill without a request, they receive an unhelpful response.

Solution: Your skill will need to include a “welcome message,” the speech customers will hear when they invoke your skill without making any additional request. This message should tell customers a little bit about your skill. Because come customers will only every launch your skill this way, you will need to include less detail about your skill over time, lest it become repetitive and annoying (which affects Retention).

For example, the first time a customer visits your skill, it might sound like …

After a customer is familiar with your skill, it might welcome them back like this …

Learn how to use a Large Language Model to add variation to your skill’s responses. <live article link here>

line-break

Problem: Missing or poor help experience

Result: The customer can’t get more information about how to use the skill; or the help the skill gives doesn’t help the customer solve their problem.

Solution: A customer should always be able to ask your skill for “help” directly, or be proactively offered it if they reach two consecutive errors. A good help message tells customers briefly about what the skill can do, and end with a question that prompts the customer to respond further. The message should also be contextually relevant. If, for example, your skill searches for and books fitness classes, your skill shouldn’t say the same thing if the customer asks for help while searching versus asks for help while paying. For example …

This hypothetical skill both offered help when the customer directly said “help,” as well as proactively offering it when they indicated they might need help (“I don’t know”).

Learn more about preventing and handling errors.

line-break

Problem: Account linking failures

Result: Customers don’t know how to or fail to account link. There is no voice response to a customer asking about account linking. Account linking cards fail.

Solution: Carefully design an account linking conversation and proactive messaging about account linking in your skill so customers understand they need to use their Alexa app. Have a helpful answer ready for when customer asks “How do I link my account?” or “Is my account linked?” and similar. Account linking is important for highly personalized experiences, creating a companion to an existing app, and for making purchases of goods, and it’s important for retention and monetization to make the process as easy as possible for the customer. Don’t leave account linking out of your testing with real customers before you launch. For example:

Learn more about designing a skill for Account Linking

line-break

Problem: Poor session management

Result: The session closes immediately after the skill’s speech, or the microphone opens for the customer’s response without asking a question.

Solution: Each time the skill responds to the customer, you must decide whether to end the session, or keep it open with a follow-up prompt before opening the mic. For example …

Don't:

Do:

Learn more about closing the skill session.

line-break

Problem: TTS errors

Result: Alexa doesn’t surface a response; or Alexa mispronounces words, acronyms, or misuses two words with the same spelling; or there are errors in your SSML.

Solution: If you’re using SSML, ensure there are no errors that cause undesirable sound (or lack of sound at all). Check for punctuation errors if you notice awkward pauses or clipped speech. If Alexa is mis-pronouncing words, check for spelling errors, then try using the “say-as” tags for correcting issues with abbreviations, characters, acronyms, numbers, and more.

Previous Article: