Test and Debug a Custom Skill

This document provides recommendations for testing custom skills.

General Testing Recommendations

  • Test your invocation name. Make sure it is easy to say and consistently recognized by Alexa. Consider changing the name if Alexa frequently fails to recognize it. For more information about invocation names, see Choose Your Invocation Name.
  • Test all the different sample utterances in your voice interface.

    Note: as you test, you will likely add additional sample utterances. After adding a new utterance, there may be a short delay before the new utterance is available for testing, particularly if you want to test the utterance with your invocation name ("Alexa, ask <invocation name> to give me the horoscope for Gemini").

  • Test variations of the sample utterances with different slot values and slightly different phrases. Pay extra attention to sample utterances that are very similar.

  • Make sure your service responds properly to the different types of requests sent by Alexa:

    • LaunchRequest
    • IntentRequest
    • SessionEndedRequest
  • Make sure all of your intents can be correctly triggered from your sample phrases and variations on those phrases. For details about defining your intents and sample utterances, see:

  • If your skill includes visual and touch interactions, ensure that the display appears correctly and that each touch interaction invokes the desired intent.
  • Review the detail page for the skill in the Amazon Alexa App (under the Skills option). Make sure the icon, description, example phrases, and other fields provide meaningful information to help users understand the capabilities you are adding to Alexa.
  • If you can observe other users during testing, note the phrases that they speak to invoke each intent and update the sample utterances accordingly.
  • Specifically test the example phrases that you provided in the developer portal. These are the phrases new users try first, so you want to be sure that they work well.

For detailed testing you should do before submitting the skill for certification, see Certification Requirements for Custom Skills.

Test Slots for User Errors

For intents that include slot values, test different permutations, such as missing or incorrect values. If you are using any of the built-in slot types such as AMAZON.DATE, AMAZON.NUMBER, or AMAZON.DURATION, verify that your code responds appropriately if the words the user says for those slots cannot be converted to the specified data type.

For example, a skill that adds two numbers provided by the user might have an intent that defines two AMAZON.NUMBER slots and a sample utterance similar to this:

NumberAddIntent add {xValue} and {yValue}

Test this phrase with non-numeric words for the xValue and yValue slots and verify that your service provides an appropriate response (such as a response letting the user know that the device did not hear numeric data).

Similarly, if a slot should always contain a value from a fixed set of values, verify that your service responds correctly if the slot provided by the user is not in this set. For example, "Daily Horoscopes" might expect a valid Zodiac sign in the Sign slot. Verify that your service provides a user-friendly error message if the Sign slot contains a value that does not match any Zodiac signs.

For recommendations around responding to unexpected input and dialog errors, see the section on errors in the Alexa Design Guide.

Use the Amazon Alexa App when Testing

The Amazon Alexa App is a companion app available for Fire OS, Android, iOS, and web browsers (http://alexa.amazon.com/). This app provides two different tools that can be useful when testing:

  • The home page "cards". These are normally used to give users a visual response in addition to the spoken response from an Alexa skill. For example, when a user asks Alexa for the weather, the detailed forecast is provided on a visual card.
  • The Alexa history. This records every spoken interaction with Alexa, regardless of whether the correct intent was triggered. This can be useful for seeing exactly how Alexa interpreted an utterance and for tracking down problems in your voice interface.

Note that you must log in to the Amazon Alexa App using the same account to which your device is registered:

  • For Fire OS devices, the Amazon Alexa App is automatically logged in to the same Amazon account to which the device is registered.
  • For iOS, non-Amazon Android devices, and web browsers, you are prompted to log in the first time you launch the app. To switch between accounts, click on Settings, then Log Out.

Amazon Alexa App Cards

To see the cards, open the Amazon Alexa App and open the main menu. From the menu, navigate to Activity.

You have control over the home page cards returned by your service. During the early stages of development, you may want to send back these cards from all LaunchRequest and IntentRequest requests. You can include basic debugging information (such as the name of the intent that was triggered) to help you see how your service is working. Later, as you prepare your service for end users, you can return cards that are more relevant to a normal end user's experience.

In addition to the cards you control, Alexa sends back cards in response to errors communicating with your service. Some common errors (note that this is not a complete list):

Error Card Possible Causes
Unexpected Communication Issue Occurs if Alexa cannot connect to your service to send a request. This can happen for many reasons:

The endpoint specified in the configuration on the developer portal is incorrect.
The web service is not online.
The endpoint is not configured to accept traffic over port 443.
SSL Handshake Failed Occurs if SSL is configured incorrectly. For example, when you use a self-signed certificate, if the public key in the certificate presented by your endpoint does not match the key provided in the configuration in the developer portal, this error occurs.
Protocol Violation Occurs if your service's response did not conform to the specification. Often this is due to an error in your code – for example, if your service throws an unhandled exception and exits, the response received by Alexa will not be correct.
Hostname verification failed Occurs if the domain name specified for the DNS.1 setting in your certificate does not match the domain name of your endpoint.

Make sure the domain name specified in the configuration file for your certificate matches exactly. Do not include the protocol (https://) or any trailing slash marks.

For example, if your service's endpoint is at https://wiseguy.mywebserver.com/wiseguy, the DNS.1 entry for your certificate should be set to

Unrecognized Name while starting SSL handshake Occurs when using an older version Apache HTTP Server to host your web service and the server is not configured with a ServerName or ServerAlias.

To resolve this, either upgrade to 2.4.10 or later, or add ServerName / ServerAlias to your server's configuration file.

Amazon Alexa App History

The Amazon Alexa App also provides a complete history of a user's interactions. Each history item shows the text that resulted from the user's speech.

To see the history, open the Amazon Alexa App and open the main menu. From the menu, navigate to Settings > Alexa Account > History.

Viewing how the speech was converted to text is especially useful when your service does not respond as expected, for example:

  • Your service did not receive any request because Alexa misunderstood the invocation name.
  • The intent sent to your service was not what you expected.
  • The correct intent was sent, but with unexpected slot data.
  • No intent was sent to your service because Alexa misunderstood the user's command.

Other Testing and Debugging Tools

Log files are another useful option when testing, especially for catching problems that prevent your service from sending a response.