Deine Alexa Dashboards Einstellungen
Vielen Dank für Ihren Besuch. Diese Seite ist momentan nur auf Englisch verfügbar. Wir arbeiten an der deutschen Version. Vielen Dank für Ihr Verständnis.

Test a Custom Skill

During development, there are multiple options for testing a new skill:


Prepare Your Skill to Test

Before you begin to test your skill, make sure you have done the following:

  1. Build the web service or Lambda function, as described in the following topics:
  2. Define the intents and sample utterances for your voice interface and enter these on the Developer Portal.
  3. If applicable, ensure that the images, audio files, and video files used in your skill are hosted on a publicly accessible web site.

High-Level Testing Steps

Test an AWS Lambda Function

If you have created the cloud-based service for the new Alexa capability as an AWS Lambda function using AWS Lambda (a service offering by Amazon Web Services), follow these steps. Sections below provide more detail for these steps:

  1. If you are using an Alexa-enabled device (such as an Amazon Echo), make sure the device is registered to your developer account.
  2. Create or update a Lambda function for your code and set Alexa Skills Kit as the Trigger for the function as described in Creating an AWS Lambda Function for a Custom Skill. Upload your code to the function.
  3. Register the new skill on the Developer Portal. See Registering and Managing Custom Skills in the Developer Portal. For testing, you must fill out the following fields:
    • Skill Information: Skill Type (Custom Interaction Model), Name, and Invocation Name.
    • Interaction Model: Intent Schema, Sample Utterances, and Custom Slot Types.
    • Configuration: Endpoint. Select the Lambda ARN option and enter the ARN for your function.
    • Test: Set to Enabled.
  4. Test the new Alexa ability by asking questions or giving commands defined in your voice interface. If applicable, test the visual and voice interactions. See Testing and Debugging.

    You can also use the Service Simulator to type commands and see the JSON responses sent by your service.

Test a Web Service

If you have created the cloud-based service for the new Alexa capability as a web service you are hosting on your own endpoint, follow these steps. Sections below provide more detail for these steps:

  1. If you are using an Alexa-enabled device (such as an Amazon Echo), make sure the device is registered to your developer account.
  2. Deploy your web service to an Internet-accessible endpoint. See Deploying Your Web Service to an Endpoint for Testing.
  3. Register the new skill on the Developer Portal. See Registering and Managing Custom Skills in the Developer Portal. For testing, you must fill out the following fields:
    • Skill Information: Skill Type (Custom Interaction Model), Name, and Invocation Name.
    • Interaction Model: Intent Schema, Sample Utterances, and Custom Slot Types.
    • Configuration: Endpoint. Select the HTTPS option and enter the URL for your service’s endpoint.
    • SSL Certificate: Select one of the options. If you want to use a self-signed SSL certificate for testing, create that certificate and upload it to the Developer Portal. See Configuring Your Web Service to Use a Self-Signed Certificate.
    • Test: Set to Enabled.
  4. Test the new Alexa ability by asking questions or giving commands defined in your voice interface. If applicable, test the visual and voice interactions. See Testing and Debugging.

    You can also use the Service Simulator to type commands and see the JSON responses sent by your service.

Register an Alexa-enabled Device for Testing

To test with an Alexa-enabled device (such as an Amazon Echo), the device must be registered to the same e-mail address you used to sign up for your developer account on the Amazon Developer Portal. If you have already set up your device using an account other than your Amazon developer account, follow these steps:

  1. Visit the web version of the Amazon Alexa app (alexa.amazon.com) and sign-in with your Amazon developer account.
  2. Follow the on-screen instructions to re-register your device with your developer account. Note that this will unregister the existing e-mail address.
  3. If you want to add back a non-developer account as a user of the device, you can do so by using the Household feature.

Deploy Your Web Service to an Endpoint for Testing

To test your service with the Service Simulator or an Alexa-enabled device, you must deploy your web service to an Internet-accessible endpoint.

You can host your service on any cloud service that supports transmitting requests over HTTPS. See the documentation provided with the cloud hosting service for details about deploying and configuring your web service.

Configure Your Web Service to Use a Self-Signed Certificate

When the Alexa service communicates with your web service, user requests and corresponding responses are transmitted over the Internet. To protect the confidentiality and integrity of this data, Alexa strictly enforces that HTTP connections are secured using SSL/TLS.

For testing purposes, you can use a self-signed SSL certificate to meet this requirement. In this case, you can create the certificate yourself, upload it to the Developer Portal when you set up a new skill, and configure your endpoint to present this certificate when it connects to Alexa. Note that this option can only be used for testing.

See the following sections to set up a self-signed certificate for testing. These steps use OpenSSL on the Linux platform.

Create a Private Key and Self-Signed Certificate for Testing

  1. Run the following command to create a private key:

       openssl genrsa -out private-key.pem 2048
    
  2. Use a text editor to create a configuration file in the following form and save it as a .cnf file (for instance, configuration.cnf):

    [req]
    distinguished_name = req_distinguished_name
    x509_extensions = v3_req
    prompt = no
    
    [req_distinguished_name]
    C = US
    ST = Provide your two letter state abbreviation
    L = Provide the name of the city in which you are located
    O = Provide a name for your organization
    CN = Provide a name for the skill
    
    [v3_req]
    keyUsage = keyEncipherment, dataEncipherment
    extendedKeyUsage = serverAuth
    subjectAltName = @subject_alternate_names
    
    [subject_alternate_names]
    DNS.1 = Provide your fully qualified domain name
    
  3. Replace the following content in the configuration file with your own values:

    ST: Provide your two letter state abbreviation
    L: Provide the name of the city in which you are located
    O: Provide a name for your organization
    CN: Provide a name for the skill
    DNS.1: Provide the fully qualified domain name for your endpoint
    

    Note that you must provide the domain name for your endpoint in the DNS.1 section, so you may want to wait to create the certificate until you have this information.

    See below for a completed sample configuration file.

  4. Use the following command to generate a certificate. Specify the names you used for your private-key.pem and configuration.cnf files:

    openssl req -new -x509 -days 365 \
                -key private-key.pem \
                -config configuration.cnf \
                -out certificate.pem
    

This produces a self-signed certificate in a file called certificate.pem.

Save the certificate .pem, private key .pem, and the configuration .cnf files in a safe place, then update the skill configuration with the certificate.

For example, a completed configuration file for a certificate looks similar to the following example:

[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no

[req_distinguished_name]
C = US
ST = WA
L = Seattle
O = My Company Name
CN = Wise Guy

[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @subject_alternate_names

[subject_alternate_names]
DNS.1 = wiseguy.mywebserver.com

Update the Alexa Skill Configuration with the Self-Signed Certificate

After creating your certificate, you need to update the configuration in the developer portal. Unlike your private key, the certificate only contains public data and can be shared with Amazon for the purposes of identifying your service. This lets Alexa confirm the validity of the public key portion of the certificate.

  1. Log on to the Alexa section of the developer portal. From the developer portal console page, click Alexa.
  2. In the Alexa Skills Kit box, click Get Started to open the Alexa Skills Kit page. This displays any skills you have already created.

  3. Find the skill to change in the list and click Edit.
  4. Click SSL Certificate. This section is available only if you are using an HTTPS endpoint.
  5. Select the option I will upload a self-signed certificate.
  6. Open your certificate’s .pem file in a text editor, copy the entire contents, and paste it into the provided text box. The text pasted into the box should look similar to this:

    ----BEGIN CERTIFICATE----
    Encrypted data
    -----END CERTIFICATE-----
    

Configure your Endpoint with the Self-Signed Certificate

When Alexa sends a request, your service must present your certificate. The subject alternate name in your certificate must match the domain name of your endpoint.

For example, assume your service’s endpoint is at https://wiseguy.mywebserver.com/wiseguy. In this case, your endpoint needs to present a valid certificate in which the subject alternate name is set to wiseguy.mywebserver.com. You specify this in the configuration file that you use to generate the certificate.

Configure your endpoint to present this certificate. The specifics for doing this depend on how you are hosting the web service. For example, if you use Amazon Web Services Elastic Beanstalk, you upload the certificate file using the AWS Command Line Interface.

Other SSL Resources

See other resources about SSL and self-signed certificates. Note that these links for these tools take you to third-party sites.

Use the Test Simulator (Beta)

The Test Simulator (beta) lets you test your skill within the developer portal without a device. To open the simulator, navigate to the Test page for your skill and click Go to Test Simulator Beta. You can use the Test Simulator to test most functionality in a skill, including dialogs, progressive responses, and entity resolution.

Within the simulator, there are three tabs on the left for different types of testing.

Tab Description

Alexa Simulator

Use this to interact with your skill with either text or your voice. The simulator maintains the skill session with your skill just as a device would, so you can test the dialog flow. The simulator also sends any cards that your skill returns to the Alexa app the same way a device would. If your skill supports multiple languages, select the language you want to test from the drop-down list.

Enable Skill I/O to see the JSON sent to your skill and the JSON returned to Alexa for each turn of the conversation. Note that the JSON is only available for custom skills. Click on a speech bubble to see the JSON related to that turn.

Enable Echo Show Display to see an approximation of how the skill displays on the Echo Show.

Enable Device Log to see the events sent to Alexa and the directives sent to the device during the skill interactions.

Manual JSON

Use this to enter a JSON request directly and see the JSON response your skill returns.

This option does not maintain the skill session. This is very similar to testing a JSON request in the Lambda console.

Note that the Echo Show Display and Device Log options are not supported for Manual JSON.

If your skill supports multiple region-specific endpoints, you can select the endpoint to test from the list near the bottom of the page.

Voice & Tone

Use this to enter plain text or SSML and hear how Alexa speaks the text. Select the language you want to hear from the list at the bottom. All supported languages are available in this list, even if your skill does not support them.

Example of testing a skill that uses the <code>Dialog.Delegate</code> directive in the Test Simulator
Example of testing a skill that uses the Dialog.Delegate directive in the Test Simulator

Test Simulator (Beta) Limitations

Feature Test Simulator (Beta) Support

AudioPlayer interface

The Test Simulator does not render the audio playback, but the Skill I/O section shows the AudioPlayer directives sent from your skill.

Since the playback does not occur, you cannot test AudioPlayer requests that are triggered by events in the playback, such as PlaybackNearlyFinished.

Device Address API

The Device Address API lets you request address information associated with the user’s device. When testing with the Test Simulator, your skill can call the Device Address API and get back a non-error response. You can also test the flow when the user has not provided the device address permission.

However, note that the address fields in the response are set to null and the postal code field is set to a default US postal code.

The Test Simulator cannot be configured as a full device in the Alexa app.

Display interface

You can test skills that use the Display interface, and you can enable the Echo Show Display option to see an approximation of how your responses will look on the Echo Show. However, this does not show how your responses would display on any other devices, such as the Echo Spot.

PlaybackController interface

These requests are sent to a skill when a user interacts with controls such as a remote control. These requests cannot be tested with the Test Simulator.

VideoApp directives

The Test Simulator does not render the video playback, but the Skill I/O section shows the VideoApp directives sent from your skill.

Use the Basic Service Simulator

You can use these options directly on the Test page of the developer portal:

  • The Voice Simulator lets you enter a phrase in either plain text or SSML and hear how Alexa speaks the text.
  • The Service Simulator lets you type in utterances, then see both the JSON request sent to your service, and the JSON response returned by your service. You can also play back the response to hear how Alexa speaks the response.

Voice Simulator

  • Enter the text you want to hear in the Voice Simulator text box, then click Listen.
  • You can enter either plain text or text marked up with SSML.
  • When entering SSML, <speak> tags are not required.
  • The Voice Simulator does not interact with the service for your skill, so you can use it to hear text-to-speech conversion before you have completed the minimum configuration for the skill. For instance, you can experiment with the SSML output you intend to use for your skill before completing the Interaction Model.

Service Simulator

  • Before you can test with the simulator, you need to enter the minimum required data for testing a skill, as described above.
  • When entering an utterance to test, only use alphabetic characters, spaces, periods, or apostrophes. If the utterance needs to include numbers, write them out as words (“one” instead of “1”).
  • Once the JSON response sent by your service is displayed, click Listen (or the play icon) below the response to hear the response in Alexa’s voice.
  • When testing with the simulator, any cards included in your skill’s responses are not sent to the Alexa app as they are when testing with a device. Therefore, the simulator cannot be used to verify the appearance of home cards in the Alexa app. Consider using the Test Simulator (beta), which does send cards to the app. If using the Display.RenderTemplate directive, the Service Simulator displays a close approximation of what the display templates look like on Echo Show, but the display is not interactive.

The service simulator does not support testing the following:

  • AudioPlayer directives
  • PlaybackController directives
  • Dialog directives
  • VideoApp directives
  • Entity resolution
  • Customer permissions
  • Device Address API
  • Progressive Response API
  • Customer account linking
  • Listening to Alexa speak a response using the English (Australia) voice (you can test responses normally, but the Listen button is disabled)

Consider using the Test Simulator (beta), as it can handle more of these situations.

Testing and Debugging

General Testing Recommendations

  • Test your invocation name. Make sure it is easy to say and consistently recognized by Alexa. Consider changing the name if Alexa frequently fails to recognize it. For more information about invocation names, see Choosing Your Invocation Name.
  • Test all the different sample utterances in your voice interface.

    Note: as you test, you will likely add additional sample utterances. After adding a new utterance, there may be a short delay before the new utterance is available for testing, particularly if you want to test the utterance with your invocation name (“Alexa, ask <invocation name> to give me the horoscope for Gemini”).

  • Test variations of the sample utterances with different slot values and slightly different phrases. Pay extra attention to sample utterances that are very similar.

    For details about defining slot values for your sample utterances, see the Custom Interaction Model Reference.

  • Make sure your service responds properly to the different types of requests sent by Alexa:

    • LaunchRequest
    • IntentRequest
    • SessionEndedRequest
  • Make sure all of your intents can be correctly triggered from your sample phrases and variations on those phrases. For details about defining your intents, see:
  • If your skill includes visual and touch interactions, ensure that the display appears correctly and that each touch interaction invokes the desired intent.
  • Review the detail page for the skill in the Amazon Alexa App (under the Skills option). Make sure the icon, description, example phrases, and other fields provide meaningful information to help users understand the capabilities you are adding to Alexa.
  • If you can observe other users during testing, note the phrases that they speak to invoke each intent and update the sample utterances accordingly.
  • Specifically test the example phrases that you provided in the developer portal. These are the phrases new users try first, so you want to be sure that they work well.

For detailed testing you should do before submitting the skill for certification, see Certification Requirements for Custom Skills.

Testing Slots for User Errors

For intents that include slot values, test different permutations, such as missing or incorrect values. If you are using any of the built-in slot types such as AMAZON.DATE, AMAZON.NUMBER, or AMAZON.DURATION, verify that your code responds appropriately if the words the user says for those slots cannot be converted to the specified data type (for details about slot values, see the Custom Interaction Model Reference).

For example, a skill that adds two numbers provided by the user might have an intent that defines two AMAZON.NUMBER slots and a sample utterance similar to this:

NumberAddIntent add {xValue} and {yValue}

Test this phrase with non-numeric words for the xValue and yValue slots and verify that your service provides an appropriate response (such as a response letting the user know that the device did not hear numeric data).

Similarly, if a slot should always contain a value from a fixed set of values, verify that your service responds correctly if the slot provided by the user is not in this set. For example, “Daily Horoscopes” might expect a valid Zodiac sign in the Sign slot. Verify that your service provides a user-friendly error message if the Sign slot contains a value that does not match any Zodiac signs.

For recommendations around responding to unexpected input and dialog errors, see What Alexa Says.

Using the Amazon Alexa App when Testing

The Amazon Alexa App is a companion app available for Fire OS, Android, iOS, and web browsers (http://alexa.amazon.com/). This app provides two different tools that can be useful when testing:

  • The home page “cards”. These are normally used to give users a visual response in addition to the spoken response from an Alexa skill. For example, when a user asks Alexa for the weather, the detailed forecast is provided on a visual card.
  • The Alexa history. This records every spoken interaction with Alexa, regardless of whether the correct intent was triggered. This can be useful for seeing exactly how Alexa interpreted an utterance and for tracking down problems in your voice interface.

Note that you must log in to the Amazon Alexa App using the same account to which your device is registered:

  • For Fire OS devices, the Amazon Alexa App is automatically logged in to the same Amazon account to which the device is registered.
  • For iOS, non-Amazon Android devices, and web browsers, you are prompted to log in the first time you launch the app. To switch between accounts, click on Settings, then Log Out.

Amazon Alexa App Home Page Cards

To see the home page cards, open the Amazon Alexa App and click Home from the navigation pane.

You have control over the home page cards returned by your service. During the early stages of development, you may want to send back these cards from all LaunchRequest and IntentRequest requests. You can include basic debugging information (such as the name of the intent that was triggered) to help you see how your service is working. Later, as you prepare your service for end users, you can return cards that are more relevant to a normal end user’s experience.

In addition to the cards you control, Alexa sends back cards in response to errors communicating with your service. Some common errors (note that this is not a complete list):

Error Card Possible Causes
Unexpected Communication Issue Occurs if Alexa cannot connect to your service to send a request. This can happen for many reasons:

The endpoint specified in the configuration on the developer portal is incorrect.

The web service is not online.

The endpoint is not configured to accept traffic over port 443.

Your service is a Lambda function configured with skill ID verification, but the skill ID specified in the Lambda trigger does not match the skill ID in the request sent from Alexa.
SSL Handshake Failed Occurs if SSL is configured incorrectly. For example, when you use a self-signed certificate, if the public key in the certificate presented by your endpoint does not match the key provided in the configuration in the developer portal, this error occurs.
Protocol Violation Occurs if your service’s response did not conform to the specification. Often this is due to an error in your code – for example, if your service throws an unhandled exception and exits, the response received by Alexa will not be correct.
Hostname verification failed Occurs if the domain name specified for the DNS.1 setting in your certificate does not match the domain name of your endpoint.

Make sure the domain name specified in the configuration file for your certificate matches exactly. Do not include the protocol (https://) or any trailing slash marks.

For example, if your service’s endpoint is at https://wiseguy.mywebserver.com/wiseguy, the DNS.1 entry for your certificate should be set to

wiseguy.mywebserver.com
Unrecognized Name while starting SSL handshake Occurs when using an older version Apache HTTP Server to host your web service and the server is not configured with a ServerName or ServerAlias.

To resolve this, either upgrade to 2.4.10 or later, or add ServerName / ServerAlias to your server’s configuration file.

Amazon Alexa App History

The Amazon Alexa App also provides a complete history of a user’s interactions. Each history item shows the text that resulted from the user’s speech.

To see the history, open the Amazon Alexa App and click on Settings, then on History.

Viewing how the speech was converted to text is especially useful when your service does not respond as expected, for example:

  • Your service did not receive any request because Alexa misunderstood the invocation name.
  • The intent sent to your service was not what you expected.
  • The correct intent was sent, but with unexpected slot data.
  • No intent was sent to your service because Alexa misunderstood the user’s command.

Other Testing and Debugging Tools

Log files are another useful option when testing, especially for catching problems that prevent your service from sending a response.

For an example of setting up logging and accessing log files in Elastic Beanstalk, see the “Accessing Log Files” section of Deploying a Sample Custom Skill as a Web Service.

Next Steps