Skill Beta Testing for Alexa Skills

Before you release your Alexa skill to production, you can test a beta version of your skill with a limited set of users. Use the skill beta testing tool to test your skill and receive customer feedback. You can also use the skill beta testing tool to test changes to an existing skill, while still keeping the live version of the skill available for the general public. At any time, you can add testers, remove testers, or end a test. You can get feedback on your skill throughout the testing cycle, and reduce costs by making fixes before you release the skill to production. Using skill beta testing can help increase your chances of skill success.

Security

To use the beta skill, the email address associated with the Alexa device used by the beta tester must be the same as the email address used to manage access to beta testing.

If the beta tester doesn't have an Alexa device registered to that email address, the tester can create an Amazon account with that email address and register it to their Alexa device. Or, the tester can contact you to request a separate invitation at the email address associated with their device.

Prerequisites

To be eligible for testing with the skill beta testing tool, your skill must meet the following criteria:

To verify that your skill is eligible for beta testing, navigate to Distribution > Availability, and then expand Beta Test. If your skill is missing required information, this area indicates that beta test isn't available.

After you complete all required information, your skill is eligible for testing with the skill beta testing tool. These prerequisites also make your skill ready for certification.

Use the beta testing tool for a beta version of a live skill

When you have a live skill, you can still create a beta version of the skill. The testers who have access to the beta version won't have access to the live version. A tester can request Amazon to remove them from a beta test. After removal, this tester can re-enable the live skill in their Alexa app.

As described in Revise and Update Your Skill after Publication, after you publish your skill, that version is available to the customer.

The developer console displays the live version and the development version of your skill, as shown. Click Edit to change the development version. In this development version, use a different endpoint or Amazon Web Services (AWS) Lambda function to make sure that your original skill isn't modified. Make sure that you follow all the criteria described in Prerequisites, and then follow the steps to create a beta test for your skill.

Create a beta test for your skill

After you complete the prerequisites, your skill is eligible for testing with the skill beta testing tool. Complete the following steps to create a beta test for the development version of your skill.

To create a beta test in the Alexa developer console

  1. Edit your skill in the Alexa developer console.
  2. Navigate to the Distribution > Availability page.
  3. Expand Beta Test, and then click Start Test.
  4. Under Manage link access, for Enter tester email addresses, add an email address for each tester. Remember that this email address must be the one that the tester uses for their Alexa account.
    • Separate each email address from the next with a semicolon or carriage return.
    • To upload a file of email addresses, click Bulk Upload. Make sure that your separate these addresses by semicolons or carriage returns, or both.
  5. To save the tester email addresses, click Add.
    Amazon validates the email addresses, checks for duplicates, and adds them to the test. If one or more email addresses aren't valid, you see an error. Correct the email addresses, and then click Add again.
  6. Review the list of eligible testers. Unless the skill is in certification, you can also add or remove testers after the test starts.
  7. To copy a link to the beta test, under Share link, click Copy link.
    In a convenient place, such as Notepad on Windows or TextEdit on Mac, paste the link. You include the link in the email to your beta testers.
  8. To view a sample email, click View email template.
Screenshot of beta test page on the Alexa developer console to create a beta test and add tester emails.
Create a beta test for your skill

Share the beta test link with your testers

After you create the beta test, email your testers to notify them of the availability of the beta skill. Invite them to test the skill and provide feedback to your email address. You can use the following sample email template.

You're invited to test this beta skill.

Skill name: [Your skill name] - Beta Version [Paste link to the beta test]

Skill description: [Add your skill description]

Before they can start testing your skill, a beta tester must click the link on their mobile device to log into their Amazon account and accept the Skill Beta Testing Terms and Conditions. Then, the tester enables the beta skill from the Alexa app.

Manage a beta test for your skill

During beta testing, the Beta Test section of the Distribution > Availability page in the Alexa developer console provides a dashboard that shows which testers have access to the beta test link. The dashboard allows you to add and remove testers, or end a test.

Remove testers from a beta test

You can remove a tester from an ongoing test at any time. After you remove a tester, that tester loses access to the beta skill. The test does continue for the remaining testers. If the skill has both a live version and a development version, the removed tester now has access to the live version.

To remove a tester from the beta test in the Alexa developer console

  1. Edit your skill in the Alexa developer console.
  2. Navigate to the Distribution > Availability page.
  3. Expand Beta Test.
  4. Navigate to the tester email address that you want to remove, and then click Revoke access.
    Amazon updates the list of testers.

Add testers to a beta test

Unless the skill is in certification, you can add one or more testers to an existing beta test. During certification, only testers who already enabled the beta skill can participate. After the skill is out of certification, either by passing or failing, you can add additional users to the beta test.

To add a tester to the beta test in the Alexa developer console

  1. Edit your skill in the Alexa developer console.
  2. Navigate to the Distribution > Availability page.
  3. Expand Beta Test.
  4. Under Manage link access, for Enter tester email addresses, add the email addresses of the testers.
    You can enter the email addresses directly in the box, or click Bulk Upload to upload a CSV file with email addresses.
  5. Click Add to validate the email addresses. If necessary, correct the errors and click Add again.

The display shows the newly added testers, along with the existing testers. Don't forget to send the beta testing link to the newly added testers.

End a beta test

By default, a beta test ends after 90 days. You can choose to end a test at any time before 90 days.

To end a beta test in the Alexa developer console

  1. Edit your skill in the Alexa developer console.
  2. Navigate to the Distribution > Availability page.
  3. Expand Beta Test.
  4. Click End test.
  5. Click Yes to the confirmation message.

Beta testing FAQ

The following frequently asked questions (FAQ) answer common questions about beta testing.

Q: Is the skill beta testing tool free?
Yes, the skill beta testing tool is a free service.
Q: How can I share my beta skill with friends and family?
Obtain the email addresses that your friends and family use for their Alexa account. Add these email addresses to the beta test that you set up. Then, send email to the testers with the secure link to your skill generated by the skill beta testing tool. When a tester clicks the link on their mobile device, they login with their Amazon credentials and opt in to test your skill. When the tester enables the beta skill in their Alexa app, the skill is automatically enabled on any device associated with that Amazon account.
Q: How long can the beta version be live?
The beta version is live for a maximum of 90 days, at which point the beta test ends automatically. During the 90-day period, you can choose to end the beta test at any time, but you can't extend it. You can create a new beta test, which requires that you take the same steps as when you created the original test.
Q: What's the maximum number of beta testers I can invite per skill?
You can invite up to 500 testers.
Q: Can a beta tester use a device that has a different region and locale than the skill?
If your skill is available for all regions, or available for the same region and locale as that of the beta tester's device, they can use your skill. If your beta tester uses a device with a different region and locale than your skill, they won't be able to use your skill.
Q: Can I have a beta test version of a skill at the same time that I have a live version of a skill?
Yes, both a live version, which is available to the general public, and a beta test version, available to selected testers, can co-exist. However, after they enable the beta skill, a beta tester can't access the live version of the skill, unless you remove the tester from the test, or a tester chooses to remove themselves from the test, or the beta test ends.
Q: Why haven't my testers received an email?
Amazon no longer sends emails to testers on your behalf. You can follow this email template and share the link directly with testers. To find the right link, navigate to Distribution > Availability > Beta Test, and then click Copy link.
Q: How do my testers get the link to enable my beta skill?
After you add the email addresses of your recruited testers to give them access to beta skill, you can share the beta test link with them. Although Amazon no longer sends emails to testers on your behalf, you can follow this email template. Testers should be able to open the link on their mobile device, and enable the beta skill from the Alexa app. Along with the link, send any specific testing instructions to the testers, share any known issues with testers to gain valuable feedback, and send any material that might help you collect feedback, such as a survey.
Q: Can I publish different beta versions of the skill to multiple groups?
No, you can't publish multiple versions of the skill.
Q: How can the beta testers provide feedback?
The testers provide feedback directly to you.
Q: How can beta users differentiate between a live skill and a beta skill?
The beta skill has a dev tag on the skill icon.
Q: Must the skill go through certification to be able to launch it as a beta?
The skill doesn't require any certification to start a beta test.
Q: Can I update the beta version of the skill after I have published my beta?
Yes, you can update the beta version of the skill, after you have published the beta.
Q: Are developers be able to use actual Account linking with the skill beta testing tool?
Yes, developers can use the account linking feature with the beta testing tool.
Q: How do I remove a tester from a beta test?
To remove access to a beta test, click the Revoke Access button next to their email address. You can also choose to end the test, which removes access for all testers.
Was this page helpful?

Last updated: Sep 10, 2024

DevAssistant