Skill Beta Testing for Alexa Skills

Note: Sign in to the developer console to build or publish your skill.

Before you release your Alexa skill to production, you can test a beta version of your skill with a limited set of users. Use the skill beta testing tool to test your skill and receive customer feedback. You can also use the skill beta testing tool to test changes to an existing skill, while still keeping the live version of the skill available for the general public. You can invite friends or family, your social network contacts, or other people for whom you have an email address to test your skill and provide feedback. You can send up to 500 invitations per skill. At any time, you can add testers, remove testers, or end a test. You can see tester feedback in the Amazon Developer Portal. You can get feedback on your skill throughout the testing cycle, and reduce costs by making fixes before you release the skill to production. Using skill beta testing can help increase your chances of skill success.

Note: The skill beta testing tool is only applicable for Alexa skill development.

Security

To use the beta skill, the email address associated with the Alexa device used by the beta tester must be the same as the email address in the invitation.

If the beta tester doesn't have an Alexa device registered to that email address, the tester can create an Amazon account with that email address and register it to their Alexa device. Or, the tester can contact you to request a separate invitation at the email address associated with their device. For testers to contact you, you must enter a contact email address, as described in Create a Beta Test for Your Skill.

Some beta testers might have opted out of receiving Amazon emails. These beta testers won't receive your invitation email. You can resolve this situation in two ways:

Send beta testers the link to the beta skill from your own email account. This Invite URL link is available from the Manage testers page. For details, see Manage a Beta Test for Your Skill.
Instruct beta testers how to opt in to receiving Amazon emails by pointing them to Managing Your Account > Subscriptions & Communication Preferences.

Set up a beta test for a skill

Edit your skill in the developer console.
Navigate to the Distribution > Availability page.
Expand Beta Test. If your skill is missing required information, this area indicates that beta test isn't available.

Make a new skill eligible for beta testing

To be eligible for testing with the skill beta testing tool, your skill must meet specific criteria. These criteria include:

The Build page must be complete.
All required fields on Distribution > Skill Preview must be complete.
All required fields on Distribution > Privacy & Compliance must be complete.

After you complete all required information, your skill is eligible for testing with the skill beta testing tool. These steps also make your skill ready for certification.

Use the skill beta testing tool for a beta version of a live skill

When you have a live skill, you can still create a beta version of the skill. The testers who have access to the beta version won't have access to the live version. A tester can request Amazon to remove them from a beta test. After removal, this tester can re-enable the live skill in their Alexa app.

As described in Revise and Update Your Skill after Publication, after you publish your skill, that version is available to the customer.

The developer console displays the live version and the development version of your skill, as shown. Click Edit to change the development version. In this development version, use a different endpoint or AWS Lambda function to make sure that your original skill isn't modified. Make sure that you follow all the criteria described in Make a New Skill Eligible for Testing and follow the steps to Create a Beta Test for Your Skill.

Live and Development Versions of a Skill

Create a beta test for your skill

Edit your skill in the developer console.
Navigate to the Distribution > Availability page.
Expand Beta Test.
Enter the Beta Test Administrator Email Address and click Add.
Under Manage Access to your Skill Beta Test, add tester email addresses. Remember that this email address must be the one that the tester uses for their Alexa account.
- Separate each email address from the next with a semicolon or carriage return.
- To upload a file of email addresses, click Bulk Upload. Make sure that these addresses are separated by semicolons or carriage returns, or both.
Click Add to save the tester email addresses. Amazon validates the email addresses, checks for duplicates, and adds them to the test. If one or more email addresses aren't valid, you will see an error. Correct the email addresses and click Add again.
Note: Amazon removes any tester who doesn't enroll in the beta test within 30 days.
Review the list of eligible testers. You can also add or remove testers after the test starts, unless the skill is in certification.
Click Enable beta testing.

Each tester receives an email with a customized link inviting them to be beta testers for your skill. Amazon controls the messaging for this invitation.

The invitation email takes the following form, with the subject line of "You're invited to test a new Alexa skill."

Before they can start testing your skill, a beta tester must click the link to log into their Amazon account and accept the Skill Beta Testing Terms and Conditions.

Note: The Alexa web app Interface has been deprecated. Therefore, the only supported method of enabling a beta test is through the iOS or Android Alexa mobile app.

Manage a beta test for your skill

During beta testing, the Beta Test section of the Distribution > Availability page provides a dashboard that shows which testers have accepted your invitation. The dashboard allows you to add and remove testers.

To open the dashboard, open the skill in the developer console and navigate to Distribution > Availability.

The Beta Test box shows the current invitation status for each tester. From here, you can add and remove testers, send reminders to and feedback requests to testers, or end a test.

Remove a tester

You can remove a tester from an ongoing test at any time.

Edit your skill in the developer console.
Navigate to the Distribution > Availability page.
Expand Beta Test.
Click Remove tester for the tester you want to remove.

Amazon updates the list of testers.

If you remove a tester, that tester receives an email that the test has ended, although the test does continue for the remaining testers. If the skill has both a live version and a development version, the removed tester now has access to the live version.

Send a reminder to testers

You can send a reminder to a tester for an ongoing test at any time. However, you're allowed to send one reminder to each tester during the course of the beta test.

Edit your skill in the developer console.
Navigate to the Distribution > Availability page.
Expand Beta Test.
Click Send a reminder for the tester.

Amazon sends the tester an email reminder, again with the messaging controlled by Amazon. This email contains the same content as the original invitation email, but the subject line is "[Reminder] You are invited to test a new Alexa skill."

Request feedback from testers

On the Beta Test page, you can request feedback from testers who have enabled the skill. Click the Request Feedback link next to the tester's email address. Make sure that you enable your Beta Test Administrator Email Address to receive messages. The email that the testers receive looks like the following:

Email Requesting Feedback From Beta Testers

Add one or more testers to an existing test

You can add one or more testers to an existing test unless the skill is in certification. When the skill is in certification, only testers who already accepted the invitation can participate. After the skill is out of certification, either by passing or failing, you can add additional users to the beta test.

Edit your skill in the developer console.
Navigate to the Distribution > Availability page.
Expand Beta Test.
Add the email addresses of the testers to the Tester emails box. You can enter the email addresses directly in the box, or click Bulk Upload to upload a CSV file with email addresses.
Click Add to validate the email addresses. If necessary, correct the errors and click Add again.

The display shows the newly added testers, along with the existing testers. Amazon sends the same invitation email as the original testers.

Note: Any tester who doesn't enroll within 30 days is automatically removed from the beta test.

End a test

You can choose to end a test at any time.

Edit your skill in the developer console.
Navigate to the Distribution > Availability page.
Expand Beta Test.
Click End test.
Click Yes to the confirmation message.

Your testers receive an email that the test has ended.

Beta testing FAQ

The following frequently asked questions (FAQ) answer common questions about beta testing.

Q: Is the skill beta testing tool free?: Yes, the skill beta testing tool is a free service.
Q: How can I share my beta skill with friends and family?: Obtain the email addresses that your friends and family use for their Alexa account. Add these email addresses to the beta test that you set up. The skill beta testing tool sends them the invitation email by sharing a secure link to your skill generated by the skill beta testing tool. When a tester clicks the link, they login with their Amazon credentials and opt in to test your skill. When the tester does opt in, the skill is automatically enabled on any device associated with that Amazon account.
Q: How long can the beta version be live?: The beta version is live for a maximum of 90 days, at which point the beta test ends automatically and Amazon notifies all participants by email. During the 90-day period, you can choose to end the beta test at any time, but you can't extend it. You can create a new beta test, which requires that you take the same steps as when you created the original test.
Q: What's the maximum number of beta testers I can invite per skill?: You can invite up to 500 testers.
Q: Can a beta tester use a device that has a different region and locale than the skill?: If your skill is available for all regions, or available for the same region and locale as that of the beta tester's device, they can use your skill. If your beta tester uses a device with a different region and locale than your skill, they won't be able to use your skill.
Q: Can I have a beta test version of a skill at the same time that I have a live version of a skill?: Yes, both a live version, which is available to the general public, and a beta test version, available to selected testers, can co-exist. However, after they enable the beta skill, a beta tester can't access the live version of the skill, unless you remove the tester from the test, or a tester chooses to remove themselves from the test, or the beta test ends.
Q: Can I prompt testers for feedback?: If at least one tester has enabled the skill, on the Manage skill page, you can click the Request Feedback button. Each tester who has enabled the skill gets an email request to send feedback to your contact email address.
Q: Can I customize the email messages sent to testers?: You can't customize the messages sent by Amazon. However, you can communicate with your testers outside the scope of the skill beta testing tool.
Q: Can I publish different beta versions of the skill to multiple groups?: No, you can't publish multiple versions of the skill.
Q: Are the emails optimized for mobile?: Yes, Amazon optimizes the email templates for mobile devices.
Q: How can the beta testers provide feedback?: The testers provide feedback by using the Give Feedback email link available within the invitation email. In addition, if you click the Request Feedback button on the Manage testers page, Amazon sends another email requesting feedback to those testers who have enabled the original beta skill link.
Q: How can beta users differentiate between a live skill and a beta skill?: The beta skill has a "dev" tag on the skill icon.
Q: Must the skill go through certification to be able to launch it as a beta?: The skill doesn't require any certification to start a beta test.
Q: Can I update the beta version of the skill after I have published my beta?: Yes, you can update the beta version of the skill, after you have published the beta.
Q: Can the beta testers unsubscribe to the invitation emails?: Yes, beta testers can unsubscribe from emails from the skill beta testing tool.
Q: Are developers be able to use actual Account linking with the skill beta testing tool?: Yes, developers can use the account linking feature with the beta testing tool.
Q: How do I remove a tester from a beta test?: To remove a tester, click the Remove Tester button next to their email address. You can also choose to end the test, which effectively removes access for all testers.

Last updated: Feb 03, 2023