Your Alexa Dashboards Settings

Product Testing Overview

Amazon Alexa users expect a high quality and consistent experience across all products. To help you meet this bar and build the best possible Alexa integration, this document explains the product testing process, and provides an overview of the typical Alexa Voice Service (AVS) development lifecycle.

Product Testing

Product testing is broken into two steps: self-test, and product submission. During self-test, you will use a series of checklists provided by Amazon to ensure that your product meets functional and user experience (UX) requirements. If your product supports music and/or is far-field enabled, additional testing is required prior to submitting your product to Amazon for evaluation.

After you’ve completed the self-test checklists, they must be submitted to Amazon for evaluation prior to commercial distribution of your AVS product. Please work with your Amazon point of contact to schedule product submission and testing. This process takes approximately 10 weeks.

Step 1: Complete Self-Test Checklists

For access to these self-tests and the certification intake form, please contact your Amazon POC or follow the instructions for how to launch with AVS.

Step 2: Submit Your Product for Testing

After you’ve completed the self-test checklists, please work with your Amazon point of contact to submit the results and schedule testing. After review, authorization to ship products along with shipping instructions will be provided to you directly from the testing team.

The table below lists the number of devices to be submitted to Amazon for testing, based what features your product supports:

Test # of Products Notes
Functional evaluation          2 Required
UX evaluation          1 Required
Music Certification          1 If supported
Far-Field/Hands-Free evaluation          2 If supported

Step 3: Testing by Amazon

This is a high-level overview of the process that begins once Amazon receives your product for evaluation:

Product testing workflow
Click to enlarge
  1. Your product will be verified using an Acceptance Test to ensure that it meets basic functional requirements. This tests a subset of the items in the Functional Checklist completed during self-testing.
  2. Your product will be tested to ensure that it meets the requirements specified in the Alexa Voice Service Agreement, the Functional Requirements for AVS and User Experience Guidelines.
  3. If access to third party music or radio services using the Alexa Service is supported by your product, then music certification will be scheduled. This is required by Music Service Providers (MSP), including Amazon Music, TuneIn and iHeartRadio. Note: As new MSPs are on-boarded, music certification will be expanded to include any additional requirements.
  4. If your product is Far-Field/Hand-free enabled and uses a wake word to initiate interactions with Alexa, testing will be scheduled and performed at the same time as all other testing.
  5. Identified issues and recommendations will be documented and returned to you within 4-11 weeks from the start of testing.

NOTE: Your Amazon point of contact will provide you with a detailed timeline outlining the testing process prior to product submission.

If your product meets all testing requirements, it will be approved for launch. You will be contacted within 48-hours by your Amazon PoC when your product passes testing. At this time, any additional assets required to launch will be provided.

Firmware Updates

Any updates or changes to the firmware must be tested by the product owner before being deployed to products in the field (OTA update). If significant updates are made, the product must be re-submitted for testing prior to deployment. This ensures that the changes are compliant with AVS requirements.

Resources

The documents listed below provide design, functional, and implementation guidance: