Run the Music Validation Tool

This page contains step-by-step instructions to prepare for running the Music Validation Tool (MVT) for the Alexa Voice Service (AVS) for third party manufacturers who are developing devices that will support Amazon Music.

Pre-requisites

Before running the MVT, complete the following prerequisites:

  1. Provision your developer account to gain access to the MVT:

    1. Log in to your Amazon Developer account on developer.amazon.com and navigate to the Music Certification Validation tool:
    2. From the Amazon Developer dashboard, go to Amazon Alexa > Alexa Voice Service > Products.
    3. Click the device that you are testing.
    4. For the selected device, click the Test Tools tab to view the MVT.

      The first time that you access the MVT, you will be prompted to provision your developer account to gain access to the tool:

      User plays music via Alexa then later via a 3P MSP app
    5. Click the Provision Account button and wait about 10 minutes for your account to be provisioned so that you can access the MVT.

      If you ever need to de-provision your account, you can click the Deprovision Account button for any provisioned account to revoke access to the tool. If you de-provision your account by mistake, you can re-provision it again using the same steps that you previously used.

  2. After provisioning your account, download and review the certification checklist for each MSP that you plan to have your device support:

Steps to use the tool

To run the MVT:

  1. Follow the certification checklists as you execute your tests. If you do not follow the checklists in the correct order and to the specified timestamps, your tests will fail.
  2. After provisioning your Amazon Developer Account, as described in Prerequisites, log in to your and navigate to the Music Validation Tool:
    1. From the Amazon Developer dashboard, go to Amazon Alexa > Alexa Voice Service > Products.
    2. Click the device that you are testing.
    3. For the selected device, click the Test Tools link to access the MVT.
    User plays music via Alexa then later via a 3P MSP app
  3. For the MSP that you are planning to certify with this test run, click Enter Results to open a Performance Validation form.
User plays music via Alexa then later via a 3P MSP app
  1. On the Performance Validation form, enter the required information in the following fields:
    • Device Serial Number (DSN): Find the DSN by logging in to alexa.amazon.com with your developer account, and going to Settings > Devices, then selecting the device that you are testing. The serial number is located in the About section.
    User plays music via Alexa then later via a 3P MSP app
    • Start Time: The time that the test started execution in MM/DD/YYYY HH:MM:SS AM or MM/DD/YYYY HH:MM:SS PM format.
    • End Time: The time that the test completed execution in MM/DD/YYYY HH:MM:SS AM or MM/DD/YYYY HH:MM:SS PM format.
  2. Click Submit to generate the report.

    MVT displays a message that it is generating results. When the MVT finishes, the Result for that MSP changes on the MVT dashboard from Not Submitted to PASSED or FAILED.

    User plays music via Alexa then later via a 3P MSP app
  3. To drill down into the results report, click the date stamp for the individual test run.

    The validation results report opens as a PDF in a new browser tab.

    User plays music via Alexa then later via a 3P MSP app

Interpret the test results report

View the test results report to see how your device performed for a given MSP.

The test results report has four sections:

  • Summary
  • Utterance Validation
  • Coverage Validation
  • API Validation

Summary

The Summary section shows the validation status for each component of certification testing:

  • Utterance Validation
  • Coverage Validation
  • API Validation
User plays music via Alexa then later via a 3P MSP app

Validation status can have four possible values:

  • PASSED: Passed the specified certification rule.
  • FAILED: Failed the specified certification rule. Certification is blocked until a new test run passes this rule.
  • WARN: The API validation failed on a non-blocking API rule.
  • UNVERIFIED: The rule was not executed due to failures in previous rules.

Utterance Validator Details

The Utterance Validator verifies that all of the submitted utterances successfully played. The MVT will retry an utterance up to 3 times if the device did not receive or respond to the utterance as expected:

User plays music via Alexa then later via a 3P MSP app

Coverage Validator Details

The Coverage Validator verifies that the workflow of the test was successfully executed for each test run for a given MSP:

User plays music via Alexa then later via a 3P MSP app

API Validator Details

The API Validator validates the API implementation for a device using the logs for each music event:

User plays music via Alexa then later via a 3P MSP app