Understanding the Music Validation Tool
Amazon requires certification for devices implementing Alexa Music before publicly launching that device. Alexa Music certification ensures that the device adequately supports Music Service Providers (MSPs) such as Amazon Music, iHeartRadio, and SiriusXM. To help automate Music Certification testing, Amazon offers a Music Validation Tool (MVT). The MVT performs three different types of validation:
- Utterance Validation: Were all required utterances successfully issued to the device?
- Coverage Validation: Does the device test coverage meet every test run scenarios specified by the certification checklists?
- AVS AudioPlayer API Validation Do the reporting events sent by the device to Alexa conform to the AVS AudioPlayer API specification?
When you submit your device to Amazon for certification testing, Amazon runs the Alexa Qualification Tool (AQT) to help with music qualification testing. AQT Music testing leverages MVT by replacing all manual utterances with AQT-generated utterances and automatically invokes MVT for log review and troubleshooting reports.
To expedite Amazon's certification of your device, ensure that your device passes all of the certification checklists provided by Alexa Music before submitting your device to Amazon. Download and review each checklist for each Music Service Provider (MSP) that you plan to support:
The checklists verify that a device runs back-end events correctly to calculate royalty payments.
The Coverage Validation performed by the Music Validation Tool ensures that your device has been self-tested against a concise set of utterances and scenarios that meet the requirements specified by Alexa's supported Music Service Providers (MSPs). If your test run does not cover all required scenarios provided in the checklists, Amazon cannot accept that test run for certification. You will need to re-run the test to ensure that all test cases are accurately tested for your certification pass.
A test run that fails Coverage Validation indicates that the test did not run correctly. Some common root causes of Coverage Validation failures include the following issues:
- Missing/incomplete/incorrect/wrong sequence of utterances.
- Wait time is less than the minimum requirement.
- Utterance was not actually issued but still ran on the device.
See Music Validation Certification Requirements and Troubleshooting to learn more about the causes and resolutions for common test failures.
AVS AudioPlayer API validation
AVS AudioPlayer API Validation validates that all API events implemented by the device or device companion app run in the correct order. This validation has the following components:
- Validate the event sequence and their response to directives of the AudioPlayer State machine.
- Validate that the playback reporting events are sent within acceptable time windows and that the reported offsets are correct.
- Validate implementation of the ProgressReportIntervalElapsed and ProgressReportDelayElapsed events.
For assistance troubleshooting API Validation failures, see the Music Validation Certification Requirements and Troubleshooting.
Last updated: Dec 17, 2020