Alexa Acoustic Testing Guide
To keep the mutual trust of our customers for the expected Alexa experience, the Alexa Voice Service (AVS) testing team verifies the performance of acoustic systems for Alexa Built-in devices. Acoustic performance is critical to make sure that wake word recognition performs as expected and that a device handles user requests appropriately.
The following sections describe how to conduct your own acoustic testing in a way that emulates how Amazon tests devices, including the following information:
Amazon also provides scoresheets and audio files to create the speech and noise conditions for executing the acoustic tests.
Common terms
The following image breaks down the structure of a typical Alexa utterance:
Definitions
| Term | Definition |
|---|---|
| Utterance | An expression spoken to an Alexa Built-in device. |
| Wake Word | The first word of the utterance, usually "Alexa", which signals an Alexa Built-in device to listen for the rest of the utterance. |
| Request | The segment of the utterance after the wake word. |
| Device Under Test (DUT) | The physical device being tested. |
Passing criteria
Before submitting the DUT to Amazon for evaluation, your own testing must verify that the DUT has passed the criteria defined in the Alexa Acoustic Testing User Guide: Download the Alexa Acoustic Testing UserGuide.
Automatic Speech Recognition (ASR) profiles
For a given test case, the physical DUT placement depends on the typical distance from the mouth of a user to the device microphone. This placement is also referred to as the Automatic Speech Recognition (ASR) profile of the device. The applicable sound or speech sources for a test also depend on the ASR.
Amazon tests three ASR profiles:
- Close-talk: 0 to 1 ft (0.3 m)
- Near-field: 0 to 3 ft (0.9 m)
- Far-field: 0 to 9 ft (2.75 m)
For general discussion on ASR, see Audio Hardware Configurations.
Room requirements
To perform acoustic self-testing for your device, use a semi-anechoic chamber for testing. If a semi-anechoic chamber is unavailable, these testing techniques are also applicable in typical, well-defined, office rooms. To prepare your acoustic environment, or playback room, meet the following requirements.
Dimensions
Room size: At least 2.2 m high.
Room treatment:
- Reflection damping – Equip any office-like rooms with some acoustical damping to meet the target reverb times
- Reverberation time – Less than 0.7 s and higher than 0.2 s; between 125 Hz and 8 kHz
Noise floor:
- To reduce the influence of external noise, the noise floor measured in a room must be less than 35 dBSPL (A).
- The room should meet the ETSI standard and be less than 30 dBSPL (A).
Space around equipment:
- There must be at least a 0.5 m clearance between the walls and the test speakers
- There must be at least a 0.5 m clearance between the walls and the DUT
For an aerial view of this configuration, see Set up your Test Environment.
About the Amazon acoustic test chamber
To help you plan a sustainable testing environment, Amazon uses the following characteristics for test rooms for our acoustic testing.
Amazon room size:
| Current | Future | |
|---|---|---|
| External dimensions | 4.6 m x 6.1 m x 2.7 m | 6.1 m x 7.6 m x 2.7 m |
| Internal dimensions | 4.4 m x 5.9 m x 2.4 m | 5.9 m x 7.4 m x 2.4 m |
Amazon room treatment:
- Ceiling track with acoustic curtains on three walls
- Carpet tiles
- Drop acoustic ceiling and ceiling panels
- Corner bass trap panels
- 10 Hz neoprene isolators
- Reverberation time: Less than 0.7 s and higher than 0.2 s; between 125 Hz and 8 kHz
Amazon noise floor: Less than 35 dBSPL (A) given an outside noise of 70 dBSPL (A).
Testing process
Use the instructions in the following sections to conduct each test described. Your DUT must pass all required tests before submitting your test results to Amazon.
Acoustic testing setup
For successful testing, review our guidance and emulate the conditions described at each step.
- Review the recommended Equipment and Materials
- Learn how to Set up your Test Environment
- Learn to Calibrate your Setup
How to test
As appropriate for your device, conduct your acoustic self testing before you submit your device for certification. The Alexa Acoustic Testing User Guide describes the ideal test room configuration, test files, test steps, and how to capture your metrics. Download the AVS Test Setup and Configuration guide to review additional test setup guidance.
Submit your results
When your metrics illustrate that the DUT meets all criteria for passing, email the results to AVS-Certification-Labs@amazon.com. After Amazon reviews and verifies your results, you'll receive instructions on how to complete the certification process. You will need to ship devices to either Amazon or one of the Authorized Third-Party Labs to complete the final certification tests. For more details, review our full Testing and Certification Process.
