Equipment and materials
To get started with acoustic testing for your device, you'll need the recommended equipment, account setup, and audio source files described in this document. The following guidance is based on the type of test room environment Amazon will use to test your device.
To create an effective test room and the run the tests for Wake Word Detection Delay, False Rejection Rate and Response Accuracy Rate, and False Alarm Rate, you'll need the devices, speakers, stands, laptops, and cables described below.
Two Alexa devices:
- Your device, the Device Under Test (DUT)
- An Echo Dot
Two speakers for playing audio files: One will be the designated Speech Speaker, the other will be the Noise Speaker. Each speaker should have these characteristics:
- A flat frequency response from 80 Hz to 15 kHz range
- A single driver
- <= 0.05% THD at 1 kHz at maximum volume (e.g. 1 W output)
A speaker which meets these requirements is the Fostex 6301NE Powered Active Monitor (Single), Electronically Balanced.
Two adjustable-height speaker stands: Whatever is needed to hold the DUT and Echo Dot at the typical use case elevation of the DUT.
- One to elevate the Speech Speaker to the elevation of the microphone array of the DUT.
- One to elevate the Noise Speaker to a fixed height of 1 m.
- The "Speech Laptop" for playing audio via an audio cable to the Speech Speaker
- The "Noise Laptop" for playing audio via an audio cable to the Noise Speaker
- The "Personal Laptop" for gathering test results
Two audio line-out cables: One to connect the Speech Laptop to the Speech Speaker, and one to connect the Noise Laptop to the Noise Speaker. Each speaker should have these characteristics:
- Male to male: 3.5 mm (1/8") to 6.35 mm (1/4")
- Length: >= 3 m
One Sound Pressure Level (SPL) meter: A brand we recommended is the General Tools DSM403SD Sound Level Meter with Data Logging SD Card, Class 1 Sound.
- A table to hold the laptops
- A chair for the test facilitator
- A tape measure and protractor (to accurately place speakers relative to the DUT)
- Masking tape (to mark the locations of devices and speakers)
Select the Excel file that corresponds to the testing you'll be doing. Place this file on the Personal Laptop described in "Equipment." This file is referred to as the "Scoresheet." It is a spreadsheet where the test facilitator enters facts.
- Far-field testing: Download the Alexa Acoustic Far-field Testing Scoresheet.
- Near-field testing: Download the Alexa Acoustic Near-field Testing Scoresheet.
Audio source files
Zips with WAV files: Download the zip files for each country of interest
- Go to the Resources section of the Developer Console. Work with your Amazon Business Development representative to gain access for your developer account.
- Find the section "AVS acoustic audio files" and click view.
- Download the zip files of interest.
- Extract and place the folders as follows:
- Put these folders on the Speaker Laptop: FRR_RAR_utterances, WWDD_utterances
- Put this folder on the Noise Laptop: FARTest
Noise files: The noise and calibration files can be found in the "AVS acoustic audio files" folder (see above), in the zip file named "Acoustic_Noise_and_Calibration_Files.zip"
.wav should be used as the (stationary) noise file
.wav should be used as the external music file
.wav should be used for calibration in (stationary) noise and external music testing
Download the (stationary) noise file from the Resources tab and make this available for playback on the Noise and Laptop speaker.
Music (audio) account setup
For some of the tests in this guide, you'll play music from the Noise Laptop through the Noise Speaker. For other tests, you'll ask the device to play a particular music playlist through the DUT.
To be able to perform the tests that require music audio, you need an Amazon account registered in each country for which you are testing the DUT. In this account, update your settings to specify a billing address and payment method appropriate for a given country. For example, for Germany, create an Amazon.de account, add a German billing address, and add a 1-click payment method issued by a bank in Germany.
Install Amazon Music: On the Noise Laptop, you need to download and install the (free) Amazon Music app:
- Visit the Amazon web site for the country of interest
- In the search box, type Amazon Music
- Accept the option to add the (free) Amazon Music app to your cart, and complete the transaction
- Complete installation of the app
Create a playlist: In this Amazon account, purchase the song "Happy" and add it to a new playlist named "Test":
- Launch the Amazon Music app on the laptop.
- Log in to the app with the Amazon account.
- Select Store at the top.
- In the search box, type "Happy by Pharrell Williams".
- Select the matching music result and purchase the song. As a result, the price is replaced with the words "In library".
- Select My Music at the top. As a result, you'll see the "Happy" song.
- To the right, locate the option to create a playlist. Select Create New and name the playlist "Test". As a result, "Test" appears under the "My Playlists" heading.
- Make "Happy" appear five times in the Test playlist. Click-drag "Happy" over to the word "Test" and release. Do this a total of five times.
- To verify "Happy" appears at least five times in your "Test" playlist, click "Test" and view the list of songs in that playlist.
You will ask Alexa to play this playlist during every test that requires "device playback" of music through the DUT.
Play the playlist: On the Noise Laptop, you need to be able to play external music via your laptop media player:
- On the laptop, launch the Amazon Music app.
- Log in with the Amazon account described above.
- With the music file visible in the My Music view, click Download.
- Verify that you are able play this downloaded music from the laptop through the Noise Speaker.
Use these equipment and materials during every test that requires "external music" as the noise condition. Next, learn to set up your acoustic test room environment.