Equipment and materials

To get started with acoustic self-testing Alexa Voice Service (AVS) functionality for your device, you'll need the recommended equipment, account setup, and audio source files described on this page. Use these recommendations to set up a test room environment that replicates the one that Amazon uses to test prospective Alexa Built-in devices.

Recommended equipment

To create an effective test room and run the tests for Wake Word Detection Delay, False Rejection Rate and Response Accuracy Rate, False Alarm Rate, and Test for Imposter Acceptance Rate (IAR), gather the following devices, speakers, stands, laptops, and cables:

  • Alexa devices – Testing requires two Alexa devices:
    • Your device, the Device Under Test (DUT)
    • An Echo Dot
  • Two speakers – Acoustic self-testing requires two speakers. One speaker is the designated Speech Speaker; the other speaker is the Noise Speaker. Each speaker should have the following characteristics:
    • A flat frequency response from 80 Hz to 15 kHz range
    • A single driver
    • <= 0.05 percent THD at 1kHz at maximum volume, 1W output

      One speaker that meets Amazon requirements is the Fostex 6301NE Powered Active Monitor (Single), Electronically Balanced.

  • Two adjustable-height speaker stands: Use any form of speaker stand to hold the DUT and Echo Dot at the appropriate test case elevation of the DUT.
    • One to elevate the Speech Speaker to the elevation of the microphone array of the DUT.
    • One to elevate the Noise Speaker to a fixed height of 1m.
  • Three laptops:
    • Speech Laptop: To play audio through an audio cable to the Speech Speaker.
    • Noise Laptop: To play audio through an audio cable to the Noise Speaker.
    • Personal Laptop: To record test results.
  • Two audio line-out cables: One to connect the Speech Laptop to the Speech Speaker, and one to connect the Noise Laptop to the Noise Speaker. Each speaker should have these characteristics:
    • Male to male: 3.5mm (1/8") to 6.35mm (1/4")
    • Length: >= 3m
  • One Sound Pressure Level (SPL) meter: Amazon recommends the General Tools DSM403SD Sound Level Meter with Data Logging SD Card, Class 1 Sound.
  • Miscellaneous equipment:
    • A table to hold the laptops
    • A chair for the test facilitator
    • A tape measure and protractor (to accurately place speakers relative to the DUT)
    • Masking tape to mark the locations of devices and speakers

Alexa Acoustic Testing Scoresheet

Record your test results on the AVS self-test checklist, which is available from the Resources tab on the AVS Developer Console.

  1. Log in to the Amazon developer console with your Amazon developer account credentials:
  2. Click the following link to access the appropriate checklists and scoresheets:

    Alexa Acoustic Testing Scoresheet

  3. From the list of checklists and scoresheets, locate the appropriate scoresheets for your tests:

    • Close Talk Testing Scoresheet – Alexa_Acoustic_Testing_Close_Talk_v*.zip
    • Far Field and Near Field Scoresheet – Alexa_Acoustic_Testing_Scoresheet_Far_Field_Near_Field_v*.xlsx
    • Push to Talk Scoresheet – Alexa_Acoustic_Testing_Scoresheet_Push_To_Talk_v*.xlsx
  4. Copy the appropriate scoresheet files to the Personal Laptop described in Recommended equipment.

Audio source files

Amazon provides a set of audio source files for you to use in your acoustic self-testing. Download and copy these files to the appropriate laptops and folders, as described:

  • Zips with WAV files: Download the zip files for each country that you plan to support:
    1. Navigate to the Resources tab of the Developer Console. Work with your Amazon Business Development representative to gain access for your developer account.
    2. Locate the AVS acoustic audio files section, and click view.
    3. Download the appropriate zip files.
    4. Extract and place the folders as follows:
      • Copy the FRR_RAR_utterances, WWDD_utterances, and if applicable, the MobileTest folders to the Speaker Laptop.
      • Copy the FARTest folder to the Noise Laptop.
  • Noise files: Locate the noise and calibration files in the AVS acoustic audio files folder, in the zip file named "Acoustic_Noise_and_Calibration_Files.zip"
    • **Noise_.wav**: Stationary noise file. Download the Stationary noise file from the **Resources** tab and make the file available for playback on the Noise and Laptop speaker.
    • **Music_.wav**: External music file.
    • **Calibration_.wav**: Calibration file for stationary noise and external music testing.

Set up your Amazon Music account

Some of the required acoustics tests instruct you to play music from the Noise Laptop through the Noise Speaker. Other tests instruct you to ask the device to play a particular music playlist through the DUT.

To be able to perform the tests that require music audio, you need an Amazon account registered in each country for which you're testing the DUT. Make sure that your account settings specify a billing address and payment method appropriate for the given country. For example, for Germany, create an Amazon.de account, add a German billing address, and add a 1-click payment method issued by a bank in Germany.

  1. Download and install the free Amazon Music app to the Noise Laptop:
    1. Visit the Amazon web site for the country of interest.
    2. In the search box, enter Amazon Music.
    3. Accept the option to add the free Amazon Music app to your cart, and complete the transaction.
    4. Complete installation of the app.
  2. From your Amazon account, purchase the song "Happy", create a new playlist named "Test", and copy the song over to that playlist. Use this "Test" playlist during for every test that requires "device playback" of music through the DUT.
    1. Launch the Amazon Music app on the laptop.
    2. Log in to the app with the Amazon account.
    3. Select Store at the top.
    4. In the search box, enter "Happy by Pharrell Williams".
    5. Select the matching music result and purchase the song to add it to your music library.
    6. Select My Music at the top to display the song "Happy" in your library.
    7. To the right, locate the option to create a playlist. Select Create New and name the playlist "Test".
    8. Make "Happy" appear five times in the Test playlist. Click-drag "Happy" over to the "Test" playlist. Repeat at least four more times.
    9. To verify "Happy" appears at least five times in your "Test" playlist, click "Test" and view the list of songs in that playlist.
  3. On the Noise Laptop, check that you're able to play external music through your laptop media player:
    1. On the laptop, launch the Amazon Music app.
    2. Log in with your Amazon account.
    3. With the music file visible in the My Music view, click Download.
    4. Verify that you can play the music that you downloaded from the laptop through the Noise Speaker.

Use the described equipment and materials for every test that requires "external music" as the noise condition. Next, learn to set up your acoustic test room environment.