Calibrating Your Setup

Calibrate your acoustic testing setup so that both your test environment will produce similar test results to the Amazon test environment.

Before testing, follow the steps in this section to gather all facts about your laptop volume levels. Then, when you're ready for each full acoustic test, set your laptop volume levels according to the facts you've gathered from your effort in this section. The details that you gather are primarily for you, serving as an aid in repeatability of your tests and in any debugging of your audio setup.

Measuring the ambient sound/noise floor of the test environment

The sound level meter (SLM) has a setting of "Slow" or "Fast." The individual tests describe which setting to use. The SLM also has a choice of "A" or "C" weighting for dB measurements. For all Alexa acoustic tests, use the "C" setting to gather dBC measurements.

To assess the ambient sound:

  1. Open the SPL tab of the Scoresheet.
  2. Prepare the sound level meter:
    • Choose the Slow setting.
    • Choose the C setting for measuring in dBC.
  3. Make the test environment as quiet as possible. With the Speech Speaker in location 4, and the Noise Speaker in place, hold the SPL wand next to the center of the microphone array of the DUT until the SPL value stabilizes over 10 seconds.
  4. Record the SPL value in the cell at the intersection of ambient noise and SPL dBC.

Control volume from each laptop

For each of your Speech Speaker/Laptop and Noise Speaker/Laptop setups, three volume controls are important:

  • Speaker volume control
  • Media player volume control for playing audio files
  • Laptop volume control

Make the laptop volume control your main volume control, as described in the following steps:

  1. With the speaker power on, turn the volume of the speaker to its minimum.
  2. Load an audio file in a media player on the laptop.
  3. If the audio file is playing, stop it.
  4. Without muting the laptop volume, reduce the laptop volume to its minimum value or position to avoid someone muting/unmuting when the volume is mistakenly at a high setting.
  5. Set the speaker volume to about 75 percent of maximum.
  6. Set the volume of the media player with the file not playing to the maximum.
  7. With the laptop volume still minimized, start playing the audio file.
  8. Raise the laptop volume until you hear the audio file.

Establishing volume levels for each test

The following sections describe the required speaker settings for each noise condition.

Speech speaker for silence

For testing device response to utterances in a silent setting, set the Speech Speaker deliver 63 dBC.

  1. Set the SLM to Slow.
  2. Place the SLM on a stand 1m away from the Speech Speaker. If necessary, temporarily move the speaker from its final setup. Remember to mark the position with masking tape.
  3. Play the Calibration WAV file through the Speech Speaker; adjust the Speech Laptop volume level until the SPL value is 63 dBC.
  4. In the Scoresheet for Speech Speaker for silence, document the Speech Laptop volume level in the Volume setting value column of the SPL table.

Speech speaker for noise

For testing device response to utterances in a noise condition, set the Speech Speaker to deliver 70 dBC, independent of the volume level for the Noise Speaker.

  1. Make sure that the Noise Speaker isn't playing anything.
  2. Set the sound level meter to "Slow."
  3. Place the SLM on a stand 1m away from the Speech Speaker.
  4. Play the calibration audio file through the Speech Speaker; adjust the Speech Laptop volume level until the SPL value is 70 dBC.
  5. In the Scoresheet for Speech Speaker for noise, document the Speech Laptop volume level in the Volume setting value column of the SPL table.

Noise speaker for stationary noise

For testing under the stationary noise condition, set the the Noise Speaker to deliver 64 dBC.

  1. Make sure that no sound plays through the Speech Speaker.
  2. Set the sound level meter to Slow.
  3. Hold the SLM near the microphone array of the DUT, pointed at the noise speaker. For the Echo Dot, hold the SLM 50mm over the center of the device.
  4. Play the calibration audio file through the Noise Speaker; adjust the Noise Laptop volume level until the SPL value is 64 dBC.
  5. In the Volume setting value column for Noise Speaker for (stationary) noise, document the Noise Laptop volume level.

Noise speaker for external music

For testing under the external music condition, set the Noise Speaker to deliver 64 dBC.

  1. Make sure that no sound plays through the Speech Speaker.
  2. Set the sound level meter to "Slow."
  3. Hold the SLM near the microphone array of the DUT, pointed at the noise speaker.
  4. Play the calibration audio file through the Noise Speaker; adjust the Noise Laptop volume level until the SPL value is 64 dBC (+/- 1dB).
  5. In the Volume setting value column for Noise Speaker for external music, document the Noise Laptop volume level.

Device playback for the DUT

For testing under the condition of music playback through the DUT, set the DUT to deliver 64 dBC.

  1. Make sure that no sound plays through the Speech Speaker or Noise Speaker.
  2. Set the sound level meter to Fast.
  3. Set up the SLM in a stand (or hold it 1m from the DUT), pointed at the DUT. If the Echo Dot is active, turn off its microphones.
  4. To the DUT, say, "Alexa, play my Test playlist." To create the Test playlist, see Set up your Amazon Music account.
  5. To the DUT, say, "Alexa, skip forward one minute and 45 seconds"; you should calibrate after the background singers say "Because I'm" at the 2:02 mark of the song.

  6. Adjust the volume of device playback until the SPL value is as close to 64 dBC as possible.

    For example, say "Alexa, volume 3" or "Alexa, volume 5", or "Alexa, volume 7", and so on, until you find a suitable volume for music playback. Due to device volume step sizes, 64 dBC might not be achievable. Note the value in the scorecard.

  7. In the Volume setting value column for device playback - DUT, document the volume level of the device.

Device playback for the Echo Dot

For testing under the condition of music playback through the DUT, set the DUT to deliver 64 dBC.

  1. Make sure that no sound plays through the Speech Speaker or Noise Speaker.
  2. Set the sound level meter to Fast.
  3. Set up the SLM in a stand, or hold it 1m from the DUT, pointed at the DUT.
  4. To the DUT, say, "Alexa, play my Test playlist." To create the Test playlist, see Set up your Amazon Music account.
  5. To the DUT, say, "Alexa, skip forward one minute and 45 seconds"; you should calibrate after the background singers say "Because I'm" at the 2:02 mark of the song.

  6. Adjust the volume of device playback until the SPL value is as close to 64 dBC as possible.

    For example, say "Alexa, volume 3" or "Alexa, volume 5", or "Alexa, volume 7", and so on, until you find a suitable volume for music playback. Due to device volume step sizes, 64 dBC might not be achievable. Note the value in the scorecard.

  7. In the Volume setting value column for device playback - Echo Dot, document the volume level of the device.

Now that you've acquired the recommended Equipment and Materials, finished Setting up your Test Environment, and calibrated your setup, you're ready to test for Wake Word Detection Delay, False Rejection Rate and Response Accuracy Rate, and False Alarm Rate, and Imposter Acceptance Rate. If you still have questions about your setup, contact your Amazon representative for guidance.

Map dBC to dBA values for calibration

To help you complete the noise level entries on the Testing scoresheet, use the the following table, which shows calibration values for both dBC and dBA.

The following table correlates the Amazon dBC value to the expected dBA values for typical acoustic testing. If you switch the SPL meter from reading dBC to dBA, you should see the dBA values shown in the following table without making any other changes.

Device Undergoing Calibration Calibration Level dBC Calibration Level dBA Scenario Calibration File SLM Location
DUT/Echo Dot
(If the DUT is a Near-Field device that doesn't reach the required levels, calibrate to the max reachable level of the DUT.)		 64dBC +/-
0.7dB at “FAST” setting		 n/a FRR/ RAR playback “Happy” song.
Calibrate on the chorus (2min 02s – 2min 41s) after the singer says “Because I’m”.

Use “Alexa, skip 90 seconds” to get close to the calibration chorus.		 1m in front of DUT with setting to FAST
Noise speaker (location 1) 64dBC +/- 0.7dB at “SLOW” setting 62dBA +/- 0.7dB at “SLOW” setting Stationary noise and external music Calibration WAV file Above the DUT (at same height as noise speaker, i.e. 1m). Unplug the DUT during this process to prevent potential interference with setting to SLOW.
Speech speakers (locations 2, 3, 4, and 5) 70dBC +/- 0.7dB at “SLOW” setting 65dBA +/- 0.7dB at “SLOW” setting Noise Calibration WAV file 1m in front of each speaker (calibrate all four speech speakers) with setting to SLOW.
Speech speakers (locations 2, 3, 4, and 5) 63dBC +/- 0.7dB at “SLOW” setting 58dBA +/- 0.7dB at “SLOW” setting Silence Calibration WAV file 1m in front of each speaker (calibrate all four speech speakers) with setting to SLOW.

Amazon requires that a test room has a noise floor with less than 35 dBSPL (A). See Test room configuration. If the noise floor in your test room is <35 dBSPL (A) and also measures >60 dBSPL (C), then consider using dBA values, which are A-weighted target values as an alternative to the dBC values for calibration.

Amazon Alexa
Amazon Appstore
Amazon Web Services
Amazon Dash Replenishment