Improve your Automatic Speech Recognition (ASR) Test Results
The Automatic Speech Recognition (ASR) Evaluation tool allows you to batch test audio files to measure the ASR accuracy of the skills that you've developed. The main benefit of the tool is to use the feedback from your evaluation reports to improve the ASR performance for your skill. Use the report results to modify your sample utterances to improve your skill's interaction model and accuracy.
You'll need the following items to run an ASR evaluation:
- An Amazon developer account. See developer.amazon.com to create your account, if necessary.
- An existing Evaluation Report generated by running the ASR tool on an annotation set. See Run an Automatic Speech Recognition (ASR) Evaluation to learn how to generate an Evaluation Report.
Suggestions for improving ASR accuracy
Review and apply proper grammar rules to improve both the interaction model and ASR accuracy for a skill. Try to have your utterances incorporate the following guidelines when creating the utterances for your annotation set:
- Homophones – Correctly use words that sound similar to each other but have different meanings.
- Exhaustive in-skill grammar – The more extensive vocabulary and grammar that you have programmed into your skill, the more your interaction model and accuracy should improve.
In addition to these guidelines, two common ways to improve ASR accuracy are to map an intent value or a slot value to a failing utterance.
Map an intent value or slot value to improve accuracy
One way to improve accuracy is to add an utterance to an intent.
To add sample utterances to an intent
- Navigate to the ASR Evaluation that you ran as described in Run an Automatic Speech Recognition (ASR) Evaluation.
In the Actions column for one of the failed utterances, click the Map button:
An Map Utterance prompt opens:
From the Map Utterance prompt, click either the Intent tab or Slot tab to choose your mapping target.
- From the Intent tab, select the intent from the Intent drop-down menu, and click the Add Utterance button.
- From the Slot tab, select the Slot Type and then add part of the utterance as a new value or as a synonym for an existing value.
At the top of the page, click the Build Model button to build your updated interaction model.
- Follow the instructions in Run an Automatic Speech Recognition (ASR) Evaluation to re-run the ASR evaluation and hopefully see improved results.