Improve your Automatic Speech Recognition (ASR) Test Results
The Automatic Speech Recognition (ASR) Evaluation tool allows you to batch test audio files to measure the ASR accuracy of the skills that you've developed. You can use the feedback from your evaluation reports to modify your sample utterances and improve your interaction model and ASR accuracy.
You need the following items to run an ASR evaluation:
- An Amazon developer account.
To create an account, see Create Your Amazon Developer Account.
- An existing Evaluation Report generated by running the ASR tool on an annotation set.
To learn how to generate an Evaluation Report, see Run an Automatic Speech Recognition (ASR) Evaluation.
Recommendations to improve ASR accuracy
Review and apply proper grammar rules to improve both the interaction model and ASR accuracy for a skill. Try to have your utterances incorporate the following guidelines when creating the utterances for your annotation set:
- Homophones – Correctly use words that sound similar to each other but have different meanings.
- Exhaustive in-skill grammar – The more extensive vocabulary and grammar that you have programmed into your skill, the more your interaction model and accuracy should improve.
In addition to these guidelines, two common ways to improve ASR accuracy are to map an intent value or a slot value to a failing utterance.
Map an intent value or slot value to improve accuracy
One way to improve accuracy is to add an utterance to an intent.
To add sample utterances to an intent
- Navigate to the ASR Evaluation that you ran as described in Run an Automatic Speech Recognition (ASR) Evaluation.
In the Actions column for one of the failed utterances, click the Map button:
An Map Utterance prompt opens:
From the Map Utterance prompt, click either the Intent tab or Slot tab to choose your mapping target.
- From the Intent tab, select the intent from the Intent drop-down menu, and click the Add Utterance button.
- From the Slot tab, select the Slot Type and then add part of the utterance as a new value or as a synonym for an existing value.
At the top of the page, click the Build Model button to build your updated interaction model.
- Follow the instructions in Run an Automatic Speech Recognition (ASR) Evaluation to re-run the ASR evaluation and hopefully see improved results.