Get Automatic Speech Recognition (ASR) Evaluation Results API
Call this API allows to return detailed Automatic Speech Recognition (ASR) evaluation results.
This call differs from the Get Automatic Speech Recognition (ASR) Evaluation Status API in that Get ASR Evaluation Status is a lightweight API that returns high-level information, whereas Get Annotation Set ASR Metadata returns more detailed information.
To learn more about the intended API flow for ASR Evaluation, see API Flow.
HTTP method and path
GET v1/skills/{skillId}/asrEvaluations/{evaluationId}/results HTTP/1.1
| Attributes | Type | Description | Required |
|---|---|---|---|
| skillId | String | Skill Id as part of the URI path. | Yes |
| evaluationId | String | Evaluation Id as part of the request URI. | Yes |
| status | Enum | Result status of the evaluation test case. Possible values are PASSED or FAILED. If you omit this query parameter, the results include both FAILED and PASSED evaluations. | No |
| nextToken | String | Continuation token returned in response object. | No |
| maxResults | Integer | Maximum number of test cases results to display. The value of this parameter has must between 1 and 1000. Default is 1000. | No |
Request
Request headers
Content-Type: application/json
Accept: application/json
Request body
None.
Response
HTTP status
HTTP 200 OK
Response headers
Content-Type: application/json
Response body
{
"paginationContext": {
"nextToken": string
},
"results": [
{
"status": enum["PASSED", "FAILED"],
"annotation": {
"evaluationWeight": integer,
"expectedTranscription": string,
"filePathInUpload": string,
"uploadId": string,
"audioAsset": {
"downloadUrl": string,
"expiryTime": string
}
},
"output": {
"transcription": string
},
"error": {
"message": string,
"code": string
}
}
]
}
| Attributes | Type | Description | Required |
|---|---|---|---|
| paginationContext | Object | Object containing pagination information. If present, the response contains incomplete evaluation results. If omitted, all evaluation results were already returned. | No |
| paginationContext.nextToken | String | Use the value of nextToken in the next request as the continuation token to list the next set of objects. | Yes |
| results | Array | Array containing all results. | Yes |
| results[i].status | Enum | Enum indicating if the result passed or failed. Possible values are PASSED or FAILED. | Yes |
| results[i].annotation | Object | Annotation object used in the current evaluation. Valid values are uploadId, filePathInUpload, evaluationWeight, expectedTranscription, and audioAsset. | Yes |
| results[i].annotation.audioAsset | Object | Object containing download information for the audio used in this evaluation test case. | Yes |
| results[i].annotation.audioAsset.downloadUrl | String | URL for downloading the audio file used in this evaluation test case. | Yes |
| results[i].annotation.audioAsset.expiryTime | String | Timestamp for when the audio download URL expires. | Yes |
| results[i].output | Object | Object containing ASR outputs. If this field is missing from the payload, an error occurred during this evaluation. The response payload contains results[i].error object with information about why the test failed. | No |
| results[i].output.transcription | String | Actual transcription returned from ASR for the audio. | Yes if results[i].output is present, no otherwise. |
| results[i].error | Object | Error object is present when the evaluation can't run. | No |
| results[i].error.message | String | Error message describing why the evaluation against the audio file cannot run, if an error occurred. | Yes if results[i].error is present, no otherwise. |
| results[i].error.code | String | error code | Yes if results[i].error is present, no otherwise. |
Error response
The following table lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.
| HTTP Status Code | Error Code | Scenario |
|---|---|---|
|
404 Not Found |
RESOURCE_NOT_FOUND |
Requested skill doesn't exist. |
|
Requested evaluation doesn't exist. |
||
|
400 Bad Request |
INVALID_REQUEST_PARAMETER |
Evaluation id is malformed. |
|
INVALID_ENUM_VALUE |
Value of resultStatus query parameter is not acceptable. |
|
|
PARAMTER_EXPIRED |
nextToken query parameter expired. |
|
|
INVALID_INTEGER_VALUE |
Value of maxResults query parameter not within allowed range. |
|
|
500 Internal Server Error |
INTERNAL_SERVER_ERROR |
Request failed due to an internal server error. |
Examples
| Sample Request | Sample Response |
|---|---|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"results": [
{
"status": "PASSED",
"annotation": {
"evaluationWeight": 1,
"expectedTranscription": "hello world",
"filePathInUpload": "hello world.wav",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-hello.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
"output": {
"transcription": "hello world",
}
},
{
"status": "FAILED",
"annotation": {
"evaluationWeight": 8,
"expectedTranscription": "alexa what is the weather now",
"filePathInUpload": "weather.wav",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-weather.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
"output": {
"transcription": "alexa what is the weather noon"
}
},
},
{
"status": "PASSED",
"annotation": {
"evaluationWeight": 1,
"expectedTranscription": "alexa good job",
"filePathInUpload": "good.mp3",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-good.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
"output": {
"transcription": "alexa good job"
}
}
]
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"results": [
{
"status": "FAILED",
"annotation": {
"evaluationWeight": 8,
"expectedTranscription": "alexa what is the weather now",
"filePathInUpload": "malformed.wav",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-weather.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
error: {
"message": "An unexpected error occurred"
"code": "INTERNAL_SERVER_ERROR"
}
]
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"paginationContext": {
nextToken": "1234-1234"
},
"results": [{
"status": "PASSED",
"annotation": {
"evaluationWeight": 1,
"expectedTranscription": "hello world",
"filePathInUpload": "hello world.wav",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-hello.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
"output": {
"transcription": "hello world"
}
},
{
"status": "FAILED",
"annotation": {
"evaluationWeight": 8,
"expectedTranscription": "alexa what is the weather now",
"filePathInUpload": "weather.wav",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-weather.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
"output": {
"transcription": "alexa what is the weather noon"
}
}
]
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"results": [
{
"status": "PASSED",
"annotation": {
"evaluationWeight": 1,
"expectedTranscription": "alexa good job",
"filePathInUpload": "good.mp3",
"uploadId": "1234-1234-12341234",
"audioAsset": {
"downloadUrl": "https://presign.s3.aws.amazon/123412341234-good.wav",
"expiryTime": "2018-10-25T08:25:04.679Z"
}
},
"output": {
"transcription": "alexa good job"
}
}
]
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1
Accept: application/json
|
HTTP 400 Bad Request
Content-Type: application/json
{
"message": "Request validations failed",
"violations": [{
"message": "Query parameter \"nextToken\" with value \"1234-1234\" is expired.",
"code": "PARAMETER_EXPIRED"
}]
}
|
Related topics
- What is Automatic Speech Recognition?
- Create an Annotation Set for Automatic Speech Recognition (ASR)
- Run an Automatic Speech Recognition (ASR) Evaluation
- Improve your Automatic Speech Recognition (ASR) Test Results
- Create annotation set API
- Update annotation set annotations API
- Post ASR evaluation API
- Create catalog API
- Batch Test Your Natural Language Understanding (NLU) Model
