Get ASR Evaluation Results API

Call the Get ASR Evaluation Results API allows to return detailed evaluation results.

This API differs from the Get ASR Evaluation Status API in that Get ASR Evaluation Status is a lightweight API that returns high-level information, whereas Get Annotation Set ASR Metadata returns more detailed information.

To learn more about the intended API flow for Automatic Speech Recognition (ASR) Evaluation, see ASR implementation steps.

HTTP method and path

GET v1/skills/{skillId}/asrEvaluations/{evaluationId}/results HTTP/1.1
Attributes Type Description Required
skillId String Skill Id as part of the URI path. Yes
evaluationId String Evaluation Id as part of the request URI. Yes
status Enum Result status of the evaluation test case. Possible values are PASSED or FAILED. If you omit this query parameter, the results include both FAILED and PASSED evaluations. No
nextToken String Continuation token returned in response object. No
maxResults Integer Maximum number of test cases results to display. The value of this parameter has must between 1 and 1000. Default is 1000. No

Request

Request headers

Content-Type: application/json
Accept: application/json

Request body

None.

Response

HTTP status

HTTP 200 OK

Response headers

Content-Type: application/json

Response body

{
   "paginationContext": {
       "nextToken": string
   },
    "results": [
       {
          "status": enum["PASSED", "FAILED"],
          "annotation": {
              "evaluationWeight": integer,
              "expectedTranscription": string,
              "filePathInUpload": string,
              "uploadId": string,
              "audioAsset": {
                  "downloadUrl": string,
                  "expiryTime": string
              }
          },
          "output": {
              "transcription": string
          },
          "error": {
              "message": string,
              "code": string
          }
       }
    ]
}
Attributes Type Description Required
paginationContext Object Object containing pagination information. If present, the response contains incomplete evaluation results. If omitted, all evaluation results were already returned. No
paginationContext.nextToken String Use the value of nextToken in the next request as the continuation token to list the next set of objects. Yes
results Array Array containing all results. Yes
results[i].status Enum Enum indicating if the result passed or failed. Possible values are PASSED or FAILED. Yes
results[i].annotation Object Annotation object used in the current evaluation. Valid values are uploadId, filePathInUpload, evaluationWeight, expectedTranscription, and audioAsset. Yes
results[i].annotation.audioAsset Object Object containing download information for the audio used in this evaluation test case. Yes
results[i].annotation.audioAsset.downloadUrl String URL for downloading the audio file used in this evaluation test case. Yes
results[i].annotation.audioAsset.expiryTime String Timestamp for when the audio download URL expires. Yes
results[i].output Object Object containing ASR outputs. If this field is missing from the payload, an error occurred during this evaluation. The response payload contains results[i].error object with information about why the test failed. No
results[i].output.transcription String Actual transcription returned from ASR for the audio. Yes if results[i].output is present, no otherwise.
results[i].error Object Error object is present when the evaluation can't run. No
results[i].error.message String Error message describing why the evaluation against the audio file cannot run, if an error occurred. Yes if results[i].error is present, no otherwise.
results[i].error.code String error code Yes if results[i].error is present, no otherwise.

Error response

The following table lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition API Error Reference.

HTTP Status Code Error Code Scenario

404 Not Found

RESOURCE_NOT_FOUND

Requested skill doesn't exist.

Requested evaluation doesn't exist.

400 Bad Request

INVALID_REQUEST_PARAMETER

Evaluation id is malformed.

INVALID_ENUM_VALUE

Value of resultStatus query parameter is not acceptable.

PARAMTER_EXPIRED

nextToken query parameter expired.

INVALID_INTEGER_VALUE

Value of maxResults query parameter not within allowed range.

500 Internal Server Error

INTERNAL_SERVER_ERROR

Request failed due to an internal server error.

Examples

Sample Request Sample Response
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1 Accept: application/json HTTP 200 OK Content-Type: application/json { "results": [ { "status": "PASSED", "annotation": { "evaluationWeight": 1, "expectedTranscription": "hello world", "filePathInUpload": "hello world.wav", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-hello.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, "output": { "transcription": "hello world", } }, { "status": "FAILED", "annotation": { "evaluationWeight": 8, "expectedTranscription": "alexa what is the weather now", "filePathInUpload": "weather.wav", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-weather.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, "output": { "transcription": "alexa what is the weather noon" } }, }, { "status": "PASSED", "annotation": { "evaluationWeight": 1, "expectedTranscription": "alexa good job", "filePathInUpload": "good.mp3", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-good.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, "output": { "transcription": "alexa good job" } } ] }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1 Accept: application/json
HTTP 200 OK Content-Type: application/json { "results": [ { "status": "FAILED", "annotation": { "evaluationWeight": 8, "expectedTranscription": "alexa what is the weather now", "filePathInUpload": "malformed.wav", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-weather.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, error: { "message": "An unexpected error occurred" "code": "INTERNAL_SERVER_ERROR" } ] }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1 Accept: application/json
HTTP 200 OK Content-Type: application/json { "paginationContext": { nextToken": "1234-1234" }, "results": [{ "status": "PASSED", "annotation": { "evaluationWeight": 1, "expectedTranscription": "hello world", "filePathInUpload": "hello world.wav", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-hello.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, "output": { "transcription": "hello world" } }, { "status": "FAILED", "annotation": { "evaluationWeight": 8, "expectedTranscription": "alexa what is the weather now", "filePathInUpload": "weather.wav", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-weather.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, "output": { "transcription": "alexa what is the weather noon" } } ] }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1 Accept: application/json
HTTP 200 OK Content-Type: application/json { "results": [ { "status": "PASSED", "annotation": { "evaluationWeight": 1, "expectedTranscription": "alexa good job", "filePathInUpload": "good.mp3", "uploadId": "1234-1234-12341234", "audioAsset": { "downloadUrl": "https://presign.s3.aws.amazon/123412341234-good.wav", "expiryTime": "2018-10-25T08:25:04.679Z" } }, "output": { "transcription": "alexa good job" } } ] }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/results HTTP/1.1 Accept: application/json
HTTP 400 Bad Request Content-Type: application/json { "message": "Request validations failed", "violations": [{ "message": "Query parameter \"nextToken\" with value \"1234-1234\" is expired.", "code": "PARAMETER_EXPIRED" }] }