List Automatic Speech Recognition (ASR) Evaluations API
Call this API to return historical Automatic Speech Recognition (ASR) evaluations. The returned results display the latest evaluation run displays first.
HTTP method and path
GET /v1/skills/{skillId}/asrEvaluations?locale={locale}&annotationSetId={annotationSetId}&stage={stage}&nextToken={nextToken}&maxResults={maxResults} HTTP/1.1
| Attributes | Type | Description | Required |
|---|---|---|---|
| skillId | String | skill id | Yes |
| annotationSetId | String | Filters results to the specified annotation set id. If omitted, the response includes all evaluations regardless of which annotation set was used in the evaluation. | No |
| locale | String | Locale in bcp 47 format. Filters results to the specified locale. If omitted, the response would include all evaluations regardless of what locale was used in the evaluation. | No |
| stage | Enum | Used to filter results with the specified skill stage. Possible values are development or live. If omitted, the response would include all evaluations regardless of what skill stage was used in the evaluation.
|
No |
| nextToken | String | String. Continuation token returned in response object of last list skills response. | No |
| maxResults | Integer | Integer. Maximum number of test cases results to display. The value of this parameter must between 1 and 1000. Default is 1000. | No |
Request
Request Headers
Content-Type: application/json
Accept: application/json
Request Body
None.
Response
HTTP Status
HTTP 200 OK
Response Headers
Content-Type: application/json
Response Body
{
"paginationContext": {
"nextToken": string
},
"evaluations": [
{
"id": string,
"status": enum["IN_PROGRESS", "COMPLETED", "FAILED"],
"totalEvaluationCount": integer,
"completedEvaluationCount": integer,
"startTimestamp": string,
"request": {
"skill": {
"stage": string,
"locale": string
},
"annotationSetId": "id": string
},
"result": {
"status": enum["PASSED", "FAILED"],
"metrics": {
"overallErrorRate": double
}
},
"error": {
"message": string,
"code": string
}
}
]
}
| Attributes | Type | Description | Required |
|---|---|---|---|
| evaluations | Array | Array containing all evaluations that you have ever run, based on the filter criteria defined in the request. | Yes |
| evaluations[i] | Object | Information about the high level evaluation details. The structure is identical to the response structure of Get ASR Evaluation Status API<. | Yes |
| paginationContext | Object | Object containing pagination information. If present, the response contains incomplete annotationSets. If omitted, all annotationSets were already returned. | No. |
| paginationContext.nextToken | String | Use the value of nextToken in the next request as the continuation token to list the next set of objects. | Yes |
Error Response
This section lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.
| HTTP Status Code | Error Code | Scenario |
|---|---|---|
|
404 Not Found |
RESOURCE_NOT_FOUND |
Requested skill does not exist |
|
400 Bad Request |
INVALID_REQUEST_PARAMETER |
Locale query parameter is malformed |
|
Stage query parameter is malformed |
||
|
The nextToken query parameter is malformed |
||
|
PARAMTER_EXPIRED |
The nextToken query parameter expired |
|
|
INAVLID_INTEGER_VALUE |
Value of maxResults is not within allowed ranges |
|
|
500 Internal Server Error |
INTERNAL_SERVER_ERROR |
Request failed due to an internal server error |
Examples
| Sample Request | Sample Response |
|---|---|
GET /v1/skills/{skillId}/asrEvaluations HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"evaluations": [
{
"id": "1234-1234-54321",
"status": "COMPLETED",
"startTimestamp": "2018-10-25T08:25:04.679Z",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"request": {
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "1234-1234-1234"
},
"result": {
"status": "PASSED",
"metrics": {
"overallErrorRate": 0.5
}
}
},
{
"id": "1234-1234-54325",
"status": "COMPLETED",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"startTimestamp": "2018-10-24T08:25:04.679Z"
"request": {
"skill": {
"stage": "live",
"locale": "en-GB"
},
"annotationSetId": "1234-1234-1234"
},
"result": {
"status": "FAILED",
"metrics": {
"overallErrorRate": 0.5
}
}
},
{
"id": "1234-1234-54322",
"status": "COMPLETED",
"startTimestamp": "2018-10-24T08:25:04.679Z",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"request": {
"skill": {
"stage": "development",
"locale": "en-GB"
},
"annotationSetId": "1234-1234-1234"
},
"result": {
"status": "FAILED",
"metrics": {
"overallErrorRate": 0.5
}
}
},
{
"id": "1234-1234-54323",
"status": "FAILED",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"startTimestamp": "2018-10-24T08:25:04.679Z"
"request": {
"skill": {
"stage": "development",
"locale": "en-GB"
},
"annotationSetId": "1234-1234-1234"
},
"error": {
"message": "An unexpected error occurred",
"code": "INTERNAL_SERVER_ERROR"
}
}
]
}
|
GET /v1/skills/{skillId}/asrEvaluations HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"paginationContext": {
"nextToken": "12341234"
},
"evaluations": [
{
"id": "1234-1234-54325",
"status": "COMPLETED",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"startTimestamp": "2018-10-24T08:25:04.679Z"
"request": {
"skill": {
"stage": "live",
"locale": "en-GB"
},
"annotationSetId": "1234-1234-1234"
},
"result": {
"status": "FAILED",
"metrics": {
"overallErrorRate": 0.5
}
}
}
]
}
|
GET /v1/skills/{skillId}/asrEvaluations HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"evaluations": [
{
"id": "1234-1234-54322",
"status": "COMPLETED",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"startTimestamp": "2018-10-24T08:25:04.679Z"
"request": {
"skill": {
"stage": "development",
"locale": "en-GB"
},
"annotationSetId": "1234-1234-1234"
},
"result": {
"status": "FAILED",
"metrics": {
"overallErrorRate": 0.5
}
}
}
]
}
|
GET /v1/skills/{skillId}/asrEvaluations HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"evaluations": [
{
"id": "1234-1234-54325",
"status": "COMPLETED",
"totalEvaluationCount": 10,
"completedEvaluationCount": 10,
"startTimestamp": "2018-10-24T08:25:04.679Z"
"request": {
"skill": {
"stage": "live",
"locale": "en-GB"
},
"annotationSetId": "1234-1234-1234"
},
"result": {
"status": "FAILED",
"metrics": {
"overallErrorRate": 0.5
}
}
}
]
}
|
GET /v1/skills/{skillId}/asrEvaluations HTTP/1.1
Accept: application/json
|
HTTP 400 Bad Request
Content-Type: application/json
{
"message": "Request validations failed",
"violations": [{
"message": "Query parameter \"nextToken\" with value \"12341234\" is expired.",
"code": "PARAMETER_EXPIRED"
}]
}
|
Related topics
- What is Automatic Speech Recognition?
- Create an Annotation Set for Automatic Speech Recognition (ASR)
- Run an Automatic Speech Recognition (ASR) Evaluation
- Improve your Automatic Speech Recognition (ASR) Test Results
- Create annotation set API
- Update annotation set annotations API
- Post ASR evaluation API
- Get ASR evaluation results API
- Create catalog API
- Batch Test Your Natural Language Understanding (NLU) Model
