Get Automatic Speech Recognition (ASR) Evaluation Status API
Call this API to return high level information about a specified Automatic Speech Recognition (ASR) evaluation run. This API doesn't give the detailed evaluation results. To view more detailed evaluation results, call the Get ASR Evaluation Results API.
This call differs from the Get Automatic Speech Recognition (ASR) Evaluation Results API in that Get ASR Evaluation Results is a heavier weight API that returns a potentially large quantity of data, whereas Get ASR Evaluation Status returns more high-level information.
To learn more about the intended API flow for ASR Evaluation, see API Flow.
HTTP method and path
GET v1/skills/{skillId}/asrEvaluations/{evaluationId}/status HTTP/1.1
| Attributes | Type | Description | Required |
|---|---|---|---|
| skillId | String | Skill Id as part of the request URI. | Yes |
| evaluationId | String | Evaluation Id as part of the request URI. | Yes |
Request
Request Headers
Accept: application/json
Request Body
None.
Response
HTTP Status
HTTP 200 OK
Response Headers
Content-Type: application/json
Response Body
{
"status": enum["IN_PROGRESS", "COMPLETED", "FAILED"],
"totalEvaluationCount": integer,
"completedEvaluationCount": integer,
"startTimestamp": string,
"request": {
"skill": {
"stage": enum["development", "live"],
"locale": string
},
"annotationSetId": string
},
"result": {
"status": enum["PASSED", "FAILED"],
"metrics": {
"overallErrorRate": double
}
},
"error": {
"message": string,
"code": string
}
}
| Attributes | Type | Description | Required |
|---|---|---|---|
| status | Enum | Enum indicating the status of the evaluation job. The following values are possible: IN_PROGRESS: Indicates the evaluation is in progress. FAILED: Indicates the evaluation cannot start due to factors unrelated to input vaidation. COMPLETED: Indicates the evaluation has completed. |
Yes |
| request | Object | The request payload that triggers this evaluation run. The structure is identical in the request payload of Post ASR Evaluation. | Yes |
| totalEvaluationCount | Integer | Total number of evaluations to be run in the evaluation request. | Yes |
| completedEvaluationCount | Integer | Number of completed evaluations. | Yes |
| startTimestamp | String | Start time stamp of the ASR evaluation job in ISO-8601 format. | Yes |
| result | Object | Object containing information about the evaluation result | Yes, if status is COMPLETED, No, otherwise. |
| result.metrics | Object | object containing information about the generated metrics. | Yes, if result is present, No, otherwise. |
| result.metrics.overallErrorRate | Double | Overall error rate for the ASR evaluation run. Only present if all the evaluation tasks have been completed. | Yes, if result is present, No, otherwise. |
| result.status | Enum | Enum indicating the evaluation result status. Possible values are PASSED or FAILED. | Yes, if result is present, No, otherwise. |
| error | Object | Information about the error. This field is present in the response payload if the evaluation was terminated due to internal server error. | No |
| error.message | String | Detailed reason describing why the evaluation run failed. | Yes, if the run resulted in an error, No, otherwise. |
| error.code | String | Error codes. TODO: NEED LINK/REFERENCE TO CODES. | Yes, if the run resulted in an error, No, otherwise. |
Error Response
This section lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.
| HTTP Status Code | Error Code | Scenario |
|---|---|---|
|
404 Not Found |
RESOURCE_NOT_FOUND |
Requested skill does not exist. |
|
Requested evaluation does not exist. |
||
|
400 Bad Request |
INVALID_REQUEST_PARAMETER |
Evaluation id is malformed. |
|
500 Internal Server error |
INTERNAL_SERVER_ERROR |
Request failed due to an internal server error. |
Examples
| Sample Request | Sample Response |
|---|---|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"id": "1234-1234-1234-12341234",
"status": "COMPLETED",
"totalEvaluationCount": 2,
"completedEvaluation": 2,
"startTimestamp": "2018-10-25T08:25:04.679Z",
"request": {
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "1234-1234-12341234"
},
"result": {
"status": "PASSED",
"metrics": {
"overallErrorRate": 0.5
}
}
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"id": "1234-1234-1234-12341234",
"status": "IN_PROGRESS",
"totalEvaluationCount": 2,
"completedEvaluationCount": 1,
"startTimestamp": "2018-10-25T08:25:04.679Z",
"request": {
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "1234-1234-12341234"
}
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"id": "1234-1234-1234-12341234",
"status": "COMPLETED",
"totalEvaluationCount": 2,
"completedEvaluationCount": 2,
"startTimestamp": "2018-10-25T08:25:04.679Z",
"request": {
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "1234-1234-12341234"
},
"result": {
"status": "FAILED",
"metrics": {
"overallErrorRate": 0.4
}
}
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1
Accept: application/json
|
HTTP 200 OK
Content-Type: application/json
{
"id": "1234-1234-1234-12341234",
"status": "FAILED",
"totalEvaluationCount": 2,
"completedEvaluationCount": 2,
"startTimestamp": "2018-10-25T08:25:04.679Z",
"request": {
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "1234-1234-12341234"
},
"error": {
"message": "An unexpected error occurred during evaluation run",
"code": "INTERNAL_SERVER_ERROR"
}
}
|
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1
Accept: application/json
|
HTTP 404 NOT FOUND
Content-Type: application/json
{
"message": "Evaluation with identifier \"1234-1234-1234-12341234\" cannot be found",
"code": "RESOURCE_NOT_FOUND"
}
|
