Get Automatic Speech Recognition (ASR) Evaluation Status API

Call this API to return high level information about a specified Automatic Speech Recognition (ASR) evaluation run. This API doesn't give the detailed evaluation results. To view more detailed evaluation results, call the Get ASR Evaluation Results API.

This call differs from the Get Automatic Speech Recognition (ASR) Evaluation Results API in that Get ASR Evaluation Results is a heavier weight API that returns a potentially large quantity of data, whereas Get ASR Evaluation Status returns more high-level information.

To learn more about the intended API flow for ASR Evaluation, see API Flow.

HTTP method and path

GET v1/skills/{skillId}/asrEvaluations/{evaluationId}/status HTTP/1.1
Attributes Type Description Required
skillId String Skill Id as part of the request URI. Yes
evaluationId String Evaluation Id as part of the request URI. Yes

Request

Request Headers

  Accept: application/json

Request Body

None.

Response

HTTP Status

HTTP 200 OK

Response Headers

Content-Type: application/json

Response Body

{
   "status": enum["IN_PROGRESS", "COMPLETED", "FAILED"],
   "totalEvaluationCount": integer,
   "completedEvaluationCount": integer,
   "startTimestamp": string,
   "request": {
       "skill": {
          "stage": enum["development", "live"],
          "locale": string
       },
       "annotationSetId": string
   },
   "result": {
      "status": enum["PASSED", "FAILED"],
      "metrics": {
          "overallErrorRate": double
      }
   },
   "error": {
      "message": string,
      "code": string
   }
}
Attributes Type Description Required
status Enum Enum indicating the status of the evaluation job. The following values are possible:
IN_PROGRESS: Indicates the evaluation is in progress.
FAILED: Indicates the evaluation cannot start due to factors unrelated to input vaidation.
COMPLETED: Indicates the evaluation has completed. 		Yes
request Object The request payload that triggers this evaluation run. The structure is identical in the request payload of Post ASR Evaluation. Yes
totalEvaluationCount Integer Total number of evaluations to be run in the evaluation request. Yes
completedEvaluationCount Integer Number of completed evaluations. Yes
startTimestamp String Start time stamp of the ASR evaluation job in ISO-8601 format. Yes
result Object Object containing information about the evaluation result Yes, if status is COMPLETED, No, otherwise.
result.metrics Object object containing information about the generated metrics. Yes, if result is present, No, otherwise.
result.metrics.overallErrorRate Double Overall error rate for the ASR evaluation run. Only present if all the evaluation tasks have been completed. Yes, if result is present, No, otherwise.
result.status Enum Enum indicating the evaluation result status. Possible values are PASSED or FAILED. Yes, if result is present, No, otherwise.
error Object Information about the error. This field is present in the response payload if the evaluation stopped due to internal server error. No
error.message String Detailed reason describing why the evaluation run failed. Yes, if the run resulted in an error, No, otherwise.
error.code String Error codes. TODO: NEED LINK/REFERENCE TO CODES. Yes, if the run resulted in an error, No, otherwise.

Error Response

This section lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.

HTTP Status Code Error Code Scenario

404 Not Found

RESOURCE_NOT_FOUND

Requested skill does not exist.

Requested evaluation does not exist.

400 Bad Request

INVALID_REQUEST_PARAMETER

Evaluation id is malformed.

500 Internal Server error

INTERNAL_SERVER_ERROR

Request failed due to an internal server error.

Examples

Sample Request Sample Response
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1 Accept: application/json HTTP 200 OK Content-Type: application/json { "id": "1234-1234-1234-12341234", "status": "COMPLETED", "totalEvaluationCount": 2, "completedEvaluation": 2, "startTimestamp": "2018-10-25T08:25:04.679Z", "request": { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "1234-1234-12341234" }, "result": { "status": "PASSED", "metrics": { "overallErrorRate": 0.5 } } }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1 Accept: application/json
HTTP 200 OK Content-Type: application/json { "id": "1234-1234-1234-12341234", "status": "IN_PROGRESS", "totalEvaluationCount": 2, "completedEvaluationCount": 1, "startTimestamp": "2018-10-25T08:25:04.679Z", "request": { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "1234-1234-12341234" } }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1 Accept: application/json
HTTP 200 OK Content-Type: application/json { "id": "1234-1234-1234-12341234", "status": "COMPLETED", "totalEvaluationCount": 2, "completedEvaluationCount": 2, "startTimestamp": "2018-10-25T08:25:04.679Z", "request": { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "1234-1234-12341234" }, "result": { "status": "FAILED", "metrics": { "overallErrorRate": 0.4 } } }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1 Accept: application/json
HTTP 200 OK Content-Type: application/json { "id": "1234-1234-1234-12341234", "status": "FAILED", "totalEvaluationCount": 2, "completedEvaluationCount": 2, "startTimestamp": "2018-10-25T08:25:04.679Z", "request": { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "1234-1234-12341234" }, "error": { "message": "An unexpected error occurred during evaluation run", "code": "INTERNAL_SERVER_ERROR" } }
GET /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status HTTP/1.1 Accept: application/json
HTTP 404 NOT FOUND Content-Type: application/json { "message": "Evaluation with identifier \"1234-1234-1234-12341234\" cannot be found", "code": "RESOURCE_NOT_FOUND" }