Post Automatic Speech Recognition (ASR) Evaluation API

Call this API to run ASR evaluations against an existing annotation set.

To learn more about the intended API flow for ASR Evaluation, see API Flow.

HTTP method and path

POST v1/skills/{skillId}/asrEvaluations HTTP/1.1
Attributes Type Description Required
skillId String Skill Id as part of the URI path. Yes

Request

Request Headers

Content-Type: application/json
Accept: application/json

Request Body

{
   "skill": {
      "stage": enum["development", "live"],
      "locale": string
   },
   "annotationSetId": string
}
Attributes Type Description Required
skill Object Object containing information about the skill used in the evaluation. Yes
skill.stage Enum Used to filter results with the specified skill stage. Possible values are development or live. If omitted, the response would include all evaluations regardless of what skill stage was used in the evaluation. Yes
skill.locale String Skill locale in bcp 47 format. Yes
annotationSetId String Id of the annotation set. Yes

Response

HTTP Status

HTTP 200 Ok

Response Headers

Content-Type: application/json
Location: v1/skills/{skillId}/asrEvaluations/{evaluationId}/status
Attributes Type Description Required
skillId String skill id. As part of the Location header Yes
evaluationId String evaluation id. As part of the Location header Yes

Response Body

{
   "id": string
}
Attributes Type Description Required
id String Evaluation Id associated with the evaluation run. The format of this ID follows the naming convention: 'amzn1.ask.asr-evaluaiton.{UUID}'.

Error Response

This section lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.

HTTP Status Code Error Code Scenario

404 Not Found

RESOURCE_NOT_FOUND

When the requested skill does not exist

When the requested skill stage does not exist

When the annotation set does not exist

400 Bad Request

INVALID_REQUEST_PARAMETER

When the annotation set id is malformed

MISSING_REQUIRED_PROPERTY

When the required fields are missing in the request payload

CONTENT_PARSE_FAILURE

When the request payload is malformed

PRECONDITION_NOT_MET When the annotation set is empty
When the skill interaction model does not exist for the specified locale

409 Conflict

CONFLICTING_INSTANCES

When the evaluation request conflicts with an evaluation request that is currently in progress for a given skill (Concurrent evaluation requests for a given skill id is not supported, there can only be 1 evaluation in progress for a given skill at a given time)

500 Internal Server Error

INTERNAL_SERVER_ERROR

When the request failed due to an internal server error

Examples

Sample Request Sample Response
POST /v1/skills/1234/asrEvaluations HTTP/1.1 Content-Type: application/json Accept: application/json { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "amzn1.ask.asr-annotation-set.1234-1234-12341234" } HTTP 200 OK Content-Type: application/json Location: /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status { "id": "amzn1.ask.asr-evaluation.1234-1234-1234-12341234" }
POST /v1/skills/1234/asrEvaluations HTTP/1.1 Content-Type: application/json Accept: application/json { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "amzn1.ask.asr-annotation-set.1234-1234-12341234" }
HTTP 400 Bad Request Content-Type: application/json Location: /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status { "message": "Skill does not have en-US internal model", "violations": [{ "message": "The skill does not have \"en-US\" interaction model", "code": "PRECONDITION_NOT_MET" }] }