Post Automatic Speech Recognition (ASR) Evaluation API

Call this API to run ASR evaluations against an existing annotation set.

To learn more about the intended API flow for ASR Evaluation, see API Flow.

HTTP method and path

POST v1/skills/{skillId}/asrEvaluations HTTP/1.1
Attributes Type Description Required
skillId String Skill Id as part of the URI path. Yes

Request

Request headers

Content-Type: application/json
Accept: application/json

Request body

{
   "skill": {
      "stage": enum["development", "live"],
      "locale": string
   },
   "annotationSetId": string
}
Attributes Type Description Required
skill Object Object containing information about the skill used in the evaluation. Yes
skill.stage Enum Used to filter results with the specified skill stage. Possible values are development or live. If omitted, the response would include all evaluations regardless of what skill stage was used in the evaluation. Yes
skill.locale String Skill locale in bcp 47 format. Yes
annotationSetId String Id of the annotation set. Yes

Response

HTTP status

HTTP 200 Ok

Response headers

Content-Type: application/json
Location: v1/skills/{skillId}/asrEvaluations/{evaluationId}/status
Attributes Type Description Required
skillId String skill id. As part of the Location header Yes
evaluationId String evaluation id. As part of the Location header Yes

Response body

{
   "id": string
}
Attributes Type Description Required
id String Evaluation Id associated with the evaluation run. The format of this ID follows the naming convention: 'amzn1.ask.asr-evaluaiton.{UUID}'.

Error response

The following table lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.

HTTP Status Code Error Code Scenario
404 Not Found RESOURCE_NOT_FOUND When the requested skill does not exist
When the requested skill stage does not exist
When the annotation set does not exist
400 Bad Request INVALID_REQUEST_PARAMETER When the annotation set id is malformed
MISSING_REQUIRED_PROPERTY When the required fields are missing in the request payload
When the referenced annotation set is missing required attributes to evaluate, return the indexes for violating annotations.
CONTENT_PARSE_FAILURE When the request payload is malformed
PRECONDITION_NOT_MET When the annotation set is empty
When the skill interaction model does not exist for the specified locale
409 Conflict CONFLICTING_INSTANCES When the evaluation request conflicts with an evaluation request that is currently in progress for a given skill (Concurrent evaluation requests for a given skill id is not supported, there can only be 1 evaluation in progress for a given skill at a given time).
500 Internal Server Error INTERNAL_SERVER_ERROR When the request failed due to an internal server error

Examples

Sample Request Sample Response
POST /v1/skills/1234/asrEvaluations HTTP/1.1 Content-Type: application/json Accept: application/json { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "amzn1.ask.asr-annotation-set.1234-1234-12341234" } HTTP 200 OK Content-Type: application/json Location: /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status { "id": "amzn1.ask.asr-evaluation.1234-1234-1234-12341234" }
POST /v1/skills/1234/asrEvaluations HTTP/1.1 Content-Type: application/json Accept: application/json { "skill": { "stage": "development", "locale": "en-US" }, "annotationSetId": "amzn1.ask.asr-annotation-set.1234-1234-12341234" }
HTTP 400 Bad Request Content-Type: application/json Location: /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status { "message": "Skill does not have en-US internal model", "violations": [{ "message": "The skill does not have \"en-US\" interaction model", "code": "PRECONDITION_NOT_MET" }] }