Post Automatic Speech Recognition (ASR) Evaluation API
Call this API to run ASR evaluations against an existing annotation set.
To learn more about the intended API flow for ASR Evaluation, see API Flow.
HTTP method and path
POST v1/skills/{skillId}/asrEvaluations HTTP/1.1
| Attributes |
Type |
Description |
Required |
| skillId |
String |
Skill Id as part of the URI path. |
Yes |
Request
Content-Type: application/json
Accept: application/json
Request body
{
"skill": {
"stage": enum["development", "live"],
"locale": string
},
"annotationSetId": string
}
| Attributes |
Type |
Description |
Required |
| skill |
Object |
Object containing information about the skill used in the evaluation. |
Yes |
| skill.stage |
Enum |
Used to filter results with the specified skill stage. Possible values are development or live. If omitted, the response would include all evaluations regardless of what skill stage was used in the evaluation.
|
Yes |
| skill.locale |
String |
Skill locale in bcp 47 format. |
Yes |
| annotationSetId |
String |
Id of the annotation set. |
Yes |
Response
HTTP status
Content-Type: application/json
Location: v1/skills/{skillId}/asrEvaluations/{evaluationId}/status
| Attributes |
Type |
Description |
Required |
| skillId |
String |
skill id. As part of the Location header |
Yes |
| evaluationId |
String |
evaluation id. As part of the Location header |
Yes |
Response body
| Attributes |
Type |
Description |
Required |
| id |
String |
Evaluation Id associated with the evaluation run. The format of this ID follows the naming convention: 'amzn1.ask.asr-evaluaiton.{UUID}'. |
Error response
The following table lists call-specific errors. For a reference of errors that can apply to all ASR APIs, see Automatic Speech Recognition (ASR) API Error Reference.
| HTTP Status Code |
Error Code |
Scenario |
| 404 Not Found |
RESOURCE_NOT_FOUND |
When the requested skill does not exist |
| When the requested skill stage does not exist |
| When the annotation set does not exist |
| 400 Bad Request |
INVALID_REQUEST_PARAMETER |
When the annotation set id is malformed |
| MISSING_REQUIRED_PROPERTY |
When the required fields are missing in the request payload |
| When the referenced annotation set is missing required attributes to evaluate, return the indexes for violating annotations. |
| CONTENT_PARSE_FAILURE |
When the request payload is malformed |
| PRECONDITION_NOT_MET |
When the annotation set is empty |
| When the skill interaction model does not exist for the specified locale |
| 409 Conflict |
CONFLICTING_INSTANCES |
When the evaluation request conflicts with an evaluation request that is currently in progress for a given skill (Concurrent evaluation requests for a given skill id is not supported, there can only be 1 evaluation in progress for a given skill at a given time). |
| 500 Internal Server Error |
INTERNAL_SERVER_ERROR |
When the request failed due to an internal server error |
Examples
| Sample Request |
Sample Response |
POST /v1/skills/1234/asrEvaluations HTTP/1.1
Content-Type: application/json
Accept: application/json
{
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "amzn1.ask.asr-annotation-set.1234-1234-12341234"
}
|
HTTP 200 OK
Content-Type: application/json
Location: /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status
{
"id": "amzn1.ask.asr-evaluation.1234-1234-1234-12341234"
}
|
POST /v1/skills/1234/asrEvaluations HTTP/1.1
Content-Type: application/json
Accept: application/json
{
"skill": {
"stage": "development",
"locale": "en-US"
},
"annotationSetId": "amzn1.ask.asr-annotation-set.1234-1234-12341234"
}
|
HTTP 400 Bad Request
Content-Type: application/json
Location: /v1/skills/1234/asrEvaluations/1234-1234-1234-12341234/status
{
"message": "Skill does not have en-US internal model",
"violations": [{
"message": "The skill does not have \"en-US\" interaction model",
"code": "PRECONDITION_NOT_MET"
}]
}
|
