Smart Home Skill Evaluation REST API Reference
Use the Smart Home Evaluation REST API to start evaluations and get evaluation results. Smart home evaluations verify that you implemented the Alexa smart home interfaces in your skill correctly.
If you want to run the Smart Home tests in the Alexa developer console, see Use the Smart Home Test Tool.
API endpoint
The endpoint of the Smart Home Evaluation API is https://api.amazonalexa.com.
Authentication
Each API request must have an authorization header whose value is the access token retrieved from Login with Amazon (LWA). For details, see Get an Access Token for SMAPI.
Operations
The Smart Home Evaluation API includes the following operations.
| Operation | HTTP Method and URI |
|---|---|
|
| |
|
| |
|
| |
|
| |
|
|
Create smart home capability evaluation
Create and start a capability evaluation against the specified smart home skill.
Before you create an evaluation, make sure that you do the following test setup:
- Enable your skill in the Alexa app.
- Your skill configures account linking and you link your device account with your Alexa account successfully.
- Alexa discovered your test device successfully and you can view your device in the Alexa app.
Request
To create an evaluation, you make a POST request to the smartHome/testing/capabilityEvaluations resource.
Request path and header example
POST /v1/skills/{skillId}/smartHome/testing/capabilityEvaluations
Host: api.amazonalexa.com
Content-Type: application/json
Accept: application/json
Authorization: Bearer {access token}
X-Amzn-RequestId: {request ID}
Request path and header parameters
| Parameter | Located in | Description | Type | Required |
|---|---|---|---|---|
|
|
Path |
Identifies the skill. |
String |
Yes |
|
|
Header |
String |
Yes | |
|
|
Header |
Unique identifier for the request, preferably a version 4 UUID. |
String |
Yes |
Request body example
{
"endpoint": {
"endpointId": "endpoint-001"
},
"capabilityTestPlan": {
"id": "test-plan-001"
},
"stage": "development"
}
Request body properties
| Property | Description | Type | Required |
|---|---|---|---|
|
|
Represents a connected device or component associated with a user's device cloud account. |
Object |
Yes |
|
|
Unique identifier for the endpoint. |
String |
Yes |
|
|
Represents the test plan to evaluate the skill against. |
Object |
Yes |
|
|
Identifies the test plan. |
String |
Yes |
|
|
Indicates the stage of the skill. |
String |
Yes |
Response
A successful response returns HTTP 200 OK, along with the identifier of the evaluation to indicate that the evaluation started successfully.
On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.
Response body example
{
"id": "testRunId12312",
"endpoint": {
"endpointId": "endpoint-001"
},
"capabilityTestPlan": {
"id": "test-plan-001"
},
"stage": "development"
}
Response body properties
| Property | Description | Type | Required |
|---|---|---|---|
|
|
Identifies the evaluation. |
String |
Yes |
|
|
Represents a connected device or component associated with a user's device cloud account. |
Object |
Yes |
|
|
Unique identifier for the endpoint. |
String |
Yes |
|
|
Represents the test plan to evaluate the skill against. |
Object |
Yes |
|
|
Identifies the test plan. |
String |
Yes |
|
|
Indicates the stage of the skill. |
String |
Yes |
HTTP status codes
| Status | Description |
|---|---|
|
|
Response body contains the evaluation identifier indicating that the evaluation started. |
|
|
Indicates that one or more properties in the request body are invalid. |
|
|
Request didn't include the authorization token or the token is invalid or expired. Or, the client doesn't have access to the resource. |
|
|
Indicates that the authorization token is valid, but the requested operation isn't allowed. |
|
|
Requested resource not found. |
|
|
Skill has exceeded the permitted rate limit (specified number of requests per unit of time). The skill can retry the request by using exponential back-off. |
|
|
Error occurred on the server. The skill can retry the request by using exponential back-off. |
|
|
Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request. |
Get smart home capability evaluation status
Get the status of an evaluation.
Request
To get the evaluation status, you make a GET request to the smartHome/testing/capabilityEvaluations resource.
Request path and header example
GET /v1/skills/{skillId}/smartHome/testing/capabilityEvaluations/{evaluationId}
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json
X-Amzn-RequestId: {request ID}
Request path and header parameters
| Parameter | Located in | Description | Type | Required |
|---|---|---|---|---|
|
|
Path |
Identifies the skill. |
String |
Yes |
|
|
Path |
Identifies the evaluation. |
String |
Yes |
|
|
Header |
String |
Yes | |
|
|
Header |
Unique identifier for the request, preferably a version 4 UUID. |
String |
Yes |
Request body example
The request has no body.
Request body properties
The request has no body.
Response
A successful response returns HTTP 200 OK, along with the evaluation status and metrics.
On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.
Response body example
The following example shows a response.
{
"startTimestamp": "2019-07-17T06:45:40.689Z",
"endTimestamp": "2019-07-17T06:55:40.689Z",
"status": "PASSED",
"metrics": {
"PowerController": {
"totalTestCases": 2,
"passedTestCases": 2,
"failedTestCases": 0,
"errorTestCases": 0
}
},
"input": {
"endpoint": {
"endpointId": "endpoint-001"
},
"capabilityTestPlan": {
"id": "test-plan-001"
},
"stage": "live"
}
}
Response body properties
| Property | Description | Type |
|---|---|---|
|
|
Start time of the evaluation. |
String |
|
|
End time of the evaluation. |
String |
|
|
Status of the evaluation. |
String |
|
|
(Optional) Included when an error occurs retrieving the evaluation status. |
Object |
|
|
Error code. |
String |
|
|
Human-readable description of the error. |
String |
|
|
Describes the number of test cases in the evaluation. |
Object |
|
|
Name of the test plan. |
Object |
|
|
Total number of test cases in the test plan. |
Integer |
|
|
Number of test cases that passed. |
Integer |
|
|
Number of test cases that failed. |
Integer |
|
|
Number of test cases that didn't complete due to an error condition, such as unplugged device. |
Integer |
|
|
Evaluation details given in Create smart home capability evaluation. |
Type |
|
|
Represents a connected device or component associated with a user's device cloud account. |
Object |
|
|
Unique identifier for the endpoint. |
String |
|
|
Represents the test plan to evaluate the skill against. |
Object |
|
|
Identifies the test plan. |
String |
|
|
Indicates the stage of the skill. |
String |
HTTP status codes
| Status | Description |
|---|---|
|
|
Response body contains an array of the customer lists. Default lists appear first in the array. |
|
|
Indicates that one or more properties in the request body are invalid. |
|
|
Request didn't include the authorization token or the token is invalid or expired. Or, the client doesn't have access to the resource. |
|
|
Indicates that the authorization token is valid, but the requested operation isn't allowed. |
|
|
Requested resource not found. |
|
|
Skill has exceeded the permitted rate limit (specified number of requests per unit of time). The skill can retry the request by using exponential back-off. |
|
|
Error occurred on the server. The skill can retry the request by using exponential back-off. |
|
|
Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request. |
Get smart home capability evaluation results
Get the results of the specified evaluation.
Request
To get the results of an evaluation, you make a GET request to the smartHome/testing/capabilityEvaluations resource.
Request header example
GET /v1/skills/{skillId}/smartHome/testing/capabilityEvaluations/{evaluationId}/results?maxResults=2&offset=1
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json
Request path and header parameters
| Parameter | Located in | Description | Type | Required |
|---|---|---|---|---|
|
|
Path |
Identifies the skill. |
String |
Yes |
|
|
Path |
Identifies the evaluation. |
String |
Yes |
|
|
Query |
Maximum number of items to return in the response. The response might contain fewer items than the value you specify, but never contains more. Default: 20 |
Integer |
No |
|
|
Query |
Offset of the result page to return. |
Integer |
No |
|
|
Header |
String |
Yes | |
|
|
Header |
Access token used for the customer. Set to |
String |
Yes |
Request body example
The request has no body.
Request body properties
The request has no body.
Response
A successful response returns HTTP 200 OK, along with the evaluation results.
On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.
Response body example
The following example shows a response.
{
"paginationContext": {
"offset": "1",
"totalCount": 2
},
"testCaseResults": [{
"name": "LightingTestCase1",
"status": "PASSED",
"capability": "PowerController",
"expectedCapabilityStates": [{
"namespace": "Alexa.PowerController",
"name": "powerState",
"value": "ON"
},
{
"namespace": "Alexa.BrightnessController",
"name": "brightness",
"value": 20
}
],
"directive": {
"header": {
"namespace": "Alexa.BrightnessController",
"name": "setBrightness"
},
"payload": {
"brightness": 20
}
},
"actualCapabilityStates": [{
"namespace": "Alexa.PowerController",
"name": "powerState",
"value": "ON"
},
{
"namespace": "Alexa.BrightnessController",
"name": "brightness",
"value": 20
}
]
}]
}
Response body properties
| Parameter | Description | Type |
|---|---|---|
|
|
Included when there are more items that match the request. |
Object |
|
|
Offset in the result page. |
Integer |
|
|
Total number of items that satisfy the request. This number might be higher than the number of items returned in this response. |
Integer |
|
|
List of test plans that satisfy the request. |
Array of objects |
|
|
Human-readable test case name. |
String |
|
|
Status of the test case. |
String |
|
|
Smart Home API interfaces and expected property values. |
Array of objects |
|
|
Interface name, such as |
String |
|
|
Property name, such as |
String |
|
|
Operation used to compare the expected property value with the actual property value. |
String |
|
|
Expected property value results. |
String |
|
|
Directive sent to the skill. |
Object |
|
|
Header information. |
Object |
|
|
Name of the interface. |
String |
|
|
Name of the directive. |
String |
|
|
JSON payload included in the directive. |
Object |
|
|
Actual property value results returned from the skill. |
Array of objects |
|
|
Interface name, such as |
String |
|
|
Property name, such as |
String |
|
|
Operation used to compare the expected property value with the actual property value. |
String |
|
|
Expected property value. |
String |
|
|
Expected response to the directive. |
String |
|
|
Actual response to the directive. |
String |
|
|
(Optional) Included when an error occurs. |
Object |
|
|
Error code. |
String |
|
|
Human-readable description of the error. |
String |
HTTP status codes
| Status | Description |
|---|---|
|
|
Response body contains an array of the customer lists. Default lists appear first in the array. |
|
|
Indicates that one or more properties in the request body are invalid. |
|
|
Request didn't include the authorization token or the token is invalid or expired. Or, the client doesn't have access to the resource. |
|
|
Indicates that the authorization token is valid, but the requested operation isn't allowed. |
|
|
Requested resource not found. |
|
|
Skill has exceeded the permitted rate limit (specified number of requests per unit of time). The skill can retry the request by using exponential back-off. |
|
|
Error occurred on the server. The skill can retry the request by using exponential back-off. |
|
|
Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request. |
List smart home capability evaluations
List the evaluations for a specified skill.
Request
To list the evaluations for a skill, you make a GET request to the smartHome/testing/capabilityEvaluations resource.
Request header example
GET /v1/skills/{skillId}/smartHome/testing/capabilityEvaluations?stage=development&startTimestampFrom=2022-07-12T09:00:00.000Z&startTimestampTo=2022-07-15T11:00:00.000Z&maxResults=10&nextToken=someToken
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json
Request path and header parameters
| Parameter | Located in | Description | Type | Required |
|---|---|---|---|---|
|
|
Path |
Identifies the skill. |
String |
Yes |
|
|
Query |
Filter based on the stage of the skill. |
String |
Yes |
|
|
Query |
Beginning of the start time to query evaluation results. |
String |
No |
|
|
Query |
End of the start time to query evaluation results. |
String |
No |
|
|
Query |
Maximum number of items to return in the response. The response might contain fewer items than the value you specify, but never contains more. Default: 20 |
Integer |
No |
|
|
Query |
Results might paginate. Include the value of |
String |
No |
|
|
Header |
String |
Yes |
Request body example
The request has no body.
Request body properties
The request has no body.
Response
A successful response returns HTTP 200 OK, along with a list of XYZ.
On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.
Response body example
The following example shows a response.
{
"paginationContext": {
"nextToken": "someToken"
},
"evaluations": [{
"id": "213123",
"startTimestamp": "2019-07-12T21:52:10.329Z",
"endTimestamp": "2019-07-12T21:55:10.329Z",
"status": "PASSED",
"endpointId": "endpoint-001",
"testPlanId": "test-plan-1234567",
"sourceTestPlanIds": ["test-plan-1", "test-plan-2"]
}]
}
Response body properties
| Parameter | Description | Type |
|---|---|---|
|
|
Included when there are more items that match the request. |
Object |
|
|
Use the |
String |
|
|
List of evaluations that match the query parameters. |
Array of objects |
|
|
Identifies the evaluation. |
String |
|
|
Start time of the evaluation. |
String |
|
|
End time of the evaluation. |
String |
|
|
Status of the evaluation. |
String |
|
|
Unique identifier for the endpoint. |
String |
|
|
Identifies the test plan. |
String |
|
|
List of all test plan identifiers included in the evaluation. |
Array of strings |
HTTP status codes
| Status | Description |
|---|---|
|
|
Response body contains an array of the customer lists. Default lists appear first in the array. |
|
|
Indicates that one or more properties in the request body are invalid. |
|
|
Request didn't include the authorization token or the token is invalid or expired. Or, the client doesn't have access to the resource. |
|
|
Indicates that the authorization token is valid, but the requested operation isn't allowed. |
|
|
Requested resource not found. |
|
|
Skill has exceeded the permitted rate limit (specified number of requests per unit of time). The skill can retry the request by using exponential back-off. |
|
|
Error occurred on the server. The skill can retry the request by using exponential back-off. |
|
|
Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request. |
List smart home capability test plans
Request available smart home capability test plans. To view the available test plans on GitHub, see alexa/alexa-smarthome/capability_evaluations/test_plans/.
Request
To list available test plans, you make a GET request to the smartHome/testing/capabilityEvaluations resource.
Request header example
GET /v1/skills/{skillId}/smartHome/testing/capabilityTestPlans?maxResults=1&nextToken=someToken
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json
X-Amzn-RequestId: {request ID}
Request path and header parameters
| Parameter | Located in | Description | Type | Required |
|---|---|---|---|---|
|
|
Path |
Identifies the skill. |
String |
Yes |
|
|
Query |
Maximum number of items to return in the response. The response might contain fewer items than the value you specify, but never contains more. Default: 20 |
Integer |
No |
|
|
Query |
Results might paginate. Include the value of |
String |
No |
|
|
Header |
String |
Yes | |
|
|
Header |
Unique identifier for the request, preferably a version 4 UUID. |
String |
Yes |
Request body example
The request has no body.
Request body properties
The request has no body.
Response
A successful response returns HTTP 200 OK, along with a list of test plans.
On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.
Response body example
The following example shows a response.
{
"paginationContext": {
"nextToken": "someToken",
"totalCount": 2
},
"testPlans": [{
"id": "1231231",
"name": "LightingTestPlan"
}]
}
Response body properties
| Property | Description | Type |
|---|---|---|
|
|
Included when there are more items that match the request. |
Object |
|
|
Use the |
String |
|
|
Total number of items that satisfy the request. This number might be higher than the number of items returned in this response. |
Integer |
|
|
List of test plans that satisfy the request. |
Array of objects |
|
|
Identifies the test plan. |
String |
|
|
Human-readable test plan name. |
String |
HTTP status codes
| Status | Description |
|---|---|
|
|
Response body contains an array of test plans. |
|
|
Indicates that one or more properties in the request body are invalid. |
|
|
Request didn't include the authorization token or the token is invalid or expired. Or, the client doesn't have access to the resource. |
|
|
Indicates that the authorization token is valid, but the requested operation isn't allowed. |
|
|
Requested resource not found. |
|
|
Skill has exceeded the permitted rate limit (specified number of requests per unit of time). The skill can retry the request by using exponential back-off. |
|
|
Error occurred on the server. The skill can retry the request by using exponential back-off. |
|
|
Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request. |