Use the Smart Home Test Tool
Use the Smart Home Test tool in the Alexa developer console to verify that you implemented the Alexa interfaces correctly in your smart home skill. The automated tests send directives to your skill and verify that you set the state correctly. By using the Smart Home Test tool, you can verify that you implemented the Alexa interfaces in your skill correctly.
The Smart Home Test tool provides automated tests for the following Alexa interfaces: BrightnessController, ColorController, ColorTemperatureController, PowerController, ThermostatController.
For details about other test tools, see Test and Debug Your Smart Home Skill. To invoke the Smart Home Test tool with REST, see Smart Home Evaluation REST API.
You run the Smart Home tests in the Alexa developer console. Each Smart Home Test sends directives to your skill, and then compares the expected device state with the actual device state. You can see the results of your tests in the console. If you want to see the contents of each test, you can download them from the console or view them on GitHub at alexa/alexa-smarthome/capability_evaluations/test_plans/.
You can run the Smart Home Tests on your Smart Home skill before you submit it for certification to reduce the potential for errors during certification. A successful Smart Home Test result doesn't guarantee certification. For more information about certification, see Smart Home Skill Publishing Guide and Works with Alexa Overview.
The following image shows the Smart Home Test page in the Alexa developer console:
Use the following procedure to run a new test on your Smart Home skill.
To run a test on your skill
- Open the Alexa developer console and log in.
- Click the name of the Smart Home skill that you want to test.
- Click the Test link to open the test page.
- Select the Smart Home Test option.
- For Device, choose the device that you want to test with your skill.
- For Capabilities, choose one or more Alexa interfaces that you want to test. You can choose more than one interface for each test. Some interfaces require additional information. For example, if you choose the Alexa.ThermostatController, you also choose Celsius or Fahrenheit for the temperature scale.
- Click Run Test.
While a test is running, the status appears as In Progress. After a test completes, the status appears as Passed, Failed, or Error.
The following are the possible statuses:
- In Progress – The test is running.
- Passed – The test completed, and all test cases were successful.
- Failed – The test completed, and one or more test cases were unsuccessful.
- Error – The test wasn't run. For example, due to the test device not plugged in.
All your tests appear in the Test Results section of the Test page. You can expand each test result row to see the details for the test. Test results remain in the Test Results section for 30 days.
Use the menus that precede the test results to filter the results. You can filter the results by device, interface, status, and age of the test.
Use the menu in the ACTIONS column to manage your test results. From the ACTIONS menu, you can do the following:
New run – Rerun a previous test with the same options. For example, rerun a failed test after updating your code.
Download test results – Download the details of the test including the expected states and actual states.
Download test cases – Download the details of the test cases for the test.
- Steps to Build a Smart Home Skill
- Test and Debug Your Smart Home Skill
- Smart Home Skill Evaluation REST API Reference
Last updated: Sep 13, 2022