Use the Smart Home Tests

You can use Smart Home Test to run a series of automated tests on your Alexa Smart Home skill. The automated tests send directives to your skill and verify that you set the state correctly. By using the Smart Home Test tool, you can verify that you have implemented the Alexa interfaces correctly in your skill. For more information about other test tools, see Test and Debug Your Smart Home Skill.

The Smart Home Test tool provides automated tests for the following Alexa interfaces: BrightnessController, ColorController, ColorTemperatureController, PowerController, ThermostatController.

Overview

You run the Smart Home tests in the Alexa developer console. Each Smart Home Test sends directives to your skill, and then compares the expected device state with the actual device state. You can see the results of your tests in the console. If you want to see the contents of each test, you can download them from the console or view them on GitHub at alexa/alexa-smarthome/capability_evaluations/test_plans/.

You can run the Smart Home Tests on your Smart Home skill before you submit it for certification to reduce the potential for errors during certification. A successful Smart Home Test result does not guarantee certification. For more information about certification, see Smart Home Skill Publishing Guide and Works with Alexa Overview.

The following image shows the Smart Home Test page in the Alexa developer console:

The Smart Home Test page in the Alexa developer console

Run tests

Use the following procedure to run a new test on your Smart Home skill.

To run a test on your skill

  1. Open the Alexa developer console and log in.
  2. Click the name of the Smart Home skill that you want to test.
  3. Click the Test link to open the test page.
  4. Select the Smart Home Test option.
  5. For Device, choose the device that you want to test with your skill.
  6. For Capabilities, choose one or more Alexa interfaces that you want to test. You can choose more than one interface for each test. Some interfaces require additional information. For example, if you choose the Alexa.ThermostatController, you also choose Celsius or Fahrenheit for the temperature scale.
  7. Click Run Test.

Test status

While a test is running, the status appears as In Progress. After a test completes, the status appears as Passed, Failed, or Error.

The following are the possible statuses:

  • In Progress – The test is currently running.
  • Passed – The test has completed, and all test cases were successful.
  • Failed – The test has completed, and one or more test cases were unsuccessful.
  • Error – The test was not able to run. For example, if the test device was not plugged in.

Manage tests

All your tests appear in the Test Results section of the Test page. You can expand each test result row to see the details for the test. Test results remain in the Test Results section for 30 days.

Use the menus that precede the test results to filter the results. You can filter the results by device, interface, status, and age of the test.

Use the menu in the ACTIONS column to manage your test results. From the ACTIONS menu, you can do the following:

  • New run – Rerun a previous test with the same options. For example, rerun a failed test after updating your code.

  • Download test results – Download the details of the test including the expected states and actual states.

  • Download test cases – Download the details of the test cases for the test.