Console

Settings

Sign out

Notifications

Alexa

Amazon Appstore

Ring

AWS

Documentation

Support

My Cases

Console

Alexa+ Smart Home Docs

Get started

Connect with Matter

Alexa Connect Kit

Add-ons

Certify

Resources

Run Test Plans

Smart home skills are now called smart home add-ons. No action is needed, and your existing integration will continue to operate normally.

Important: Amazon offers this feature as beta. The feature might change as Amazon receives feedback and iterates on the functionality.

After you create a test plan, use the Alexa Automated Testing Service (ATS) to run the plan against your device and review the results. ATS sends voice commands to your device through Alexa, validates the device response and state changes, and generates a detailed test report.

Before you begin

Before you run a test plan, verify the following:

Your Echo device is powered on, connected to the internet, not muted, and responding to voice commands.
Your Echo device is in the same room as, or within range of, your device under test.
Both the Echo device and the device under test are registered to the same Amazon developer account.
Both devices are visible in the Alexa app.

Important: Don't interact with your Echo device while a test is running. Voice commands, alarms, timers, or notifications can interfere with test execution and cause unexpected failures.

Preview a test plan

Before you run a test, you can review the test plan to verify the test cases and steps.

To preview your test plan

Under Run a Test, for Test Plan Name, select the test plan that you want to review.
To view the test plan, choose Preview Test Plan.
To return to the Alexa Automated Testing Service page, choose Back.

Run a test plan

Select and run your test plan on ATS. You can run test plans based on pre-built or custom test cases. The testing service displays the Echo devices and devices under test registered to your Alexa app. Choose the appropriate devices for the test run.

Note: Wait for the test plan to complete before you run another test plan on the same device.

To run a test in the Alexa developer console

Under Run a Test, choose the following items:
1. For Test Plan Name, select the test plan that you created.
2. For Device Name, select the name of your device under test.
3. For Alexa Device, select the name of your Alexa-enabled device.
To run the test, choose Run Test.
Wait for the tests to complete.
If an error appears at the top of the page, correct the error, and then rerun the test plan.

View test results

After the tests in the selected test plan complete, the testing service generates a test report that shows the overall test results, individual test case results, and error messages for any failures. The report also includes detailed step-by-step execution and device state information.

When a test case fails, use the error messages to help debug the issues. For more help, see Alexa Automated Testing Service Troubleshooting Guide. After you update your add-on or device, rerun the test plan.

Note: ATS stores test plans and test reports for 30 days and then automatically deletes them. Save your test plan and download test reports before they expire.

To view the test report for a specific test run

In the Test Run History table, navigate to the most recent entry of your test plan with the status COMPLETED.
Under ACTIONS in your test plan's row, choose View Report.
Under Test Report, view the test run ID, test plan name, and overall test status.
Under Device Information, view the device name and Appliance ID.
Under Test Summary, view the total number of test cases run, number of test cases that succeeded, and the number that failed.
Under Test Cases, you can view the status of each test case: Success or Failed. If the test case failed, the report includes an error message.
To expand the test case results, choose the arrow icon ().
You can view the results of each step in the test case.
To return to the Alexa Automated Testing Service page, choose Back.

Test status values

During your test run, you might see one of the following statuses in the Test Run History table.

Status	Description
IN PROGRESS	The test cases in the specified test plan are running.
COMPLETED	Test run finished. The results are ready to view.
TERMINATED	An internal error occurred during test execution.

Download test reports

Follow these steps to download your test report.

Note: The testing service deletes test reports after 30 days.

To download your test report

In the Test Run History table, navigate to the most recent entry of your test plan with the status COMPLETED.
Under ACTIONS in your test plan's row, choose Download Report.
View the JSON-formatted report download in your default download location.

Save your test plan

You might want to include your test plan in your CI/CD pipeline. Follow these steps to save your test plan. If your test plan expires, you can create a new test plan based on the saved version.

To save your test plan

In the Test Run History table, navigate to the most recent entry of your test plan with the status COMPLETED.
Under ACTIONS in your test plan's row, choose View Test Plan.
The JSON-formatted test plan opens in a separate browser tab or window.
To save your test plan to your computer, choose Save.

Was this page helpful?

Provide feedback

Last updated: May 01, 2026