Alexa Automated Testing Service Troubleshooting Guide

Having trouble creating or running test plans in the Alexa Automated Testing Service (ATS)? If you encounter errors, review the information in this guide for solutions to common issues.

Test plan issues

You might encounter errors when creating or running a test plan in the developer console. ATS displays errors in a red banner at the top of the page. You can dismiss the banner after you review the error and make corrections.

Issue: General errors

You encounter one or more errors when you try to create or run a test plan.

Symptoms

Errors appear in red banners at the top of the page.

Try this

Review the following table for possible causes and solutions. The error messages you see might be more specific than the ones listed here.

Error message What to do

The request was invalid. Please check your inputs and try again.

Review your selections for device and test plan. Make sure that you fill all required fields.

You are not authenticated. Please log in again.

Your session expired. Log out and log back in.

You do not have permission to perform this action.

Use the correct vendor account in the Alexa developer console.

The requested resource was not found.

The test plan or report might have been deleted. Refresh the page and verify the resource exists.

The request timed out. Please try again.

Retry the request.

There was a conflict with the current state. Please refresh and try again.

Refresh the page and retry.

Too many requests. Please wait a moment and try again.

Wait about a minute before trying again.

An internal server error occurred. Please try again later.

Wait several minutes and retry the request.

The server received an invalid response. Please try again later.

Wait several minutes and retry the request.

The service is temporarily unavailable. Please try again later.

The service might be under maintenance. Try again later.

The server timed out. Please try again later.

Wait several minutes and retry the request.

Unable to reach the server. Please check your network connection and try again.

Check your internet connection, and then refresh the page and try again.

An unexpected error occurred. Please try again.

Refresh the page and try again.

Something went wrong. Please try refreshing the page.

Hard-refresh the page using the Cmd+Shift+R (Mac) or Ctrl+Shift+R (Windows) shortcut.

Test plan validation timed out. Please try again.

Test plan preparation took too long. Try running the test again.

Test plan validation failed. Please try again.

ATS couldn't load the test plan. Check the test plan for errors and try running the test again.

An error label precedes each error and defines the category of errors as shown in the following table.

Error label Description

Run Test

The test failed to run.

Test Validation

Test plan preparation failed.

Load Test Plan

The test plan didn't load.

Load Device

Your smart home device didn't load.

Load Alexa Device

Your Alexa device didn't load.

Test Run History

The test history table failed to load.

Live Test Status

Live status updates failed.

Test run errors

For each test run, review the test report details and associated error messages. For skill-related errors, review the Amazon Web Services (AWS) CloudWatch logs generated by your Lambda function. To verify that you can control your device outside of the test run, use voice commands with Alexa and verify your device works in your device app.

Issue: "ERROR": "Failed to switch to the specified endpoint"

The Echo device can't connect to the specified endpoint.

Symptoms

The setup test case fails with the error, Failed to switch to the specified endpoint.

Try this

To resolve the issue, try the following options:

  • Register the Echo device under the same developer account as your smart home device.
  • Make sure your Echo device is online and connected to the Wi-Fi network.
  • Power off the Echo device, and then power it back on.

Issue: "ERROR": "Not able to retrieve Device state: The device is offline"

The device is offline or unreachable.

Symptoms

The test times out waiting for a response from your device and returns the error, The device is offline.

Try this

Review the following table for possible causes and solutions.

Cause Action

Device is offline

Check the power to the device.
Verify the network connection.
Then, restart the device.

Network connectivity issues

Check the router settings.
Test device from device app. Test device from the Alexa app. Try to control the device manually and with Alexa by voice.

Echo device issues

Verify that the Echo has power and that mute is off. Make sure the Echo is within range of your smart home device.
Try to control the device manually and with Alexa by voice.

Account linking expired

Open the Alexa app, navigate to More > Alexa+ Store > Browse Alexa Skills & Games, and then re-link the accounts.

Device not discovered

Open the Alexa app, navigate to Devices > Discover Devices, and verify that the device appears in the list.

Issue: TTS response validation failures

The Alexa response doesn't match the expected response.

Symptoms

TTS Response failed with validation errors, such as the following examples.

Error:
"Device woke": "False"

or

Error:
"Device woke": "True",
"Device responded": "ok",
"Intent": "TurnOffApplianceIntent",
"Device heard": "Alexa, turn off the Kitchen Light"

Try this

Review the following table for possible causes and solutions.

Cause Action

Expected response is too specific

Update the test case to use shorter phrases.
Try "OK" instead of full response text.

Alexa returned an error message

Check the actual response in the test report and address the underlying error.

Device returned an error

Check the AWS CloudWatch logs. For more details, see View the skill log entries. For devices that use a smart home skill, review your skill's error handling.

Locale or language mismatch

Verify that the Alexa device locale matches the locales supported by your skill.

Issue: "ERROR": "Reported Device state not matching expected state"

The Device State Change (State Match) exact match validation failed.

Symptoms

The test fails with the error, Reported Device state not matching expected state or the device state is incorrect after the command.

Try this

Review the following table for possible causes and solutions.

Cause Action

Device state didn't update

Verify device state in the device app.
Increase the wait time in the test case. For more details, see Wait time guidance.

State reporting not implemented

Verify that the skill implements state reporting.
Verify the ReportState directive.
Check the reported ChangeReport and StateReport events.

Expected state value is incorrect

Check actual state value in the test report. Verify associated reportable properties, such as hue, saturation, brightness (HSB) color values, and temperature scale.

State property name is incorrect

Verify correct property name and namespace for the capability in the test case.

Device reports state differently

Adjust the expected values in the test case.
For approximate matching, update the test case to use the Device State Change (Natural Language State Match) validation.

Issue: "ERROR": "Not able to retrieve Device state: Unable to retrieve state for this device."

The testing service can't retrieve the state for your device.

Symptoms

The test fails with the error, Not able to retrieve Device state: Unable to retrieve state for this device or the device state isn't returned.

Try this

Review the following table for possible causes and solutions.

Cause Action

Network connectivity issues

Check the router settings.
Test device from device app. Test device from the Alexa app. Try to control the device manually and with Alexa by voice.

State reporting not implemented

Verify that the skill implements state reporting.
Verify the ReportState directive.
Check the reported ChangeReport and StateReport events.

State property name is incorrect

Verify correct property name and namespace for the capability in the test case.

Issue: "ERROR": "Test step timed out"

The device state didn't update in the allotted time.

Symptoms

The test fails with the error, Test step timed out or no response from the device or Alexa.

Try this

Review the following table for possible causes and solutions.

Cause Action

Device is slow to respond

Increase the wait time in the test case. For more details, see Wait time guidance.

Network latency

Test network speed.
Check for network congestion. Try at a different time.

Skill processing is slow

Review Lambda execution time in CloudWatch. Optimize your skill code.

Device cloud is slow

Monitor device cloud health.
Consider implementing caching in your skill.

Issue: "ERROR": "Account linking failed"

Account linking between the Alexa app and your device cloud failed.

Symptoms

The test fails with the error, Account linking failed or you see Authorization failed or Invalid token errors.

Try this

Review the following table for possible causes and solutions.

Cause Action

OAuth token expired

Re-link the accounts in Alexa app.
Verify the token expiration settings in your OAuth server configuration.

Account linking not configured

Verify the account linking configuration for the smart home skill under test in the Alexa developer console. Test the account linking flow manually.

Incorrect credentials

Verify username and password are correct. Try to sign in to the device app with the same credentials.

OAuth server issues

Check OAuth server logs to verify the server endpoints are accessible.

Issue: "ERROR": "Device not found"

The test couldn't find your smart home device.

Symptoms

The test fails with the error, Device not found or you don't see the device in the device list in the Alexa app.

Try this

Review the following table for possible causes and solutions.

Cause Action

Device not discovered

Run discovery in Alexa app and wait 20–30 seconds for discovery to complete.

Device name mismatch

Check the exact device name in Alexa app. Names are case-sensitive.
Make sure you select the same name in the Run a test > Device Name selection list.

Device deleted or disabled

Verify the device is enabled in the Alexa app and run discovery, if needed.

Skill disabled

Re-enable the smart home skill in Alexa app; verify the skill is in development mode.

Issue: "SKIPPED"

You see a SKIPPED status on one or more test steps.

Symptoms

The test fails on a previous step and causes the service to skip the step indicated.

Try this

ATS skips remaining steps in a test case after a previous step fails. Correct the error on the previous step, and then rerun the test.

Was this page helpful?

Last updated: May 01, 2026