Test plan issues
You might encounter errors when creating or running a test plan in the developer console. ATS displays errors in an error banner at the top of the page. You can dismiss the banner after you review the error and make corrections.Issue: General errors
You encounter one or more errors when you try to create or run a test plan.Symptoms
Errors appear in error banners at the top of the page.Try this
Review the following table for possible causes and solutions. The error messages you see might be more specific than the ones listed here.| Error message | What to do |
|---|---|
The request was invalid. Please check your inputs and try again. | Review your selections for device and test plan. Make sure that you fill all required fields. |
You are not authenticated. Please log in again. | Your session expired. Log out and log back in. |
You do not have permission to perform this action. | Use the correct vendor account in the Alexa developer console. |
The requested resource was not found. | The test plan or report might have been deleted. Refresh the page and verify the resource exists. |
The request timed out. Please try again. | Retry the request. |
There was a conflict with the current state. Please refresh and try again. | Refresh the page and retry. |
Too many requests. Please wait a moment and try again. | Wait about a minute before trying again. |
An internal server error occurred. Please try again later. | Wait several minutes and retry the request. |
The server received an invalid response. Please try again later. | Wait several minutes and retry the request. |
The service is temporarily unavailable. Please try again later. | The service might be under maintenance. Try again later. |
The server timed out. Please try again later. | Wait several minutes and retry the request. |
Unable to reach the server. Please check your network connection and try again. | Check your internet connection, and then refresh the page and try again. |
An unexpected error occurred. Please try again. | Refresh the page and try again. |
Something went wrong. Please try refreshing the page. | Hard-refresh the page using the Cmd+Shift+R (Mac) or Ctrl+Shift+R (Windows) shortcut. |
Test plan validation timed out. Please try again. | Test plan preparation took too long. Try running the test again. |
Test plan validation failed. Please try again. | ATS couldn’t load the test plan. Check the test plan for errors and try running the test again. |
| Error label | Description |
|---|---|
Run Test | The test failed to run. |
Test Validation | Test plan preparation failed. |
Load Test Plan | The test plan didn’t load. |
Load Device | Your smart home device didn’t load. |
Load Alexa Device | Your Alexa device didn’t load. |
Test Run History | The test history table failed to load. |
Live Test Status | Live status updates failed. |
Test run errors
For each test run, review the test report details and associated error messages. For add-on-related errors, review the Amazon Web Services (AWS) CloudWatch logs generated by your Lambda function. To verify that you can control your device outside of the test run, use voice commands with Alexa and verify your device works in your device app.Issue: “ERROR”: “Failed to switch to the specified endpoint”
The Echo device can’t connect to the specified endpoint.Symptoms
The setup test case fails with the error,Failed to switch to the specified endpoint.
Try this
To resolve the issue, try the following options:- Register the Echo device under the same developer account as your smart home device.
- Make sure your Echo device is online and connected to the Wi-Fi network.
- Power off the Echo device, and then power it back on.
Issue: “ERROR”: “Not able to retrieve Device state: The device is offline”
The device is offline or unreachable.Symptoms
The test times out waiting for a response from your device and returns the error,The device is offline.
Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| Device is offline | Check the power to the device. Verify the network connection. Then, restart the device. |
| Network connectivity issues | Check the router settings. Test device from device app. Test device from the Alexa app. Try to control the device manually and with Alexa by voice. |
| Echo device issues | Verify that the Echo has power and that mute is off. Make sure the Echo is within range of your smart home device. Try to control the device manually and with Alexa by voice. |
| Account linking expired | Open the Alexa app, navigate to More > Alexa+ Store > Browse Alexa Skills & Games, and then re-link the accounts. |
| Device not discovered | Open the Alexa app, navigate to Devices > Discover Devices, and verify that the device appears in the list. |
Issue: Text-to-speech (TTS) response validation failures
The Alexa response doesn’t match the expected response.Symptoms
TTS Response failed with validation errors, such as the following examples.Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| Expected response is too specific | Update the test case to use shorter phrases. Try “OK” instead of full response text. |
| Alexa returned an error message | Check the actual response in the test report and address the underlying error. |
| Device returned an error | Check the AWS CloudWatch logs. For more details, see View the log entries. For devices that use a smart home add-on, review your add-on’s error handling. |
| Locale or language mismatch | Verify that the Alexa device locale matches the locales supported by your add-on. |
Issue: “ERROR”: “Reported Device state not matching expected state”
The Device State Change (State Match) exact match validation failed.Symptoms
The test fails with the error,Reported Device state not matching expected state or the device state is incorrect after the command.
Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| Device state didn’t update | Verify device state in the device app. Increase the wait time in the test case. For more details, see Wait time guidance. |
| State reporting not implemented | Verify that the add-on implements state reporting. Verify the ReportState directive.Check the reported ChangeReport and StateReport events. |
| Expected state value is incorrect | Check actual state value in the test report. Verify associated reportable properties, such as hue, saturation, brightness (HSB) color values, and temperature scale. |
| State property name is incorrect | Verify correct property name and namespace for the capability in the test case. |
| Device reports state differently | Adjust the expected values in the test case. For approximate matching, update the test case to use the Device State Change (Natural Language State Match) validation. |
Issue: “ERROR”: “Not able to retrieve Device state: Unable to retrieve state for this device.”
The testing service can’t retrieve the state for your device.Symptoms
The test fails with the error,Not able to retrieve Device state: Unable to retrieve state for this device or the device state isn’t returned.
Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| Network connectivity issues | Check the router settings. Test device from device app. Test device from the Alexa app. Try to control the device manually and with Alexa by voice. |
| State reporting not implemented | Verify that the add-on implements state reporting. Verify the ReportState directive.Check the reported ChangeReport and StateReport events. |
| State property name is incorrect | Verify correct property name and namespace for the capability in the test case. |
Issue: “ERROR”: “Test step timed out”
The device state didn’t update in the allotted time.Symptoms
The test fails with the error,Test step timed out or no response from the device or Alexa.
Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| Device is slow to respond | Increase the wait time in the test case. For more details, see Wait time guidance. |
| Network latency | Test network speed. Check for network congestion. Try at a different time. |
| Add-on processing is slow | Review Lambda execution time in CloudWatch. Optimize your add-on code. |
| Device cloud is slow | Monitor device cloud health. Consider implementing caching in your add-on. |
Issue: “ERROR”: “Account linking failed”
Account linking between the Alexa app and your device cloud failed.Symptoms
The test fails with the error,Account linking failed or you see Authorization failed or Invalid token errors.
Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| OAuth token expired | Re-link the accounts in Alexa app. Verify the token expiration settings in your OAuth server configuration. |
| Account linking not configured | Verify the account linking configuration for the smart home add-on under test in the Alexa developer console. Test the account linking flow manually. |
| Incorrect credentials | Verify username and password are correct. Try to sign in to the device app with the same credentials. |
| OAuth server issues | Check OAuth server logs to verify the server endpoints are accessible. |
Issue: “ERROR”: “Device not found”
The test couldn’t find your smart home device.Symptoms
The test fails with the error,Device not found or you don’t see the device in the device list in the Alexa app.
Try this
Review the following table for possible causes and solutions.| Cause | Action |
|---|---|
| Device not discovered | Run discovery in Alexa app and wait 20–30 seconds for discovery to complete. |
| Device name mismatch | Check the exact device name in Alexa app. Names are case-sensitive. Make sure you select the same name in the Run a test > Device Name selection list. |
| Device deleted or disabled | Verify the device is enabled in the Alexa app and run discovery, if needed. |
| Add-on disabled | Re-enable the smart home add-on in Alexa app and verify the add-on is in development mode. |
