New Smart Home Tools and Metrics to Debug, Test, and Manage the Performance of Your Smart Home Skills

When implementing your skill, you will need to validate that your smart home skill is sending the right responses back to Alexa. You can now use the Smart Home schemas to validate your skill. These schemas can be used with any JSON Schema Validator and can help you validate and correct errors for any "response" JSON object for your Smart Home skills. To learn more about this tool and supported capabilities, visit the Alexa Smart Home GitHub.

During a skill implementation, you will often need to check to see whether your devices are changing to the right state. It can be difficult to verify device state with visual verification. For example, if you are implementing the BrightnessController interface for a smart light bulb, you may want to verify that your device is changing to the correct brightness percentage value when a customer says, “Alexa, turn the light to 50%”. This is hard to visually verify. Now, you can use the State Reporting Test Tool to verify the states for all your test Smart Home devices in your developer account. For more information on this tool, read the technical documentation here.

We also built the Smart Home Live Debugger tool to help you debug ChangeReport, AddOrUpdateReport, and DeleteReport events that Alexa receives from your skill. These events allow you to keep Alexa updated when a customer interacts with your device outside of Alexa, for example, turning on a smart light through a switch, or renaming a smart device through your app. These reports keep Alexa in sync with your device’s latest state change and help ensure a consistent customer experience. The Smart Home Live Debugger will debug asynchronous events that Alexa receives, including the JSON for your request and processing errors (if any) for those events. You can access the Smart Home Live Debugger under Test tab in the Alexa Skills Kit (ASK) developer console or learn more about it here.

Now you can use Smart Home Tests tool to verify your lighting or thermostat device’s functionality before submitting it for Works with Alexa (WWA) Certification. We’ve added a set of automated tests in the Alexa Skills Kit developer console that will send directives to your skill and verify that it correctly sets the state of the device. With the Smart Home Test tool, you can reduce the time and cost to certify your device by quickly testing that your smart home skill is correctly handling Alexa directives for the capabilities it supports, reducing manual testing efforts and improving your skill’s quality. The tool currently provides automated tests for the following Alexa interfaces: BrightnessController, ColorController, ColorTemperatureController, PowerController, and ThermostatController. With the tool, you’ll be able to select a test device to run the automated tests against, download a summary report, and download detailed test execution results. To get started, read our documentation and navigate to the Test tab in the Alexa Skills Kit developer console.

To help you understand how end-users interact with your Alexa skill, we’ve built a set of usage metrics including latency and success rates for device discovery, device control, query state requests, and change reports. To view your skill metrics reports, visit the Analytics page in the Alexa Skills Kit developer console or read the technical documentation here. We also launched a new feature in the Analytics page that allows you to download message IDs and associated metadata such as latency, errors, and exceptions for skill failures which will help you you troubleshoot and solve issues that impact your customers. Learn more about downloading message IDs here.

You can debug, test, and improve your Alexa Smart Home skill today by accessing these smart home tools and metrics in the developer console, or learn more by visiting the technical documentation below:

• Smart Home Debugger – View the ChangeReport, AddOrUpdateReport, and DeleteReport events that Alexa receives from your skill, including your JSON, and any processing errors.
• Smart Home Test – Run automated tests that send directives to your skill and verify that you set the state correctly. Available for some of the most common Alexa interfaces.
• State Reporting Test Tool – View the current values of all properties of smart home devices that are associated with your skill and your developer account, and whether the current value came from a StateReport request or proactive reporting such as a ChangeReport.