Regular skill testing can help you deliver consistently great and reliable voice experiences to customers. But as your skill becomes more complex, testing might become a time-consuming task if you have to test hundreds of possible dialogs manually. The right tools can help you simulate complex interactions in an automated way, which reduces the level of effort spent on repetitive tasks.
Test automation can also help you find bugs early in the cycle of the software development process – and the earlier you find a bug, the more time you have to fix it Furthermore, when multiple people work on the same code base, having test automation before checking in the code can prevent regressions from happening. As a result, your customers will benefit from a consistent and reliable voice experience.
We previously shared a high-level overview of automated testing and some best practices here and here, and introduced a cmd-line simulator called TestFlow. Today’s post shares how to create unit tests to validate a skill’s output against predefined assertions.
The classic test pyramid (referenced in the diagram below) outlines the different types of testing you can leverage to create an effective and balanced test automation suite. You should have many more unit tests than tests that validate the voice interaction with Alexa. In this way, you can address the major functional test cases with unit tests and you can treat the voice user interface (VUI) interaction as a sanity test to ensure the voice experience works as expected.
Unit testing involves breaking Alexa skill backend code into pieces and subjecting each piece to a series of tests. We declare the correct outputs and conditions we expect to see from each unit test. The testing tool will report on the number of passing and failing tests.
There are several open source unit test frameworks available. Bespoken Tools has an example setup within the Fact Skill sample. Another is BrainMacInosh’s Alexa-Skill-Test-Framework. Both of the frameworks currently support the latest Alexa Skills Kit (ASK) Software Development Kit (SDK).
These frameworks are easy to set up and they support different types of skill features like cards, dialog models, Amazon DynamoDB, and session attributes. Under the covers they rely on standard test packages like Jest or Mocha.
In this post, we’ll use BrainMacInosh’s Alexa-Skill-Test-Framework to test a sample skill. If you’d like to follow along on your own device, make sure you have Node.JS and GitHub installed.
Step 1: Clone the sample skill to your local desktop and install the node_modules dependency
Step 2: Set up the test framework from npm
Step 3: Run a sample test case
The code shoud look like this:
Example 1: Unit Test for Hello World Skill
Now that we are setup, we will continue to use the Hello World Skill Test example to learn how to write custom unit tests. This test case test snippet demonstrates how to test the launch request in the Hello World Skill. The script runs “alexaTest.getLaunchRequest()” to simulate a launch request, and then defines the expected speech output under “says”. “repromptsNothing”: true means no reprompt is included in the launch request.
Let’s review the first unit test in your test script, LaunchRequest:
describe("Hello World Skill", function () {
// tests the behavior of the skill's LaunchRequest
describe("LaunchRequest", function () {
alexaTest.test([
{
request: alexaTest.getLaunchRequest(),
says: "Welcome to the Alexa Skills Kit, you can say hello!",
repromptsNothing: true,
shouldEndSession: true
}
]);
});
]);
});
};
After you run the test, errors are displayed. From the screenshot you should see two errors, and we will be focusing on the first one:
Example 2: Unit Test for Checking Session Attributes
Replace the code in index.js with the code from this helloworld.js file. This skill uses session attributes to store values inside a session.
Here is the portion of the code that sets the session attributes in this sample skill; it calls an attributeManager from handlerInput to get all the session attributes and assigns them to a variable. We can customize the session attributes value for this sample, change “attributes.foo = ‘bar’” to “attributes.activity = ‘eating’”.
const HelloWorldIntentHandler = {
canHandle(handlerInput) {
return handlerInput.requestEnvelope.request.type === 'IntentRequest'
&& handlerInput.requestEnvelope.request.intent.name === 'HelloWorldIntent';
},
handle(handlerInput) {
const speechText = 'Hello World!';
const attributes = handlerInput.attributesManager.getSessionAttributes();
attributes.activity = 'eating';
return handlerInput.responseBuilder
.speak(speechText)
.withSimpleCard('Hello World', speechText)
.getResponse();
},
};
To test, in the test file under the “describe” block, we simply need to put the expected pair of values after hasAttributes. If you are using version 1 of the ASK SDK, the test method implementation is the same. Here is the sample test code:
//tests the behavior of the Skill's HelloWorldIntent
describe("HelloWorldIntent", function () {
alexaTest.test([
{
request: alexaTest.getIntentRequest("HelloWorldIntent"),
says: "Hello World!", repromptsNothing: true, shouldEndSession: true,
hasAttributes: {
activity: ‘eating'
}
}
]);
});
};
Hopefully you have a better understanding how to leverage unit testing to ease your Skill testing. Reach out via the Alexa developer forums to learn more!