Unit Testing: Creating Functional Alexa Skills

Kaiyin Hu Aug 07, 2018
Share:
Tutorial Tips & Tools
Blog_Header_Post_Img

Regular skill testing can help you deliver consistently great and reliable voice experiences to customers. But as your skill becomes more complex, testing might become a time-consuming task if you have to test hundreds of possible dialogs manually. The right tools can help you simulate complex interactions in an automated way, which reduces the level of effort spent on repetitive tasks.

Test automation can also help you find bugs early in the cycle of the software development process – and the earlier you find a bug, the more time you have to fix it Furthermore, when multiple people work on the same code base, having test automation before checking in the code can prevent regressions from happening. As a result, your customers will benefit from a consistent and reliable voice experience. 

We previously shared a high-level overview of automated testing and some best practices here and here, and introduced a cmd-line simulator called TestFlow. Today’s post shares how to create unit tests to validate a skill’s output against predefined assertions.

Types of Automated Tests for Alexa Skills

The classic test pyramid (referenced in the diagram below) outlines the different types of testing you can leverage to create an effective and balanced test automation suite. You should have many more unit tests than tests that validate the voice interaction with Alexa. In this way, you can address the major functional test cases with unit tests and you can treat the voice user interface (VUI) interaction as a sanity test to ensure the voice experience works as expected.

Alexa Blog

Unit testing involves breaking Alexa skill backend code into pieces and subjecting each piece to a series of tests. We declare the correct outputs and conditions we expect to see from each unit test. The testing tool will report on the number of passing and failing tests. 

Choosing a Unit Test Framework

There are several open source unit test frameworks available. Bespoken Tools has an example setup within the Fact Skill sample. Another is BrainMacInosh’s Alexa-Skill-Test-Framework. Both of the frameworks currently support the latest Alexa Skills Kit (ASK) Software Development Kit (SDK).

These frameworks are easy to set up and they support different types of skill features like cards, dialog models, Amazon DynamoDB, and session attributes. Under the covers they rely on standard test packages like Jest or Mocha.

In this post, we’ll use BrainMacInosh’s Alexa-Skill-Test-Framework to test a sample skill. If you’d like to follow along on your own device, make sure you have Node.JS and GitHub installed.

Sample Skill Unit Testing

Step 1: Clone the sample skill to your local desktop and install the node_modules dependency

Step 2: Set up the test framework from npm

  • Make sure you are under the project root folder.
  • From the root, create a folder and name it “test.”

Alexa Blog

  • Under “test” folder, run the command to install the test framework:
    • npm install alexa-skill-test-framework --save-dev
    • npm install mocha --global
  • Check the node_modules folder under “test” folder to verify the alexa-skill-test-framework module is installed successfully.

Step 3: Run a sample test case

  • Copy the Hello World sample test file: helloworld-tests.js from GitHub and put it under the “test” folder.
  •  Make the following updates to helloword-test.js:
    • Remove the comment out from “const alexaTest = require(‘alexa-skill-test-framework’”.
    • Comment out “const alexaTest = require(‘../index’).
    • Change alexaTest.initialize(require(‘./helloword.js’), to alexaTest.initialize(require(‘../lambda/custom/index.js’).

The code shoud look like this:

Alexa Blog

  • Navigate to “test” folder, run “mocha helloworld-tests.js”. You will be able to see the test result for your Hello World sample Skill.

Write Your Own Tests Under Test Folder

Example 1: Unit Test for Hello World Skill

Now that we are setup, we will continue to use the Hello World Skill Test example to learn how to write custom unit tests. This test case test snippet demonstrates how to test the launch request in the Hello World Skill. The script runs “alexaTest.getLaunchRequest()” to simulate a launch request, and then defines the expected speech output under “says”. “repromptsNothing”: true means no reprompt is included in the launch request.

Let’s review the first unit test in your test script, LaunchRequest:

Copied to clipboard
describe("Hello World Skill", function () {  
// tests the behavior of the skill's LaunchRequest  
 describe("LaunchRequest", function () {  
 alexaTest.test([  
    {  
          request: alexaTest.getLaunchRequest(),  
          says: "Welcome to the Alexa Skills Kit, you can say hello!",  
          repromptsNothing: true,  
          shouldEndSession: true  
            }  
       ]);  
   });  
]);
});
};

After you run the test, errors are displayed. From the screenshot you should see two errors, and we will be focusing on the first one:

Alexa Blog

Example 2: Unit Test for Checking Session Attributes

Replace the code in index.js with the code from this helloworld.js file. This skill uses session attributes to store values inside a session.

Here is the portion of the code that sets the session attributes in this sample skill; it calls an attributeManager from handlerInput to get all the session attributes and assigns them to a variable. We can customize the session attributes value for this sample, change “attributes.foo = ‘bar’” to “attributes.activity = ‘eating’”.

Copied to clipboard
const HelloWorldIntentHandler = {

canHandle(handlerInput) {

return handlerInput.requestEnvelope.request.type === 'IntentRequest'

&& handlerInput.requestEnvelope.request.intent.name === 'HelloWorldIntent';

},

handle(handlerInput) {

const speechText = 'Hello World!';

const attributes = handlerInput.attributesManager.getSessionAttributes();

attributes.activity = 'eating';

return handlerInput.responseBuilder

.speak(speechText)

.withSimpleCard('Hello World', speechText)

.getResponse();

},

};

To test, in the test file under the “describe” block, we simply need to put the expected pair of values after hasAttributes. If you are using version 1 of the ASK SDK, the test method implementation is the same. Here is the sample test code:

Copied to clipboard
//tests the behavior of the Skill's HelloWorldIntent

describe("HelloWorldIntent", function () {

alexaTest.test([

{
request: alexaTest.getIntentRequest("HelloWorldIntent"),
says: "Hello World!", repromptsNothing: true, shouldEndSession: true,
hasAttributes: {
activity: ‘eating'

}
}
]);
});
};