With in-skill purchasing (ISP) now available in Alexa skills, you can sell premium content to enrich your voice experience. You can sell in-skill products as one-time purchases that unlock access to premium content or you can sell subscriptions that offer access to premium content for a period of time. No matter which in-skill product you build, it’s important to create a seamless customer experience and test your ISP to reduce friction in the certification process.
To help you prepare your ISP for testing and certification, check out the following 10 test cases below. To get started, set up the ISP sample skill from GitHub to your local environment:
(Read the install instruction if you have questions about the ISP skill set up.)
Next, choose your testing environment and test accounts.
Test Simulator UI: This is a fast and straightforward testing environment. You can view the JSON I/O for request and response, which can help you quickly troubleshoot issues. You will be able to experience the entire purchase workflow from Test Simulator UI.
Command-Line Interface: Another option is to test from the Alexa Skills Kit (ASK) Command-Line Interface (CLI). The advantage is that you can write a script to automate the workflow. However, currently you are not able to use CLI to test the purchase workflow. For example, if you say, “Alexa, ask Premium Facts to buy the Science Pack” there will be an error message from JSON response. After you buy the ISP, you can use the CLI to test the rest of workflow. For instance, say, “Alexa, ask Premium Facts to tell me more about Science Pack” and the expected response should appear in the Alexa Developer Console.
Another important feature is that every time you purchase an entitlement from developer account, use “ask api reset-isp-entitlement <-i|--isp-id <isp-id>> <-g|--stage <stage>> [-p|--profile <profile>] [--debug]” to reset the purchase. For subscriptions, you can reset by saying “cancel subscription” and then subscribe again.
Devices: You should also test your ISP from an Alexa-enabled device to make sure it can handle required tasks in real-world scenarios. Use the beta testing feature in the Alexa Developer Console to invite customers to use the skill on their own devices for user acceptance testing.
Test Using Your Developer Account: Using your developer account, should be able to cover all testing scenarios. As we mentioned earlier, you can test the purchase for multiple (endless) times using “ask api reset-isp-entitlement” to reset your ISP purchasing back to purchasable.
In order to test your ISP skill, your account needs to be a US account. You can check if it’s a US account by navigating to https://www.amazon.com/. Go to “Your Account,” click “Content and Devices,” “Settings,” and then “Country/Region Settings” to check the country. If the country is not United States, change it to US.
Test with Beta Test Accounts: You will mostly benefit from test accounts when your developer account can’t be configured as a US account. By sending out beta test invites to users’ US accounts, they will be able to test ISP. But in this scenario, the purchase can’t be reset because a user can only test once with the beta test account.
After choosing your testing environment and test accounts, it’s time to set up your test cases. You can follow this check list for functional tests and user experience tests before you submit your ISP skill for certification. Follow the below steps to create each test case, using this ISP sample skill as our example.
Test Case #1: Invoke the purchase suggestion (upsell) scenarios.
Steps:
Expected Result: The upsell message should clearly explain what it should provide to users. The upsell message is provided at the beginning of the sample skill with, “To hear about the premium categories for purchase, say 'What can I buy'.” When navigating through the free content, you should be able to reach the upsell scenario.
Test Case #2: Start your skill and ask to buy each product your skill offers. Try to use the product immediately.
Steps:
Expected Result: The customer should be able to buy each of the ISPs and use them immediately.
Test Case #3: Start your skill and ask to buy each product your skill offers. End the skill session. Resume the skill and try to use each product.
Steps:
Expected Result: Each in-skill product should be available for the user when the session restarts.
Test Case #4: Buy each product your skill offers, and check the purchase card in the Alexa app.
Steps:
Expected Result: In the developer account, there will be an offer card and a purchase card, which will show up in the companion Alexa app or alexa.amazon.com. Find the receipt for your test purchase under your Amazon account digital orders. The information for the purchase should match with what you configured with your ISP. The receipt could be found only if you purchase the ISP from a live skill, but you can see the purchase card in both the developer console and live skill.
Test Case #5: Buy a product and start a flow that would prompt a purchase suggestion if the product had not been purchased.
Steps:
Expected Result: The skill response should be, “Good news, you already have the Science Pack.”
Test Case #6: Set up invalid payment information for your account and try to buy a product.
Steps:
Expected Result: You should be able to purchase using the invalid payment information when test developer stage skill from developer account or a beta test. If you use invalid payment information to purchase the in-skill product in a live skill, the purchase flow will provide an error message.
Test Case #7: Complete a scenario that would result in a purchase suggestion. Decline the suggestion.
Steps:
Expected Result: There shouldn’t be any purchase suggestion in the response.
Test Case #8: Complete a scenario that would result in a purchase suggestion. Accept the suggestion, but decline the purchase prompt.
Steps:
Expected Result: The in-skill product will not be purchased and the session will continue with free content.
Test Case #9: For skills that offer a subscription, cancel the subscription with a voice request.
Steps:
Expected Result: A customer should be able to cancel the subscription “all access” and the skill should have the option to continue using the free content.
Test Case #10: Complete a skill flow that results in a purchase suggestion. Decline the offer, and complete the same flow again.
Steps:
Expected Result: There should not be a purchase suggestion in the same session.
Use these test scenarios with your own ISP skills before submitting for certification. For more tips and best practices when building your ISP, check out the following resources:
When you create delightful skills with compelling content, customers win. You can make money through Alexa skills using in-skill purchasing or Amazon Pay for Alexa Skills. You can also make money for eligible skills that drive some of the highest customer engagement with Alexa Developer Rewards. Learn more about how you can make money with Alexa skills.