Amazon launches new Skill Quality Coach to guide developers to build high quality skills

Staff Writer Jul 20, 2022
Alexa Live Alexa Skills

At Alexa Live 2022, Amazon Alexa announced Skill Quality Coach (SQC), a personalized guide to help skill developers build high-quality skills on Alexa. Skill quality is essential to delivering an engaging customer experience and driving repeat customer engagement. 

SQC learns from thousands of Alexa Skills to identify the common attributes of successful skills. It then evaluates your skill against those identified attributes to create a quality improvement plan for your skill. 

The Skill Quality Coach will also include a Skill Quality Score (SQS). The SQS is a number between 0-5, and is designed to provide a measure of of skill quality. The SQS is calculated based on several aspects of skill design, and measures of implicit and explicit customer feedback. 

The SQC provides Recommended Insights, which are personalized and curated recommendations to help you improve Skill Quality Score.

Skill Quality Score 

SQS is a composite score between 0 and 5, is made up of different components that include: 

  • Skill Design features: We have analyzed thousands of skills and identified common patterns and design elements of skills that have been successful in driving customer engagement and retention. Our machine learning model analyzed more than 120+ different features and design elements to shortlist a handful of features that are hallmarks of successful skills. These design elements predominantly fall into 4 themes: 
    • Rich Multimodal & Voice Responses: High-quality skills leverage rich multi-modal and voice response to engage their customers. With the proliferation of multimodal devices, richness in responses that work across visual and voice modalities, becomes an even more critical element of engaging skill design. 
    • Voice Model Coverage: High-quality skills have accurate and rich voice models that can easily understand the intention behind a customer utterance. For example, high quality skills gracefully handle utterances not supported by the skill by leveraging the AMAZON.FallbackIntent, which enables Alexa to redirect customer requests to the right experience, preventing customer dissatisfaction.
    • Content Richness, Freshness & Personalization: High-quality skills have sufficient content that is kept up to date to drive sustained long-term engagement. These skills are also able to form deep, meaningful and personalized connections with their customers by understanding their needs and request patterns over time. 
    • Endpoint Health: High-quality skills have robust and reliable skill endpoints that provide high utility responses with low latency. An ambient experience is incomplete without a high performing skill endpoint.
  • Customer Experience: The customer experience portion of SQS takes into account customer habituation and customer feedback received through channels such as customer reviews. The customer habituation score measures stickiness of a skill based on how recently and how often customers use the skill. A high score on customer habituation indicates that skill has a higher portion of engaged and repeat customers (fans of your skill) when compared to similar skills. For example: For the two skills in the Games category below, we see that Skill A has greater proportion of customers who are fans of the skill compared to Skill B, which has a lower Customer Habituation score. As Skill A gets more customers, it will be important to retain and engage users to maintain a high customer habituation score. 

Register for Alexa Live

Scores three and above can be interpreted as high. In order to maximize the SQS, a skill needs to score well on both the skill design and customer experience facets. 

Recommended Insights
The Skill Quality Coach provides personalized and curated recommendations to improve SQS. Our recommendation framework anchors around the four key pillars identified for SQS calculations: Rich Multi-modal & Voice Responses, Voice Model Coverage, Content Richness, Freshness & Personalization and Endpoint Health. The recommendations shared, are generated by:

  1. Automatic Insights Engine: The Automatic Insights Engine tracks and analyzes usage of 100+ skill design features, attributes and runtime signals across hundreds of thousands of skills to learn common actions associated with successful skills - skills with high retention and large user bases. It performs near real time evaluation of your published skills to apply these learnings to generate recommendations by taking skill category, locale and other details into account. The engine is constantly learning from feedback and new usage patterns to make recommendations more meaningful, actionable and contextual.
  2. Skill Insights team: The Skills Insights team is a team of testers and builders passionate about building quality conversational experiences. They work backward from the user experience to curate recommendations that focus on delivering a friction-free customer experience. Select high SQS skills will receive manually curated recommendations that complement insights from the Automated Insights engine based on thorough evaluation of their skill .

We’re constantly evolving the insights engine to be more personalized for your skill. Please share your feedback on the insights by voting on insights, so that we can learn from your feedback as well. 

Skill Quality Score, a critical input for developers incentives and skill promotions 
Amazon Alexa also announced the Skill Developer Accelerator Program that provides updated benefits and incentives to custom skill builders for building high-quality skills and growing their business on Alexa. As part of SDAP, we are announcing incentives based on Skill Quality Score for developers who are invested in building high-quality skills on Alexa. This incentive will be available to custom skill developers next year.

Skill Developers in the US can now access Skill Quality Coach by visiting Analytics tab for your skill in the ASK Developer Console. Please review the technical documentation here

Related Articles

Alexa Live 2022: Features and announcements
Amazon announces new Skill Developer Accelerator Program for Alexa
Become a part of customers' daily habits with the Alexa Routines Kit