About Skill Metrics
To help you understand how your users interact with your Alexa skills, Amazon provides a set of tools to generate reports of usage metrics for your skills. To view your skill metrics reports, visit the Analytics page in the Alexa Skills Kit developer console. You're also able to access skill metrics with the Metrics API.
Access the Alexa skill Analytics dashboard
To access the Analytics dashboard and learn how to generate the available reports, see View Skill Metrics.
Available skill metric categories
Different metrics are available for different categories of skills, such as Custom or Smart Home skills. (No metrics are available for Meeting skills.) Most metrics are available for skills at both the development and live stages. All metrics data excludes invocations from the Test your skill page in the developer console.
The following table provides an overview of the different categories of metrics available for your skills:
| Metric | Description | Skill types |
|---|---|---|
| Actions | Unique users per action, total actions, and total utterances per action. | Smart Home |
| Customers | Total number of unique users who have accessed the skill. | Custom Flash Briefing Smart Home |
| Intents | Unique users per intent, total utterances per intent, total intents, and failed intents. | Custom |
| Interaction Path | Paths users take when interacting with your skill. For example, view the percentage of users who invoked a particular intent after the LaunchRequest. |
Custom |
| Plays | Total number of times that a user played the skill content. | Flash Briefing |
| Retention (live skills only) | Usage of your skill over time by groups of users or cohorts. View the number or percentage of users who returned to your skill over a 12-week period. | Custom |
| Sessions | Total sessions, successful session types (sessions that didn't end due to an error), average sessions per user. Includes a breakdown of successful, failed, and no-response sessions as a percentage of total sessions. | Custom |
| Utterances | Metrics for utterances depend on the skill category.
|
Custom Smart Home |
Summary of skill metrics
The Summary section of the Analytics page displays an overview of the metrics available for the skill. For each metric, you have options to view infographic-style charts, grids, or other visualizations. Available metric data export formats are PNG or JPEG images for infographics, or CSV format for raw data.
Skill activation and token retrieval metrics
For skills that use account linking, the Analytics page has an Account Linking section that displays metrics related to skill activation and token retrieval. These metrics are available for live skills.
To understand these metrics, first learn about account linking initiations and completions. For more details, see Understand Account Linking.
The following table shows the metrics that are available under Skill Activation. These metrics show data up to two days before the viewing time.
| Metric | Description | Skill types |
|---|---|---|
| Total Enablements |
Definition of this metric depends on the type of skill.
|
Custom (without account linking) Flash Briefing Music (without account linking) Smart home Video |
| Total Account Linking Events | Total number of account linking initiations and completions. | Custom (with account linking) Music (with account linking) Smart home |
| Account Linking Completion Rate | Percentage of successful account links. The percentage is the total number of account linking completions divided by the total number of initiations. Available separately for account linking in the Alexa app (the Alexa-app-only flow and app-to-app account linking that starts from the Alexa app) and account linking in your app (app-to-app account linking and the Login with Amazon fallback flow). |
Custom (with account linking) Music (with account linking) Smart home |
The following table shows the metrics that are available under Token Retrieval. These metrics appear in near real-time (within a few minutes of being emitted).
| Metric | Description | Skill types |
|---|---|---|
| Access Token Requests: Success Rate | Percentage of successful access token requests from the Alexa service to your skill's authorization server. For details, see Overview of the authorization code grant flow. | Custom (with account linking - authorization code grant) Music (with account linking) Smart home |
| Refreshing Access Token Requests: Success Rate | Percentage of successful access token refresh requests from the Alexa service to your skill's authorization server. For details, see Overview of the authorization code grant flow. | Custom (with account linking - authorization code grant) Music (with account linking) Smart home |
| Access Token Requests: Error Responses | Number of error responses to access token requests from the Alexa service to the authorization server for the skill. For details on errors that come from OAuth 2.0, see Error Response in the OAuth 2.0 authentication framework specification. If the Alexa service receives an OAuth 2.0 error that it doesn't recognize, it displays it as an UNKNOWN_ERROR_CODE. A TRANSIENT_ERROR indicates that the Alexa service wasn't able to communicate with the partner service. For other error situations, Alexa returns a null error. |
Custom (with account linking - authorization code grant) Music (with account linking) Smart home |
| Refreshing Access Token Requests: Error Responses | Number of error responses to access token refresh requests from the Alexa service to your skill's authorization server. For details on errors that come from OAuth 2.0, see Error Response in the OAuth 2.0 authentication framework specification. If the Alexa service receives an OAuth 2.0 error that it doesn't recognize, it displays it as an UNKNOWN_ERROR_CODE. A TRANSIENT_ERROR indicates that the Alexa service wasn't able to communicate with the partner service. For other error situations, Alexa returns a null error. |
Custom (with account linking - authorization code grant) Music (with account linking) Smart home |
Operational metrics
On the Analytics page, in the Operational Metrics section, the Discovery, Capability Directives, and Report State pages display the latency and success rate metrics for device discovery, device control, and query state requests, respectively. For each metric, infographics, such as charts, grids, and other visualizations are available. Export infographics in PNG or JPG file formats, or download raw data in CSV file format.
The reports for Operational Metrics are based on the following analytics:
- Latency – Latency is the time until Alexa receives a response from your skill, after Alexa sends a request to your skill. Amazon measures latency through percentile statistics, such as P99, P90, and P50. Use the operational statistics menu (+ icon) in the latency graphs to choose different percentile statistics or the average. A single graph displays multiple statistics options.
- Success rate – Success rate is the total number of successful responses Alexa receives from your skill divided by the total number of requests Alexa sends to your skill.
- Request volume – Both latency and success rate graphs show request volume on the second Y-Axis. Use the + icon on the graphs to toggle the request volume on or off.
- Accuracy rate – Accuracy rate indicates the percentage of times that the response to a report state directive matched with a previously sent change report.
The following reports are available for Operational Metrics:
- Discovery – Alexa sends a discovery request to your skill to discover your user’s smart devices. The Discovery page shows the latency and success rate graphs for all discovery requests to your skill. For more details about discovery, see Alexa Discovery interface.
- Capability Directives – When a user asks Alexa to control their smart device, Alexa sends a directive request to your skill to trigger a capability, for example, turning on a light. The Capability Directives page shows Latency and Success Rate tabs. Both tabs show multiple graphs, one for the aggregate of all capability directive requests, and one per capability for all capabilities supported by your skill. The capability graphs show the aggregate across all directives and the breakdown per directive for all directives under that capability. For more details about capability directives, see Understand the Smart Home Skill API.
- Report State – Alexa sends a report state directive to your skill to request the state of an endpoint. The Report State page shows the latency and success rate graphs for all the report state requests to your skill. For more details about report state, see State Reporting for a Smart Home Skill.
- Change Report – When the state of a device changes for any reason, such as a user turns on a light, the device reports that change to Alexa in a
ChangeReportevent. Alexa can then communicate that status change to the user. The Change Report page shows the accuracy rate of these change reports when Alexa sends aReportStatedirective as perceived by Alexa. For more details about change reports, see Report state in an Alexa.ChangeReport.
In-skill purchasing (ISP) metrics
Alexa has two categories of in-skill purchasing options for skills:
- Entitlements and consumables – Entitlements and consumables are single transaction in-skill purchases. An entitlement unlocks access to features or content within a skill, such as access to a new level within a game. A consumable provides content or features that are purchased, depleted, and re-purchased repeatedly, such as in-game currency or extra lives.
- Subscriptions – An in-skill purchase that offers access to premium content or features for a period of time. Users are charged on a recurring basis until they cancel their subscription.
| Metric | Description | Applicable ISP type(s) |
|---|---|---|
| Total Purchases | Total number of offers accepted for all purchases for all entitlements and consumables within a skill. For consumables, Total Purchases includes repeat purchases by the same user. | Entitlements and consumables |
| Purchases | Number of offers accepted for each entitlement or consumable within a skill. For consumables, Purchases includes repeat purchases. | Entitlements and consumables |
| Active Subscriptions | Total number of active subscriptions for your skill, including paid subscriptions with future expiration dates. | Subscriptions |
| Trial to Paid Conversion Rate | Percentage of trial subscriptions that became paid subscriptions. | Subscriptions |
| Churn | Percentage of active paid subscriptions that expired or that a user canceled. | Subscriptions |
| Subscriptions Started | Total number of new subscriptions. | Subscriptions |
| Offer Impressions | Total number of times users heard the purchase prompt from Amazon, describing a product and its price, and decided whether to buy. Offer Impressions includes both free-trial and paid offers. For consumables, this metric includes both the first-time purchase and repeat purchases of the same item. Offer Impressions doesn't include messages provided within the upsell directive. | Entitlements and consumables, Subscriptions |
| Offer Conversion Rate | Percentage of offer impressions that resulted in the user signing up for a new trial or paid subscription | Subscriptions |
| Upsell to Offer Conversion | The percentage of upsell impressions that resulted in a purchase. Upsell to Offer Conversion is available for the upsell directive only. | Entitlements and consumables, Subscriptions |
| Offer to Purchase Conversion | The percentage of offer impressions that resulted in a purchase. | Entitlements and consumables |
| Upsell to Purchase Conversion | The percentage of upsell impressions that resulted in a purchase. This metric is available for the upsell directive only. | Entitlements and consumables, Subscriptions |
| Subscriptions Ended | The total number of trial subscriptions that a user canceled or paid subscriptions that expired. If a user cancels a paid subscription, the subscription remains active until it expires at the end of the billing period for the user. | Subscriptions |
Skill performance metrics
Use skill performance metrics to view how well your skill is functioning.
- Endpoint Latency – Use Endpoint Latency to see how responsive your skill is to user utterances. Skills with low endpoint latency have slow responses and cause friction for users.
- Endpoint Response – The Endpoint Response metric shows the number of successful and failed responses your skill has over time. Skills that have no response confuse users and lower user satisfaction with your skill. If users aren't responding to your skill, your endpoint-response metrics reflect a low score.
- Intent Confidence – The Intent Confidence metric shows how your voice interaction model is performing. The confidence setting options are high, medium, or low. A low confidence request from a user triggers a re-prompt instead of sending the utterance to your skill as an intent request, resulting in a poor user experience.
-
Estimated System Errors – The Estimated System Errors metric estimates the potential success of a conversation turn based on system signals from ASR/NLU and context. Use this metric to review utterances that frequently cause failed interactions and to view aggregate skill improvements over time so that you can modify your skill to further improve accuracy.
Note: The Estimated System Errors metric is available for high-traffic skills in en_US, en-GB, and de-DE.
Customer friction metrics
Use customer friction metrics to see how users interact with your skill. Alexa uses various user interaction patterns and other contextual signals to predict if a user experienced friction when interacting with your skill.
-
Customer friction summary – Use the Customer friction summary dashboard to see relative trends for your skill plus a list of frequent utterances that Alexa has identified as causing potential customer friction.
-
Barge-ins – Use the Barge-ins metric to see how often user interrupt an ongoing skill response. High barge-in rates could have causes, such as long skill responses, unexpected up-selling options, or not allowing enough time for a user to respond to a prompt.
-
Terminations – Use the Terminations metric to see the number of times users prematurely end a skill session. Premature terminations have causes, such as a user making a request that's out of scope for the skill, an incorrect slot recognition, or an incorrect use of Name Free Invocation.
-
Unhandled Requests – Use the Unhandled Requests metric to see the number of times your skill is unable to handle user requests. New user requests that might be relevant for your skill can contribute to these unhandled requests.
In this section
- View Skill Metrics — Learn how to view data and generate visualizations for the metrics available for your Alexa skills.
- Interpret and Troubleshoot Skill Performance Metrics — Learn how interpret the skill performance metrics reports.
- Download Message IDs for Troubleshooting — Learn about downloading message ID logs to debug skill failures and latency issues.
- Skill Metrics FAQ — Answers to Frequently Asked Questions (FAQ) about Alexa skill metrics and usage.
