View Skill Metrics

Use the Analytics page in the Alexa Skills Kit developer console to see metrics for a skill. This dashboard can give you useful information about your skill and how customers are using it. You can also access skill metrics with the Metrics API.

View the metrics for a skill

In the developer console, use one of the following methods to access the Analytics page for a skill.

To access the Analytics page from the developer console landing page

  1. If you haven’t already, start the developer console.
  2. From the developer console landing page, click the Skills tab.
  3. In the list of your skills, locate the skill row you want metrics for.
  4. Under the ACTIONS column, click Analytics.

To access the Analytics page from the developer console skill page

  • In a skill page that you’re viewing or editing, from the top navigation menu, click Analytics.

The metrics available on the Analytics page depend on the type of your skill.

On the Analytics page in the developer console, you can perform the following tasks.

  • Use the language drop-down list in the upper left to switch between language models, or choose All to see combined metrics for all of your skill's language models.
  • Use the Jump to analytics for drop-down list in the upper right to go directly to the Analytics page for a different skill in your account.
  • Use the Skill Stage drop-down list to switch between development and live versions of the skill.
  • Use the Time Interval controls buttons to change the time range.
  • Use the Aggregation Period drop-down list to aggregate on different time periods (i.e. show average data every X period for latency).
Analytics page
Analytics page

Skill metrics

The Summary section of the Analytics page displays an overview of the metrics available for the skill. For each metric, you can view infographics like charts, grids, or other visualizations. You can export these infographics as PNG or JPEG images, or download the raw data in CSV format.

You can access most metrics for skills in both the development and live stages, except as noted in the following tables. You only have access to some metrics for certain types of skills (custom, smart home, and so on), as noted in the following tables. Metrics aren’t available for meeting skills. All metrics data excludes invocations from the Test page in the developer console.

Metric Description Skill types
Actions Unique customers per action, total actions, and total utterances per action. Smart Home
Customers Total number of unique customers who have accessed the skill. Custom

Flash Briefing

Smart Home
Intents Unique customers per intent, total utterances per intent, total intents, and failed intents. Custom
Interaction Path Paths users take when interacting with your skill. For example, you can see the percentage of users who invoked a particular intent after the LaunchRequest. Custom
Plays Total number of times your skill's content was played. Flash Briefing
Retention
(live skills only)
Usage of your skill over time by groups of customers (cohorts). You can see the number or percentage of customers who returned to your skill over a 12-week period. Custom
Sessions Total sessions, successful session types (sessions that didn't end due to an error), average sessions per customer; and a breakdown of successful, failed, and no-response sessions as a percentage of total sessions. Custom
Utterances Metrics for utterances depend on the type of skill.

For custom skills:
Total utterances, average utterances, and utterance responses.
For smart home skills:
Total utterances and average utterances per customer.
Custom

Smart Home

The Skill Activation section of the Analytics page displays metrics related to skill activation, as described in the following table. Skill activation metrics are available from January 1, 2018. They are only available for live skills.

Many of the following metrics relate to account linking. To get the most from these metrics, you should understand account linking initiations and completions. For more information, see Understand Account Linking.

Metric Description Skill types
Account Linking Completion Rate Percentage of successful account links. The percentage is the total number of account linking completions divided by the total number of initiations. Custom (with account linking)

Smart Home
Account Linking Initiations Per User Average number of account linking initiations per user. Custom (with account linking)
Total Account Linking Events Total number of account linking initiations and completions. Custom (with account linking)

Smart Home
Total Account Linking Users Total number of unique users who:
  • Enabled the skill.
  • Initiated account linking.
  • Completed account linking.
Custom (with account linking)
User Enablement's Definition of this metric depends on the type of skill.

For custom skills without account linking, all flash briefing skills, and music skills without account linking:
Total number of unique users who enabled the skill.
For smart home, video, and baby activity skills:
Total number of unique users who enabled the skill and completed account linking.
Note: Some users might enable your skill, but not launch it. Only users who launch a skill are counted in the number of unique customers. This can help explain a difference between the number of user enablements and the number of unique customers.
Baby Activity

Custom (without account linking)

Flash Briefing

Music (without account linking)

Smart Home

Video

Skill performance metrics

Use skill performance metrics to view how well your skill is functioning.

Metric Description Skill types
Endpoint Latency Metrics that tells you that 90% of your customer utterances resulted in X milliseconds of endpoint latency or less, during your chosen time period. Custom
Endpoint Response Number of utterances where your skill endpoint responded successfully. Custom

See how quickly your skill responds to users

On the Endpoint Latency page, you can see how well your skill responds to the utterances of users. Skills that have low endpoint latency respond slowly and cause friction for users.

To view Endpoint Latency analytics

  1. Open the Alexa Developer Console and sign in.
  2. In the Alexa Skills list, click Analytics next to the skill you want to view analytics for.
  3. On the Summary page, in the side bar, under custom, click Endpoint Latency.
  4. Use the Skill Stage drop-down list to switch between development and live versions of the skill.
  5. Use the Time Interval buttons to change the time range.
  6. Use the Aggregation Period drop-down list to change the interval of time at which the data is refreshed (every 5 minutes, 15 minutes, hourly, daily, weekly, or monthly).

Understand the search results

This metric shows you that 90 percent of your customer utterances resulted in X milliseconds of endpoint latency or less, during the time period you selected.

endpoint latency screen
Endpoint Latency page

Tips to improve data

There are a few different ways you can improve your skills confidence.

  • Configure your AWS Lambda function by increasing the amount of memory for faster function execution. Lambda allocates CPU proportionally to the memory you allocate to your function. If the memory you provision is higher than 1.8 gigabytes (GB), Lambda assigns multiple CPUs.

  • Create region specific Lambda functions if your skill supports multiple locales to avoid cross-regional calls. For more information about selecting the optimal region for your AWS Lambda function, see Select the optimal region for your AWS Lambda function.

  • To address a cold start by select the specific modules or SDKs that you need, rather than full SDKs to reduce your Lambda package file. If you have multiple skills that share common code such as SDKs or other libraries, you can move them to AWS Lambda Layers and exclude them from your deployment package.

  • Host Amazon S3 bucket in same AWS region as your Lambda function if you're hosting files for your skill. You can use cross-region replication to maintain copies in different regions.

  • If your skill stores data, choose a capacity mode for your Amazon DynamoDB. If you get consistent traffic, you should use Provisioned Mode. If your traffic is less-predictable, use On-Demand Mode.

  • Use the Best Practices for Amazon RDS You can avoid running out of database connections or slow database calls if you use Amazon RDS to store data.

See whether your skill responds to users

On the Endpoint Response page, you can see how many successful and failed responses your skill has over time. Skills that have no response confuse users and lower user satisfaction with your skill. If users aren’t responding to your skill, your endpoint-response metrics reflect a low score.

To view Endpoint Response analytics

  1. Open the Alexa Developer Console and sign in.
  2. In the Alexa Skills list, click Analytics next to the skill you want to view analytics for.
  3. On the Summary page, in the side bar, under custom, click Endpoint Response.
  4. Use the Skill Stage drop-down list to switch between development and live versions of the skill.
  5. Use the Time Interval (UTC) buttons to choose a time range.
  6. Use the Aggregation Period drop-down list to change the interval of time at which the data is refreshed (every 5 minutes, 15 minutes, hourly, daily, weekly, or monthly).

Understand the search results

The Endpoint Response results show the number of successful and failed skill endpoint responses.

endpoint response screen
Endpoint Response page

Tips to improve data

There are a few different ways you can improve your skills confidence.

  • Monitor endpoint health by using Amazon CloudWatch.

  • Set an alarm on CloudWatch to learn about failures immediately.

  • Improve your Lambda function code to prevent failures. For example, make sure that your responses don't exceed your plan size limits.

Operational metrics

In the developer console, the Analytics page shows operational metrics for a smart home skill. Metrics include latency and success rate for the skill that are related to device discovery, capability directives, query state requests, and change report events. The operational metrics are from as recent as five minutes prior.

On the Analytics page, in the Operational Metrics section, the Discovery, Capability Directives, and Report State pages display the latency and success rate metrics for device discovery, device control, and query state requests, respectively. For each of these metrics, you can view infographics like charts, grids, or other visualizations. You can export these infographics as .png or .jpg file formats, or download the raw data in .csv file format.

On each of the operational metrics pages, use the following controls to adjust your view of the metrics:

  • Use the Jump to analytics for list to go directly to the Analytics page for a different skill in your account.
  • Use the Skill Stage drop-down list to switch between development and live versions of the skill.
  • Use the Region drop-down list to switch between NA, FE, and EU regions.
  • Use the Time Interval buttons to change the time range.
  • Use the Aggregation Period drop-down list to change the interval of time at which the data is refreshed (every 5 minutes, 15 minutes, hourly, daily, weekly, or monthly).

Discovery page

Alexa sends a discovery request to your skill to discover your customer’s smart devices. The Discovery page shows the latency and success rate graphs for all discovery requests to your skill. For more information about discovery, see Alexa Discovery interface.

Capability Directives page

When a customer asks Alexa to control their smart device, Alexa sends a directive request to your skill to trigger a capability, for example turning on a light. The Capability Directives page shows Latency and Success Rate tabs. Both tabs show multiple graphs, one for the aggregate of all capability directive requests, and one per capability for all capabilities supported by your skill. The capability graphs show the aggregate across all directives and the breakdown per directive for all directives under that capability. For more information about capability directives, see Understand the Smart Home Skill API.

Report State page

Alexa sends a ReportState directive to your skill to request the state of an endpoint. The Report State page shows the latency and success rate graphs for all the report state requests to your skill. For more information about report state, see State Reporting for a Smart Home Skill.

Change Report page

When the state of an endpoint changes for any reason, for example turning on a light, you report that change to Alexa in a ChangeReport event. Alexa can then provide that status change to the customer. The Change Report page shows the accuracy rate of these change reports as perceived by Alexa when a ReportState directive is sent. For more information about change reports, see Report state in an Alexa.ChangeReport.

Operation metric definitions

Latency
Latency is the time until Alexa receives a response from your skill, after Alexa sends a request to your skill. Latency is measured using percentile statistics such as P99, P90, and P50. Use the operational statistics menu (+ icon) in the latency graphs to choose different percentile statistics or the average. You can view multiple statistics on the same graph.
Success rate
Success rate is the total number of successful responses Alexa receives from your skill divided by the total number of requests Alexa sends to your skill.
Request volume
Both latency and success rate graphs show request volume on the second Y-Axis. You can use the + icon on the graphs to toggle the request volume on or off.
Accuracy rate
Accuracy rate is the percentage of times that the response to a report state directive matches a previously sent change report.

In-skill purchasing metrics

The metrics for in-skill purchasing differ depending on the type of in-skill product:

One-time purchases (entitlements) and consumables

The following metrics are available for one-time purchases and consumables:

  • Total Purchases – The total number of offers accepted for all in-skill products of this type (one-time purchases or consumables) within a skill. For consumables, this metric includes repeat purchases by the same customer.
  • Purchases – The number of offers accepted for each in-skill product. For consumables, this metric includes repeat purchases.
  • Upsell to Offer Conversion – The percentage of upsell impressions for which customers agreed to hear the offer. This metric is only available when you use the upsell directive.
  • Offer to Purchase Conversion – The percentage of offer impressions that resulted in a purchase.
  • Upsell to Purchase Conversion – The percentage of upsell impressions that resulted in a purchase. This metric is only available when you use the upsell directive.
  • Offer Impressions – The total number of times that customers heard the purchase prompt from Amazon, describing your product and its price, and decided whether or not to buy. This metric doesn't include messages provided within the upsell directive. For consumables, this metric includes both the first-time purchase and repeat purchases of the same item.

Subscriptions

The following metrics are available for subscriptions:

  • Active Subscriptions –The total number of active subscriptions for your skill. This metric includes paid subscriptions that are set to expire at a future date.
  • Trial to Paid Conversion Rate – The percentage of trials that became paid subscriptions.
  • Churn – The percentage of active paid subscriptions that expired or canceled.
  • Subscriptions Started – The total number of new subscriptions.
  • Offer Conversion Rate – The percentage of offer impressions that resulted in the customer signing up for a new trial or paid subscription.
  • Upsell to Offer Conversion – The percentage of upsell impressions for which customers agreed to hear the offer. This metric is only available when you use the upsell directive.
  • Offer Impressions – The total number of times customers heard the purchase prompt from Amazon, describing your product and its price, and decided whether or not to buy. This metric includes both free-trial and paid offers. This metric doesn't include messages provided within the upsell directive.
  • Upsell to Purchase Conversion – The percentage of upsell impressions that resulted in a purchase. This metric is only available when you use the upsell directive.
  • Subscriptions Ended – The total number of trial subscriptions that were canceled or paid subscriptions that expired. If a customer cancels a paid subscription, the subscription remains active until it expires at the end of the customer's billing period.

Skill Metrics FAQ

Q: Does a skill need to be live before I can see its metrics?
No. Many metrics, including operational metrics, are available for skills in both the development stage and live stage.
Q: How long does it take for the data on the Analytics page to update?
For the retention metric, the data is refreshed weekly. The data doesn't update until a week's worth of data is available.

For the enablements metric, the data is delayed by as much as one day. The data is updated as it becomes available.

For operational metrics, the dashboards show data collected from as recently as five minutes prior.

For other metrics, the data can be delayed by as long as two hours.

Q: Can I export the metrics data?
Yes, you can export infographics (charts, grids, and other visualizations) as PNG or JPEG images. You can also download the raw data in CSV format.
Q: What is a unique customer?
A unique customer is a customer who launched a skill using a unique Amazon account. Unique customers are determined by user accounts, not devices. If a customer launches your skill on multiple devices that are all registered to the same user account, this is counted as one unique customer.
Q: What is a user enablement?
User enablements are the unique user accounts that enabled your skill, either by voice, the Alexa app, or the Alexa Skills Store website. For some types of skills, users must perform account linking to enable the skill. For more information, see Skill metrics.
Q: Why is the number of unique customers and user enablements different?
Some users might enable your skill, but not launch it. Only users who launch a skill are counted in the number of unique customers.
Q: Why does the number of unique customers change when I change how the data is aggregated?
The number of unique customers changes because the meaning of unique changes depending on how you aggregate the data. For example, consider the following scenario in which several customers use the same skill in the same day:
  • Customer A uses the skill at 01:00.
  • Customers B and C both use the skill at 11:00.
  • Customer B uses the skill again at 15:00.

When you aggregate the data by hour, you see one unique customer at 1:00, two unique customers at 11:00, and one unique customer at 15:00.

When you aggregate the data by day, you see a total of three unique customers.

Adding together the number of unique customers per hour doesn't necessarily give you the number of unique customers per day, because it doesn't account for repeat customers. The metrics that you see on the Analytics page account for repeat customers, which can help explain why you see differences when you aggregate the data differently.

The Summary section of the Analytics page shows the total number of unique customers for the entire date range that you choose.

Q: Can I see the operational metrics for my skill for all the regions where I published my skill?
If you have the same Skill ID across the regions, you can see operational metrics across regions Use the Region drop-down list to choose a region that you want to see metrics for.
Q: Can I see the total metrics for all of a skill's locales or languages in one place?
Yes. In the language drop-down list in the upper left, choose All to see combined metrics for all of your skill's language models.
Q: How can I see my customer retention rate?
The cohort chart in the Retention section of the Analytics page shows the number of returning users and the percentage increase or decrease of retention increase or decrease over time.
Q: How far back in time can I go to view the operational metrics?
Operational metrics are available from June 27th, 2019. You can view data for up to 90 days prior, but for no earlier than June 27th, 2019.
Q: Why am I not seeing the metrics for my smart home v2 skill?
You can view operational metrics for smart home v3 skills only. Upgrade your skill to v3 to access operational metrics. For more information on how to upgrade, see Smart Home Skill v2-to-v3 Migration Guide.
Q: How can I provide feedback or report issues?
Email Amazon alexa-smarthome-ops-dashboard-feedback@amazon.com. Alternatively, click the Send Feedback link at the bottom of each of the operational metrics pages. This link opens your email client with the URL already populated. In your feedback, enter the description for the issue you're reporting.

The developer console: