Amazon Developer

as

Settings
Sign out
Notifications
Alexa
Amazon Appstore
Ring
AWS
Documentation
Support
Contact Us
My Cases
Develop
Test
Publish
Monetize
Engage users
Device specifications
Resources
Skip to main content
Performance testing is the process of app testing in areas such as compatibility, reliability, speed, response time, stability, and resource usage on Amazon Fire TV devices running Fire OS. You can use this testing to identify and address your app’s performance bottlenecks. Performance testing involves gathering and evaluating key performance indicator (KPI) metrics. To gather KPI metrics, you run a specific set of steps on an Amazon device and then find or calculate the metrics using device resources, such as logs. Make sure to run performance tests before submitting your app to Amazon Appstore. By regularly tracking these KPIs, you can verify that your app delivers a smooth, responsive experience. Understanding latency and memory:
  • Latency measures how quickly your app launches and becomes usable. Users expect near-instant app startup and minimal delay before interacting with content.
  • Memory measures how efficiently your app uses device RAM. Excessive memory consumption leads to system kills, poor multitasking performance, and degraded user experience.

Prerequisites

Before you measure app KPIs on Fire OS:
  1. Install the following software packages on your development computer:
  2. Complete the following setup:
    • Set up a path for JAVA_HOME and ANDROID_HOME folders.
    • Enable developer mode on the device and enable USB Debugging. For instructions, see Enable Debugging on Amazon Fire TV.
    • Capture the serial number of the attached device using:
  3. Read the following sections:

App KPI metrics and guidelines

The following table lists KPI metrics to measure for Fire TV apps on Fire OS, and the recommended guideline for each. These metrics measure launch performance and memory efficiency.
KPIUnitScenarioDescriptionGuideline
Time to first frame (TTFF)Seconds (s)Cold start app launchMeasures time from app launch to first frame render. Calculated from Atrace: AMS.startActivityAsUser to eglSwapBuffersWithDamageKHR.< 2.0 s
TTFFSeconds (s)Warm start app launchMeasures time an app takes to display its first frame when moving from background to foreground.< 1.0 s
Time to full display (TTFD)Seconds (s)Cold start app launchMeasures time from launch until app is ready for user interaction. Requires reportFullyDrawn().< 10.0 s
TTFDSeconds (s)Warm start app launchMeasures time for app to become fully interactive after moving from background to foreground.< 2.0 s
Foreground memoryMBApp in foreground (4K video)Measures app proportional set size (PSS) total when active and playing 4K video content.< 400 MB
Foreground memoryMBApp in foreground (1080P or lower)Measures app PSS total when active with 1080P or lower resolution content.< 300 MB
Background memoryMBApp in backgroundMeasures app PSS total after playing content for 10 min and moving to background.< 100 MB
eMMC write (FG)MB/hrApp in foregroundAverage rate of data written to eMMC storage while app is active.< 50 MB/hr
eMMC write (BG)MB/hrApp in backgroundAverage rate of data written to eMMC storage while app is in background.< 2 MB/hr

Launch scenarios

Performance testing measures two types of app launch scenarios to evaluate TTFF and TTFD:
  • Cold start – When a user launches an app after they force stop the app process or reboot the device. The system must load all resources and dependencies into memory.
  • Warm start – When a user moves an app from background (inactive state) to foreground (active state) with some resources and dependencies already in memory.

Fully drawn marker

A fully drawn marker signals when your app becomes interactive for users. The marker:
  • Indicates when your app completes loading its essential components.
  • Marks when users can start interacting with your app.
  • Helps measure TTFD performance.
To implement, call reportFullyDrawn() from your Activity at these key points:
  • Cold start – After loading initial data and rendering the main screen (sign-in page or home page).
  • Warm start – When your app becomes responsive after foregrounding.
The following example code shows how to add a fully drawn marker.

Measure KPIs with ADB and Atrace

Use Android Debug Bridge (ADB) and Atrace to measure each KPI on a Fire TV device running Fire OS.

Test strategy

During testing, you launch the app and force stop it several times using the app launcher intent or the Monkey tool. Between each iteration, capture Atrace logs, perform navigation actions, capture timer values, and capture memory usage before force stopping. Amazon recommends running the following minimum number of iterations.
Performance test categoryMinimum iterations
Latency – time to first frame (TTFF)50
Ready-to-use – time to full display (TTFD)10
Memory5

Devices to test

For test coverage across Fire OS versions, test on the following reference devices.
Fire OS versionReference device
Fire OS 6Fire TV Stick 4K (2018)
Fire OS 7Fire TV Stick with voice remote (2020)

Cold start KPIs

Cold start KPIs measure app performance when launching the app after force stopping the app process or rebooting the device.

TTFF - cold start

  1. Force stop the app:
  2. Start Atrace:
  3. Launch the app:
  4. Wait 30 seconds.
  5. Pull the Atrace logs:
  6. Calculate time difference:
    • Start marker: AMS.startActivityAsUser
    • End marker: eglSwapBuffersWithDamageKHR
  7. Force stop the app.
  8. Repeat steps 1–7 for 50 iterations.

TTFD - cold start

Follow the same steps as TTFF - cold start, but use these markers:
  • Start marker: AMS.startActivityAsUser
  • End marker: Activity.reportFullyDrawn (matching app PID)
Repeat for 10 iterations.

Warm start KPIs

Warm start KPIs measure app performance when bringing the app from background to foreground, with some resources already in memory.

TTFF - warm start

  1. Launch the app and press Home to send it to background.
  2. Start Atrace.
  3. Launch the app (bringing it to foreground) using the Monkey tool.
  4. Wait 30 seconds.
  5. Pull the Atrace logs.
  6. Calculate time difference:
    • Start marker: AMS.startActivityAsUser
    • End marker: eglSwapBuffersWithDamageKHR
  7. Press Home button to send the app to background.
  8. Repeat steps 2–7 for 50 iterations.

TTFD - warm start

Follow the same steps as TTFF - warm start, but use Activity.reportFullyDrawn as the end marker. Repeat for 10 iterations.

Foreground memory KPIs

Foreground memory KPIs capture your app’s memory consumption while it’s actively running and playing content.
  1. Download, install, and sign in to the app (if applicable).
  2. Force stop the app.
  3. Launch the app using the Monkey tool.
  4. Play video content for 10 minutes.
  5. Wait 120 seconds.
  6. Capture foreground memory:
  7. Force stop the app.
  8. Repeat steps 2–7 for 5 iterations.
The PSS Total from dumpsys meminfo is the primary metric.

Background memory KPIs

Background memory KPIs capture your app’s memory consumption after it has been moved to the background. When RAM is low, the system is more likely to kill apps with high background memory usage.
  1. Download, install, and sign in to the app (if applicable).
  2. Launch the app and play video content for 10 minutes.
  3. Press Home button to send the app to background.
  4. Wait 60 seconds.
  5. Capture background memory:
  6. Repeat steps 2–5 for 5 iterations.

Summary of guidelines

The following table summarizes the recommended guidelines and measurement iterations for each KPI.
KPIGuidelineMeasurement
TTFF cold start< 2.0 s50 iterations
TTFF warm start< 1.0 s50 iterations
TTFD cold start< 10.0 s10 iterations
TTFD warm start< 2.0 s10 iterations
Foreground memory (4K)< 400 MB5 iterations
Foreground memory (1080P)< 300 MB5 iterations
Background memory< 100 MB5 iterations
eMMC write (foreground)< 50 MB/hrContinuous
eMMC write (background)< 2 MB/hrContinuous

KPI health assessment

Use the following indicators to evaluate whether your app’s KPI results meet the recommended guidelines.
IndicatorMeaning
PassMeets the recommended guideline
WarningWithin 10% of the guideline
FailExceeds the guideline by more than 10%

Tools

The following tools can help you measure and monitor app KPIs on Fire TV devices:
  • Android Debug Bridge (ADB) – Command-line tool for communicating with Fire TV devices.
  • Atrace/Systrace – System tracing utility for capturing performance markers.
  • Monkey tool – App launcher for automated testing.
  • Appium – Test automation framework.
  • Maestro – Script-based UI automation for memory/playback scenarios.
  • Android Studio Profiler – Real-time CPU, memory, and network profiling.
  • App Health Insights Dashboard – Developer Console dashboard for tracking live app metrics.
Last modified on July 23, 2026