In-App Voice Scrolling and Selection on Fire TV

Fire TV allows you to use your voice to perform scrolling and selection across all screens in the device. Voice commands for selection and scrolling can be uttered to the Fire TV Cube or to the paired Echo devices for other Fire TV devices.

• How It Works
• Voice-Enabled Scrolling and Selection
  • Scrolling
  • Selection
• Common Issues That Affect In-App Voice Commands
  • Issue #1: Focused Element Gets Reset
  • Issue #2: Screen Refreshes Make Navigation Inconsistent
  • Issue #3: App Updates that Limit D-pad Navigation
• Activity Lifecycle Handling
• Testing Voice-Enabled Scrolling and Selection
• FAQ

How It Works

The in-app voice scrolling and selection works by mapping D-pad navigation events to your voice commands. D-pad refers to the remote control's directional keypad (the "Navigation" ring around the Select button), which is used to scroll right, left, up, and down. Alexa converts these voice commands into D-pad navigation events that are sent to the app.

Voice-Enabled Scrolling and Selection

Voice-driven scrolling and selection capabilities are supported across all screens in your app. More details are as follows:

Scrolling

To scroll, users can say "scroll" combined with a direction (up, down, left, right) to move the focus through scrollable content in your app. For example, "Alexa, scroll left," or "Alexa, scroll down." With these commands, Fire TV moves the focus on screen in the desired direction. Fire TV performs the scrolling action by simulating the remote's RIGHT, LEFT, UP, DOWN (DPAD_RIGHT/LEFT/UP/DOWN) events multiple times.

The number of elements the focus moves (either horizontally or vertically) is set at an app-level and is based on the particular app's layout. The default will be set to 3 elements for horizontal movements and 2 elements for vertical movements. However, Amazon will manually adjust this default to provide the optimal experience for your app. For example, if the optimal experience in your app involves moving 4 elements instead of 3, Amazon will automatically choose this setting. To adjust the movements, you can submit a Contact Us Case to request changes.

In addition to scrolling, Fire TV also supports simple one-step navigation. One-step navigation moves the focus one element at a time through utterances with the keywords "move" and "go" combined with a direction (up, down, left, right). For example, "Alexa, move right," or "Alexa, go up." Fire TV performs the one-step navigation using the same D-pad events as before, but instead of simulating the remote's RIGHT, LEFT, UP, DOWN (DPAD_RIGHT/LEFT/UP/DOWN) events multiple times, Fire TV simulates it just once to move the focus by one element.

Selection

To select an item in focus or to play a highlighted title, users can say, "Alexa, select" or "Alexa, select this/that/it." This utterance performs the same action as clicking on or selecting the currently focused element. Fire TV performs the selection by simulating the remote's select (DPAD_CENTER) key-press event.

Common Issues That Affect In-App Voice Commands

To enable your app for in-app voice scrolling and selection, your app should handle overlays without resetting the focus or refreshing the screen. Most apps can handle this already by default, but if your app does not, it might be due to the following issues.

Issue #1: Focused Element Gets Reset

The focused element should not change or be reset when an overlay or dialog (related to a different process, such as an Alexa interaction) appears on top of your app's screen and then is dismissed.

For example, suppose your app is in the foreground and the focus is on an element in the third row, and the user invokes Alexa by saying, "Alexa, what time is it?" When Alexa finishes answering the user, the focus should continue to be where it was before the user invoked Alexa.

If the focus shifts to some other element after the Alexa interaction, the voice scrolling or selection will not work consistently, leading to a poor user experience. In such cases, if the user were to say, "Alexa, select this," Fire TV would (incorrectly) select the newly focused item instead of the item the user intended.

Issue #2: Screen Refreshes Make Navigation Inconsistent

Similar to focus reset issues, some apps experience issues because the screen refreshes or reloads after the Alexa voice interaction. If the screen refreshes or reloads while responding to a voice command, Fire TV will not have the context of the original focus when the user invoked Alexa — this will also result in a poor user experience.

For example, when a user says, "Alexa, scroll right," Fire TV sends a keypress event to the app to simulate the scrolling. During this time, if the app's screen is loading or partially loaded, the keypress might not land on the intended element, and the scrolling or selection will be inconsistent.

Issue #3: App Updates that Limit D-pad Navigation

Some updates to your app might not directly break scrolling and selection. However, if the updates reduce the ability for users to navigate your app through D-pad controls, in-app voice scrolling and selection will be more limited.

For example, over-reliance on the Menu button on the remote can limit the in-app voice experience. If the Menu button remains the only way to perform significant functions in your app (for example, to display a sidebar menu, or to show episodes within a season, or to see more information) rather than allowing these actions to be performed through D-pad direction keys, the voice scrolling and selection features will be limited in their use and relevance.

Additionally, if you map the D-pad navigation keys to other functions (such as to surface menu options when DPAD_UP is pressed), you will limit the user's ability to use D-pad functions for app navigation. As such, the voice scrolling and selection features in your app will be limited.

As you make updates to your app, continue to test the new version's compatibility with voice scrolling and navigation. Your app should continue to be support voice scrolling and selection with each update.

Activity Lifecycle Handling

You can avoid the Common Issues That Affect In-App Voice Commands by ensuring your app follows best practices for activity lifecycle handling. When users navigate through your app, the Activity instances in your app will transition through different states in their lifecycle. Within the lifecycle callback methods, you can define how an activity should behave based on the user's actions. For a more detailed explanation, see Android's Activity Lifecycle documentation.

The following are key lifecycle events for Activity UI handling and describe the proper handling and flow for apps:

• An onPause() callback will be called when the Activity is partially covered by other activities (such as when Fire TV shows a heads-up notification after a user long-presses HOME or invokes Alexa on a Fire TV device). After onPause(), technically the activity is still partially visible, but most often onPause() indicates that the user is leaving the activity, and the activity will soon enter the Stopped or Resumed state.

• After receiving onPause(), the activity will usually be followed by a call to onStop() (after the next activity has been resumed and displayed). However, in some cases, there will be a direct call back to onResume() without proceeding through the Stopped state.

• onResume() will be called just before the activity starts interacting with the user. At this point, the activity is at the top of the activity stack and captures all user input. onResume() must restore the original UI state without changes and update the UI only if there are changes between onPause() and onResume(). Ideally, do not change the focused element or refresh the screen during onResume() if you receive onResume() after onPause().

  Warning: If onPause() and onResume() aren't handled properly, the focused element on the screen can become lost. For example, when Alexa is invoked and your app's screen refreshes, the focus shouldn't automatically reset to the default element on the screen but rather stay focused on whatever element the user had selected prior to invoking Alexa.

• Fire TV calls onStop() when the activity is no longer visible to the user. This might happen when the activity is destroyed, a new activity starts, or an existing activity enters a resumed state and covers the stopped activity. In all of these cases, the stopped activity is no longer visible. The activity should release all its resources during onStop(). In the onStop() method, the app should release or adjust resources that are not needed while the app is not visible to the user.

• The next callback Fire TV calls is either onRestart() (if the activity is coming back to interact with the user), or onDestroy() (if this activity is completely terminating). Your app should refresh its UI or reset the focused element during onResume() only if the activity has completed onStop() and restarted again.

Testing Voice-Enabled Scrolling and Selection

1. During app development, you can test the selection and scrolling features in your app by sideloading your app onto the device by using adb. See Connect to Fire TV Through ADB for details.
2. After you finish your tests, you can submit your app to be published as usual.
3. For apps that fail the tests, you can address the failures and submit an updated version of your app. Use steps 1 and 2 to validate the new version of your app.

Apart from sideloading your app to test it, you can also test your app using Live App Testing. After you upload an app for Live App Testing, the accounts you specify for the testing coverage can be used to download and test the app.

Note that the voice scrolling and selection will be tested by Amazon before it is published in the Appstore. If the new version of your app is incompatible with the scrolling and selection experience, Amazon will reach out to you to resolve the issues.

To protect the user experience, if the previous version of your app was working with voice scrolling and selection, but the new version breaks this experience, your app will be rejected.

FAQ

What actions can customers perform using voice?

Our primary objective is to offer a basic voice experience across the larger selection of apps with zero developer effort. Customers can say things like "Alexa, Scroll Right" or "Alexa, Move Down" to navigate through the content, and "Alexa, Select" or "Alexa, Select This" to select the focused element. Supported phrases include the following:

• Navigation: Scroll [direction], Page [direction], Move [direction], Go [direction]. For the "[direction]", you can say up, down, right, or left.
• Selection: Select, Select this/that/it

Will this capability be made available on all apps?

Yes, this voice capability will be available for all apps as a system feature in Fire TV Cube. For the rest of Fire TV devices, while Amazon will make its best effort to make this experience available on all apps, to protect the customer experience, we add apps to the allow list on a case-by-case basis. As long as your app adheres to the criteria outlined in previous guidelines, your app will be enabled for this experience automatically.

Will there be any impact to my app because of this capability?

No, there will be no negative impact to your app's experience or performance.

If want more advanced voice capabilities, what should I do?

For advanced voice experiences, you should integrate the Alexa video skill into your Fire TV app. This offers the most comprehensive, richest voice experience for your app.

What devices are supported?

All Fire TV devices are supported except Fire TV Stick (1st Gen) and Fire TV Stick Basic Edition.

How do I pair my Alexa device with my Fire TV?

See Use Your Alexa Device to Control Your Fire TV.

How do I give feedback on this experience?

Use the Contact Us link to share any feedback you have.

---

Last updated: Nov 11, 2021