Your Alexa Dashboards Settings

Video Skill Testing Guide

Before you submit your video skill for certification, you must perform functional testing.

Following are our current recommended functional test cases. The functional tests provided are designed to be executed manually to evaluate your skill from an end-user perspective. You need to complete the test cases that correspond to capabilities supported by your skill. Each test case listed includes the prerequisites, steps to perform the test, example data, and expected results. Video skills should pass these tests before you submit the skill for certification.

Check this page for updates as the video skill capabilities evolve.

The tests are divided into categories.

In addition to the tests listed, skill responsiveness is important for a good user experience so make sure your skill complies with performance requirements listed in the [Alexa.RemoteVideoPlayer Interface][alexa-remotevideoplayer] topic.

Video Skills Functional Tests

These tests covers video-specific functionality such as discovering devices, playing content or searching content. They are organized by functional area such as device discover or search or play content

Device Discovery

These tests cover user account linking, device pairing and discovery.

Prerequisite: Skill is not enabled, no user account is linked, and no device is paired.

Test Expected Results
Enable the skill using the Alexa App, open the skill and account link the skill with an account in your system. Pair the skill to a device. The Alexa app indicates a device has been paired.

Prerequisite: Skill is enabled, a user account is linked and the skill is paired to a device.

Test Expected Results
Open the skill using the Alexa App and remove the device pairing. Disassociate the user account. The Alexa app indicates a device pairing has been removed.

Search Content

Test the search functionality using different entity types.

Prerequisite: Skill is enabled, a user account is linked, and the skill is paired to a device.

Test Expected Results
Ask, “Alexa, show me franchise movies” where franchise is the title of a movie franchise. For example, “The Muppets” Output screen shows list of search results for the requested franchise.
Ask, “Alexa, show me sport/league/team games” where sport/league/team is a value such “University of Washington Huskies” Output screen shows list of search results for the requested sport/league/team.
Ask, “Alexa, show me movies with actor” where actor specifies an actor’s name. Output screen shows list of search results for the requested actor.
Ask, “Alexa, show me genre movies”, where genre is a value such as science fiction or drama Output screen shows list of search results for the requested genre
Ask, “Alexa, find title” where title is a movie or TV show title. For example, “Manchester by the Sea” Shows results for the requested title.
Ask “Alexa show me genre movies with actor” where actor is the name of an actor. For example, “show me comedy movies with Tom Hanks”. Search results are shown for the requested genre and actor. For the example, movies could include “Big”, but not “The Green Mile”.
Ask, “Alexa find title movies with actor”, specifying a specific title and actor name. For example, “Find Total Recall with Arnold Schwarzenegger” Search results are shown for the requested actor and title. For the example, the result would be the original 1990 version, but not the 2012 remake.
Ask, “Alexa, find genre movie title/partial title”, where title/partial title can include the whole title or a word or two from a title. For example, “Find comedy movie Saving” Search results are shown for the requested genre and title. The example returns “Saving Mr. Banks”, but not “Saving Private Ryan”
Ask, “Alexa find genre movie title/partial title with actor” specifying at least a part of a content title. For example, “find comedy movie Saving with Tom Hanks” Search results are shown for the requested genre, partially-matched title and actor. The example returns “Saving Mr. Banks”, but not “Saving Private Ryan”.
Ask, “Alexa find character movie”, where character specifies the name of a character in the movie, such as “find Snow White movies”. Search results are shown for the requested character.
Ask, “Alexa find character movie title”, where character specifies the name of a character in a specific movie title, such as “find the James Bond movie Skyfall”. Search results are shown for the requested character and title.
Ask, “Alexa find title”, where title is a TV show. For example, “Alexa, find Transparent” Search results are shown for the requested TV show with the requested title.
Ask, “Alexa find title”, where title content that needs to be purchased. Search results include the detail page for the requested title that allows the user to purchase.
Ask, “Alexa, find title”, where title may include non-video content such as games Search results are shown for a standard search, which may include app and game results, and is not limited to video content.

Play Content

Test content playback using a variety of search and play entity types.

Prerequisite: Skill is enabled, a user account is linked, and the skill is paired to a device.

Test Expected Results
Ask, “Alexa, watch franchise movies”, where franchise is the title of a movie franchise such as “The Muppets”. A video from the requested franchise begins playing, if applicable.
Ask, “Alexa, watch sport/league/team games” where sport/league/team is a value such “University of Washington Huskies” A video from the requested sport/league/team begins playing, if applicable.
Ask, “Alexa, watch episode of show”. Where episode provides the season and episode. For example, “Alexa, watch season 2, episode 4 of Mozart in the Jungle” The requested episode begins playing, if applicable.
Ask, “Alexa, watch a genre”, where genre is a value such as “comedy” or “action” A video of the requested genre begins playing, if applicable.
Ask, “Alexa, watch an actor”, where actor specifies an actor’s name. A video with the requested actor begins playing, if applicable.
Ask, “Alexa, watch a genre movie with actor”, specifying a genre and an actor’s name. For example, “Play a comedy with Tom Hanks” A video in the requested genre with the requested actor begins playing, if applicable.
Ask, “Alexa, watch title with actor”, specifying a specific title and actor name. For example, “Find Total Recall with Arnold Schwarzenegger” Search results are shown for the requested actor and title. For the example, the result would be the original 1990 version, but not the 2012 remake.
Ask, “Alexa, watch genre title/partial title” specifying at least part of a content title. For example, “watch the comedy Saving” A video in the requested genre with a partially-matching title begins playing, if applicable. For the example, valid matches could include “Saving Christmas”, but not “Saving Private Ryan”.
Ask, “Alexa, watch the genre title/partial title with actor” to specifying at least part of a content title, the genre and an actor’s name. For example, “watch the comedy Saving with Tom Hanks”. A video of the requested genre, partially-matching title and actor begins playing, if applicable. For the example, valid matches could include “Saving Christmas”, but not “Saving Private Ryan”.
Ask “Alexa watch character movies” where character provides the name of a character that appears in the movie. For example, “watch Snow White movies”. A video with requested character begins playing, if applicable.
Ask “Alexa watch character movie title”, specifying a character name that appears in a specific movie title, such as “watch James Bond movie Sky Fall”. A video with requested character and title begins playing, if applicable.
Ask, “Alexa, watch title” where title is a movie title. For example, “watch Manchester by the Sea”. The requested title begins to play, if applicable.
Ask, “Alexa, watch title”, where title is a TV show. For example, “watch Transparent” A TV show with the requested title begins to play, if applicable.
Ask, “Alexa, watch title”, where title is content that needs to be purchased. Detail page for the requested title is displayed allowing user to purchase, if applicable.
Ask, “Alexa, watch title”, where title may include non-video content such as games. Search results are shown for a standard search, which may include app and game results, not just videos

Change Channel

Test changing channels using channel number, name and affiliateCallSign.

Prerequisite: Skill is enabled, a user account is linked, and the skill is paired to a device.

Test Expected Results
Ask, “Alexa, go to channel number”, where number is a value like 35 or 105. Video changes to requested channel.
Ask, “Alexa, go to channel name”, where name is a value like “ABC” or “Food Channel”. Video changes to requested channel.
Ask, “Alexa, go to channel name”, where name is a provider or location-dependent value like “KOMO” in the Seattle area. Video changes to requested channel.
From the Alexa App, disable the skill, and issue any Alexa video command such as “Change the channel to ABC” Error message indicating no devices are available to support the command.

Control Playback

Test controlling playback using various commands.

Prerequisite: Skill is enabled, a user account is linked, and the skill is paired to a device.

Test Expected Results
Ask, “Alexa, watch title” where title specifies a content title. For example, “watch Manchester by the Sea”. Video begins to play.
Ask, “Alexa, pause” Playback pauses.
Ask, “Alexa, resume” Playback resumes from where it paused.
Ask, “Alexa, stop” Playback stops.
Ask, “Alexa, play” Playback resumes.
Ask, “Alexa, fast forward” Playback position moves forward.
Ask, “Alexa, fast forward 2 minutes” Playback position moves forward 2 minutes.
Ask, “Alexa, rewind” Playback position moves back.
Ask, “Alexa, rewind 10 seconds”. Playback position moves back 10 seconds.
Ask, “Alexa, play title season 1, episode 1”, where title is a TV series with multiple episodes. TV series episode 1 begins to play.
Ask, “Alexa, next” TV series episode 2 begins to play.
Ask, “Alexa, previous” TV series episode 1 begins to play.

Additional Alexa Skills Tests

In addition, you should perform some of the same tests that are applicable to custom skills. Following are links to functional tests for custom skills that are also relevant to video skills.

Policy Testing

Test your skill to ensure its content meets Amazon’s policy guidelines. See Alexa Skills Kit Policy Testing.

Security Testing

Test the Lambda portion of your skill to ensure it meets Amazon’s security requirements. See Alexa Skills Kit Security Testing.

Example Phrases

Make sure the example phrases you provide for your skill work well and provide a good user experience. See Review and Test Example Phrases in the functional testing guide for custom skills.

Skill Description and Detail Page

Test the description that teaches users how to use your skill. See Skill Description in the functional testing guide for custom skills.

Account Linking

Ensure a user can successfully account-link your skill. See Account Linking in the functional testing guide for custom skills.

Duplicates of Samples or Templates

Make sure your skill is not an exact duplicate of any of the Amazon-provided samples or templates. See Duplicates of Samples or Templates in the functional testing guide for custom skills.