This is a guest post by Ling Shi, principal software development lead at Hulu, originally published on the Hulu Tech Blog. 

Smart home device use is expanding rapidly — almost a third of all households in the US used a smart speaker to make a purchase in Q4 2017. With more consumers leveraging these devices to make their daily lives easier, we knew bringing Hulu commands to smart home devices would be impactful and useful for our viewers. That’s why this past fall, we announced Hulu’s first foray into voice control using Amazon Alexa with Fire TV.

In this post, we will explain how we used the Alexa Skills Kit and Video Skill API to deliver a voice-centric experience to Alexa.

This past fall, Hulu viewers watching on Amazon Fire TV were able to enjoy our voice control system on Alexa-enabled devices such as the Echo and the Dot or their Fire TV voice remote.

Below are some examples of commands viewers can leverage to control Hulu via voice:

“Alexa, watch ESPN on Hulu.”

“Alexa, tune to Bravo.”

“Alexa, watch The Handmaid’s Tale.”

“Alexa, watch This is Us.”

“Alexa, pause.”

“Alexa, play next”

“Alexa, rewind”

The technical details of our integration

We developed the Hulu voice command system by leveraging the Video Skill API in the Alexa Skills Kit. The Hulu system consists of the following parts:

- Hulu Metadata Service — This service exports Hulu catalog information (metadata and show availability windows) to Amazon. By delivering up Hulu’s content catalog to Amazon, Alexa can understand and serve up requested content.

• Alexa Service — This consists of a set of core services that bind the Hulu app, Fire TV, and Alexa-enabled device together to convert the viewer’s voice command to Alexa directives
• AWS Lambda — This is a microservice that runs on AWS to convert Alexa directives to Hulu-specific actions
• ADM Service — This is an Amazon push notification service that exists to deliver Hulu directives to the Hulu Fire TV App
• Hulu App — This is the core Hulu app and user interface running on the Fire TV. It retrieves Hulu directives from ADM and executes them within the app.
• Alexa Video Library — This is a library provided by Amazon to simplify device pairing for the Hulu App
• Amazon Echo — This is the device inside the viewer’s home that receives voice commands and replies accordingly
   

The workflow is structured as follows:

1. Amazon calls the Hulu Metadata Service regularly to retrieve the Hulu catalog.
2. The viewer launches the Hulu App by simply saying “Alexa, open Hulu.”
3. When the Hulu App is launched, the Alexa Video Library registers it with the Alexa service and pairs it to the Hulu video skill.
4. The viewer invokes the Hulu video skill by saying “Alexa, watch [title] on Hulu”, or “Alexa, pause” etc.
5. Alexa service handles the voice input commands and converts them to Alexa directives, finally pushes out to AWS Lambda.
6. For a video on demand playback directive (e.g. “Alexa, watch The Handmaid’s Tale”), the Alexa service will send Hulu a unique show ID to the Hulu video skill.
7. For a live program playback directive (e.g. “Alexa,tune to HBO”), the Alexa service will send a “go to network X” directive to the Hulu video skill.
8. Playback control directives (pause, seek, etc…) are forwarded directly to Hulu video skill.
9. Hulu video skills converts the Alexa directives to Hulu directives and sends them to the Hulu App via the ADM push notification mechanism.
10. The Hulu App gets a directive from ADM and runs that command for the users.
11. For all launching playback directives, the Hulu App will check the user’s entitlement, content availability, geo location, and other restrictions.
12. If the directive is to launch a live program, the Hulu App will handle local affiliate channel selection — map a network, like Fox Sports, to a local channel “Fox Sports Arizona” based on user’s geolocation.

This design was just the beginning, and we’re currently researching additional improvements to enrich our viewers’ experience since the launch of this feature last fall.

Engage your customers with voice

Want to create a more robust voice-forward experience for your customers? Here are three steps to enable voice control in your apps:

1. First, check to see if you have already implemented the Media Sessions API. If you have, all you need to do is add the permission to the manifest file and implement voice commands for transport controls like play, pause, resume, rewind, fast-forward, and skip content. If you haven't, click here to get started with Media Session.
2. Integrate your media catalog with Amazon Fire TV so your content can be included in universal browse and search results. And, regardless of where users find your content in the Fire TV interface, they can play it directly without needing to open your app first. If your titles are indexed on IMDB or Amazon Video, you are eligible for catalog integration.
3. Once your app has gone through catalog integration, you can use the Video Skill API to allow customers to launch content, search within your app, change the channel, and use transport controls with their voice.