Documentation

Console

Settings

Sign out

Notifications

Alexa

Amazon Appstore

AWS

Documentation

Support

My Cases

Console

Vega Developer Docs

Home

Get Started

Design and Develop

Publish

Reference

Support

W3C Media API in React Native for Vega

Open Beta Documentation Amazon offers this technical documentation as part of a pre-release, open beta. The features described might change as Amazon receives feedback and iterates on the features. For the most current feature information, see the Release Notes.

Today, media streaming application developers face the challenge of developing and maintaining more than one media playback stack on different device families based on different operating systems like Android, iOS, and web browsers. Maintaining more than one media player incurs additional engineering effort, increases maintenance cost, and can yield a diverse customer experience and fragmentation of features across platforms.

Through Vega, we found an opportunity to unify the playback API and stack based on the open web standards and an open source multimedia framework like Gstreamer. We are building a W3C (World Wide Web Consortium) Media React Native JavaScript component that brings the W3C Media APIs to Vega SDK and Vega devices. This offers a solution to converge on a common media playback stack across Web Browsers, Vega, and Fire OS devices.

What are W3C Media APIs?

Web app developers are quite familiar with the media APIs HTMLMediaElement, Media Source Extension (MSE) and Encrypted Media Extensions (EME) defined by WHATWG and W3C open standard groups.

HTMLMediaElement is the main player API that supports playback of any media URL and provides high level playback control APIs like pause, play, and seek. It also supports querying the state of the playback and media tracks related information of the content. We use the term URL Mode Playback to refer to this mode of playback.

Media Source Extension (MSE) is an extension to HTMLMediaElement API that enables JavaScript (JS) app developers to playback adaptive streaming contents like HLS and MPEG DASH. Through this extension, the JS app downloads and parses the HLS/DASH manifests, downloads the required media segment based on the calculated bandwidth and injects them into the playback pipeline through this API. This provides app developers greater control over the playback experience best suited for their content. We use the term “MSE Mode Playback” to refer to this mode of playback.

Encrypted Media Extension (EME) is an extension to HTMLMediaElement API that enables JS app developers to acquire and provide DRM license to the playback pipeline to decrypt the DRM protected secure content, and playback through a Secure Video Path in the system.

These APIs are traditionally available in desktop and embedded web browsers like Chromium, Microsoft Edge, Safari, Opera and others. Web application developers code in HTML/CSS/JavaScript to playback DRM and clear media in web browsers using these W3C Media APIs.

Note: Vega SDK provides the W3C Media API as React Native components.

What are the benefits of W3C Media API?

The MSE API allow app developers to build advanced playback experiences like gapless episodic playback (for web series and playlists) and gapless advertisement insertion (pre-roll, mid-roll and post-roll) during playback of main content. This is achieved by exposing functions that allow app developers to manage (add/remove/update) the media samples that are queued to be decoded and rendered in the media pipeline, all in a familiar JavaScript syntax.

What Streaming Formats does W3C Media API support?

Through MSE Mode Playback, apps can play any adaptive streaming format such as MPEG DASH, HLS, and SmoothStreaming as long as the content container format is compatible to the list specified in MSE byte stream registry.
Through URL Mode Playback, apps can play non-adaptive streaming of file formats such as MP4, MP3, OGG, FLV, and MKV.

What audio/video codecs are supported by W3C Media API?

W3C Media API supports popular audio/video popular codecs used in streaming media use cases like Dolby Audio (AC3, eAC3, AC4, EC3_JOC), AAC ( LC, AAC+ and eAAC+), MP3, Opus , FLAC, and Vorbis for audio, and H.264, H.265, VP8, VP9, MPEG4, and AV1 for video.

What DRM schemas does the W3C Media API support?

W3C Media API supports DRM schemas like Widevine (L1) and PlayReady (SL3000).

What Closed Captions/Subtitle formats are supported?

W3C Media APIs will support in-band and out-of-band closed captions and subtitle in CEA 608, CEA 708, TTML, WebVTT, and SRT formats. Apps will be able to select the closed captions and subtitles based on language.

What does the native stack powering the W3C API look like?

The following section describes the layer cake starting from bottom to top.

Platform core media components: These set of platform/OS native APIs allows access to underlying h/w processing blocks to decrypt, decode and render Audio/Video frames and to display closed caption on top of the video surface.
1. Audio/Video Decoder: The platform API that provides access to H/W accelerated or S/W audio/video decoder supported by the SoC and BSP of the platform to decode a frame.
2. DRM: The platform API for handling DRM license and keys, and works with decoder component to decrypt the frames in Secure Trust Zone (TEE).
3. Video Surface: The platform API to render video on screen.
4. Audio: The platform API to playback PCM and Dolby audio frames.
5. Captions/Subtitles Component: A UI component that renders captions and subtitles.
Open Source GStreamer Multimedia Framework: This open source multimedia framework based on Glib C is the foundation of media playback framework in Vega and Fire OS. We use the core Gstreamer library, some open source plugins and custom plugins built by Amazon to decrypt, decode and render audio/video and captions using the platform core media components.
Media Player: This layer is a native (C/C++) component that uses Gstreamer APIs under the hood to provide ExoPlayer like high level simple playback API to play injected A/V Samples, or a content URL.
W3C Media JS API as React Native Components. This layer consists of W3C spec compliant HTMLMediaElement, MediaSourceExtension(MSE) and EncryptedMediaExtension(EME) JavaScript APIs. Native W3C Media is the native implementation of this specification that uses MediaPlayer layer to demux and play the audio/video fragments/segments of adaptive streaming protocols like MPEG DASH and HLS.

Note: React Native for Vega app developers will use MSE/EME JavaScript players for playback.

Which MSE/EME JavaScript player is supported?

The Vega W3C Media Player supports the Shaka Player. For information on using the Shaka Player, see Playing Adaptive Streaming Content (HLS/DASH) with Shaka Player.

As we enhance the Media Player we are investigating other MSE/EME JavaScript players in the market that are used by app developers in their Web Apps. The following is a list of some of popular ones that support adaptive streaming playback using MSE APIs. We appreciate feedback on which players you currently use.

Open source MSE/EME JavaScript Players:

Shaka Player

Commercial MSE/EME JavaScript Players

We are using Shaka Player (open source) and MSE Test Suite provided by W3C standard committee to test and validate our stack.

Recommendation for React Native for Vega App integration

We recommend React Native for Vega apps to use React Native W3C Media JavaScript API through any JavaScript MSE/EME player. We will enable app developers to “Bring Your Own Player” and integrate with W3C Media APIs of Vega SDK.

If this recommendation does not satisfy your app requirements please consult your Solutions Architect or your Point of contact in Amazon to arrange a discussion with our engineering team for alternate integration options.