This section provides an overview of WASM Player, a Tizen WebAssembly extension allowing for low-level elementary media stream playback. Essential information & guidelines on WASM Player APIs for WebAssembly are included.
Tizen WASM Player API Documentation
Application Multitasking
Media Specifications
Samsung Emscripten SDK Download
W3C HTML Media Element Specifications
Tizen WASM Player Sample (Normal Latency)
Tizen WASM Player Sample (Low Latency) - Moonlight port to Tizen WASM Player
Tizen WASM Player is a Samsung Web extension API which grants a WebAssembly application low level access to Tizen platform's media player. The WASM Player operates on Elementary Media Stream Packet level, accepting encoded packets (audio/video frames) that are decoded and rendered by the multimedia pipeline. Depending on operation mode, WASM Player is fit to be used as a video on-demand player (normal latency mode, suitable for adaptive streaming scenarios), as a streaming player (low latency mode) or as a game streaming player (ultra low latency mode).
Tizen WASM Player is a low-level API that accepts Encoded Audio/Video Frames (or Elementary Media Packets) to render media.
When used in this documentation, Audio/Video Frames are defined as raw, encoded audio or video data that can be rendered after decoding in the WASM Player. The complementary term (often used interchangeably) is Elementary Media Packet, which is defined as an Audio/Video Frame with added metadata.
Tizen WASM Player is labelled as a low level media API, because it works at the Elementary Media Packet level: Tizen WASM Player is responsible for decoding and rendering media content, while acquiring media content and splitting it into Audio/Video Frames (Elementary Media Packets) is entirely dependent on the application itself.
Rendering of Audio/Video Frames is configurable in Tizen WASM Player:
The configurable rendering allows for great flexibility: the application has full control over downloading and demuxing of data, and implementation of either low latency or adaptive streaming protocol. When the WASM Player is used, a major part of the multimedia pipeline can be implemented as a WebAssembly module and is therefore platform-independent, allowing for a wide variety of multimedia applications.
This document describes both latency modes in detail and covers all API components required to use the WASM Player in a WebAssembly application.
Flexible configuration options allow for Tizen WASM Player usage in a wide range of applications. However, 2 typical usage scenarios can be highlighted:
These typical scenarios are described in detail below.
Note The WASM Player's latency mode is orthogonal to the scenarios listed above. The latency mode is decoupled from an application's media data processing method (container- or streaming-based playback) and display mode (the WASM Player renders content, the application renders content using OpenGL). While the low latency modes are usually used with streaming protocols, they can be also used with container-based applications (and vice versa).
The WASM Player's latency mode is orthogonal to the scenarios listed above. The latency mode is decoupled from an application's media data processing method (container- or streaming-based playback) and display mode (the WASM Player renders content, the application renders content using OpenGL). While the low latency modes are usually used with streaming protocols, they can be also used with container-based applications (and vice versa).
In this usage scenario:
Figure 1. High-Level Architecture of a Media Container-Based Playback Application
Figure 2. High-Level Architecture of a Streaming Protocol-Based Multimedia Playback Application
Tizen WASM Player is a more low-level alternative to Media Stream Extensions (MSE):
Refer to the following table for more key differences between Tizen WASM Player and MSE.
Tizen WASM player has three distinct latency modes: normal latency, low latency, and ultra low latency.
ElementaryMediaStreamSource
The following table shows how the 3 latency modes differ from each other with their handling of specific features.
Note FPS rate does not apply in low latency mode, as the rendering rate is dependent entirely on packet sending frequency (therefore, FPS is freely and entirely controlled by the application). The seek operation does not apply in low latency modes, as it implies working with a live, non-seekable source. DRM support is available only in conjunction with the Media Element rendering mode.
FPS rate does not apply in low latency mode, as the rendering rate is dependent entirely on packet sending frequency (therefore, FPS is freely and entirely controlled by the application).
The seek operation does not apply in low latency modes, as it implies working with a live, non-seekable source.
DRM support is available only in conjunction with the Media Element rendering mode.
Tizen WASM Player has 2 rendering modes: Media Element and Video texture.
The following table shows a comparison between the 2 rendering modes.
Note DRM support is available only in conjunction with the normal latency mode.
DRM support is available only in conjunction with the normal latency mode.
The following sections contain a detailed look into the WASM Player architecture and its components.
The WASM Player is meant to be used in Tizen Web widget applications that employ WebAssembly modules. Tizen Web widget applications work in a HTML5 environment: the WASM Player provides an ElementaryMediaStreamSource component that is designed to be used as a source object of HTMLMediaElement.
Figure 3. WASM Player Architecture Overview
In order to use Media Player with the WASM Player, the application must use the provided WASM Player classes (which act as a source of data for playback) and the HTMLMediaElement class (to control Media Player). The following table defines the class responsibilities and functions:
HTMLMediaElement
ElementaryMediaStreamSource manages a set of ElementaryMediaTrack objects. Together, their purpose is essentially feeding the platform with media data required to play content; EMSS classes are acting as a source of media data.
ElementaryMediaTrack
Each instance of this class represents a single elementary media track (either audio or video). The track object allows sending Elementary Stream Packet data to the platform for playback.
HTMLMediaElement is used alongside WASM Player classes as a playback control element, providing operations such as play, pause, and seek. That is, Media Element controls Media Player.
HTMLMediaElement is associated with either the <audio> or <video> tags in HTML, allowing for the positioning of the playback area.
<audio>
<video>
The relevant part of the HTMLMediaElement API is wrapped by a C++ API available alongside the WebAssembly EMSS C++ API.
HTMLMediaElement is not covered in this document in detail. For more information, see HTML Media Element Specification.
ElementaryMediaStreamSource and HTMLMediaElement serve different purposes in Media Player. As shown above, Elementary Media Stream Source objects act as the means to deliver media data to HTMLMediaElement for playback. Controlling playback (such as issuing Play, Pause, and Seeking commands) on Media Player is done through the HTMLMediaElement API. Elementary Media Stream Source allows you to deliver media data in form of Elementary Media Packets.
When operating Media Player, the application must rely on HTMLMediaElement events and methods. For example, ElementaryMediaStreamSource signals when Media Player requires media data. However, the moment Media Player starts reading media data is not always the exact moment that playback can be started. Readiness for playback is signalized by HTMLMediaElement through the HTMLMediaElementListener::OnCanPlay() event.
HTMLMediaElementListener::OnCanPlay()
The following figure and table describe the ElementaryMediaStreamSource states and the transitions between them.
Figure 4. ElementaryMediaStreamSource State Diagram
kDetached
ElementaryMediaStreamSource is not assigned to HTMLMediaElement.
This is the initial state of an ElementaryMediaStreamSource object. It also enters this state after disconnecting from HTMLMediaElement.
kClosed
Tracks are not configured and the player is not initialized. Media cannot be played in this state.
ElementaryMediaTrack objects can be added to and removed from ElementaryMediaStreamSource. Track layout can be changed only in this state.
This state is entered after ElementaryMediaStreamSource is attached to HTMLMediaElement, when unrecoverable playback error occurs, or on request by the application.
kOpen
Media Player is fully initialized and Elementary Media Stream Source is ready to accept Elementary Packet Data from the application through ElementaryMediaTrack objects.
When the application finishes configuring Tracks, it can request entering this state. The kOpen state is entered when possible. Some operations (like Seek) trigger a temporary transition to kOpenPending state instead.
kOpenPending
Opening ElementaryMediaStreamSource was requested by the application, but the current state of the multimedia pipeline prevents immediately entering the kOpen state. The kOpen state will be entered as soon as possible.
This is a temporary state, entered on several occasions:
kEnded
Stream has ended but multimedia pipeline remains initialized.
This state is entered when the application marks all active ElementaryMediaTrack objects as ended. The kEnded state reverts to the kOpen state when the multimedia pipeline resumes playback (such as due to Seek).
Important ElementaryMediaStreamSource's and HTMLMediaElement's states must not be confused. ElementaryMediaStreamSource represents the state of the source of data, while HTMLMediaElement represents the state of the multimedia player. As such, ElementaryMediaStreamSource signals the application whether the multimedia pipeline needs data and is able to accept Elementary Media Packets. This is not in sync with the multimedia player's state. For example, the kEnded state can be associated with a still-playing Media Element. This occurs in normal latency mode when the application signals the end of all tracks, but the multimedia pipeline has buffered Elementary Media Packets remaining to play.
ElementaryMediaStreamSource's and HTMLMediaElement's states must not be confused. ElementaryMediaStreamSource represents the state of the source of data, while HTMLMediaElement represents the state of the multimedia player. As such, ElementaryMediaStreamSource signals the application whether the multimedia pipeline needs data and is able to accept Elementary Media Packets. This is not in sync with the multimedia player's state.
For example, the kEnded state can be associated with a still-playing Media Element. This occurs in normal latency mode when the application signals the end of all tracks, but the multimedia pipeline has buffered Elementary Media Packets remaining to play.
Depending on the latency mode, the ElementaryMediaStreamSource's preconditions for entering the kOpen state are slightly different.
Note There are numerous reasons for EMSS to switch between states. The WASM Player Usage Guide provides detailed information on when such a change happens.
There are numerous reasons for EMSS to switch between states. The WASM Player Usage Guide provides detailed information on when such a change happens.
The state of a ElementaryMediaTrack object reflects the state of an associated ElementaryMediaStreamSource, however in a simplified manner. A track can be either open or closed, represented by a single boolean attribute.
The track can accept Elementary Media Packets.
The track opens just before the source enters the kOpen state.
The track cannot accept Elementary Media Packets.
The track closes just after the source leaves the kOpen state.
Note When a track closes, an event is generated carrying a reason why the closure occurred.
When a track closes, an event is generated carrying a reason why the closure occurred.
The C++ HTMLMediaElement class supplied with Elementary Media Stream Source is a wrapper for a HTML5's HTMLMediaElement. For information on HTMLMediaElement states, see HTML Media Element Specification.
Applications that target devices work in a multitasking environment and must respond to multitasking appropriately. For a multimedia application, this often means having to suspend media playback when the application becomes invisible (such as goes into background when the user launches another application) and resuming playback when the application becomes visible (such as goes into foreground when the user resumes the application).
The WASM Player tracks an application's multitasking state and generates appropriate events for the application to react to. Specifics of the event sequence depends on a selected EMSS mode of operation.
Note For more details on supporting multitasking, see the Using Advanced Player Features. For more details on multitasking on Samsung devices, see Application Multitasking.
For information on supported codecs and configurations, see Samsung Media Specifications.