WASM Player Usage Guide

This section provides a detailed Tizen WASM Player API usage guide.

---

Related Info

• Tizen WASM Player API Documentation
• Samsung SmartTV Media Specification
• Samsung SmartTV Widget Multitasking Guide
• WebAssembly for Tizen TV Download
• W3C HTML Media Element Specification
• Tizen WASM Player Sample (Normal Latency)
• Moonlight port to Tizen WASM (Low Latency)

---

Interacting with Tizen WASM Player

Tizen WASM Player provides an ElementaryMediaStreamSource component that acts as a source object for HTMLMediaElement and provides a low level multimedia API targeting WebAssembly that enables App to work with Media Player on elementary media packets level.

A figure below shows general idea of interaction between App and Media Player when using WASM Player:

App manages Media Player (controlled by HTMLMediaElement) and sends Elementary Media Packet data via ElementaryMediaTrack objects.

The way App interacts with WASM Player is partially dependent on the selected mode of operation, however most of the operations are common to all modes. Paragraphs below describe available operations in detail, highlighting areas where the selected mode influences behavior of the source.

Initializing Media Player to work with WASM Player

Setting up HTMLMediaElement with ElementaryMediaStreamSource as a source object

In WASM Player, the ElementaryMediaStreamSource class acts as a source object for HTMLMediaElement. In order to set up ElementaryMediaStreamSource and HTMLMediaElement to work together:

1. Create HTMLMediaElement object:

   auto html_media_element = std::make_unique<samsung::html::HTMLMediaElement>(element_id);
   

   Note :

   HTMLMediaElement class connects to either a <video> or an <audio> element in a HTML document hosting WebAssembly module. The HTML element is chosen by the element_id specified during construction.

2. Create an object implementing HTMLMediaElementListener interface and associate it with the media element. This step is optional, however events generated by the media element are essential for creating a functional media player.

   1. First, define a custom implementation of HTMLMediaElementListener:

      class MyMediaElementListener : public samsung::html::HTMLMediaElementListener {
        // override virtual methods that App uses
      };
      

   2. Instantiate the custom listener and associate it with the HTMLMediaElement object:

      auto my_media_element_listener = std::make_unique<MyMediaElementListener>();
      html_media_element->SetListener(my_media_element_listener.get());
      

      Note :

      Please note HTMLMediaElement::SetListener() method does not take ownership of the listener. Make sure that the listener outlives the media element object!

3. Create ElementaryMediaStreamSource object:

   using LatencyMode = samsung::wasm::ElementaryMediaStreamSource::LatencyMode;
   using RenderingMode = samsung::wasm::ElementaryMediaStreamSource::RenderingMode;
   
   auto elementary_media_stream_source = std::make_unique<samsung::wasm::ElementaryMediaStreamSource>(LatencyMode::kNormal, RenderingMode::kMediaElement);
   

   Note :

   Choose LatencyMode::kNormal to set up WASM Player to work in Normal Latency mode, LatencyMode::kLow for Low Latency mode and LatencyMode::kUltraLow for Ultra Low Latency mode.
   
   

Choose RenderingMode::kMediaElement to set up WASM Player to work in Media Element rendering mode or RenderingMode::kVideoTexture for Video Texture rendering mode.

4. Create an object implementing ElementaryMediaStreamSourceListener interface and associate it with the source. This step is optional, however events generated by the source are essential for creating a functional media player.

   1. First, define a custom implementation of ElementaryMediaStreamSourceListener:

      class MySourceListener : public samsung::wasm::ElementaryMediaStreamSourceListener {
        // override virtual methods that App uses
      };
      

   2. Instantiate the custom listener and associate it with the ElementaryMediaStreamSource object:

      auto my_source_listener = std::make_unique<MySourceListener>();
      elementary_media_stream_source->SetListener(my_source_listener.get());
      

      Note :

      Please note ElementaryMediaStreamSource::SetListener() method does not take ownership of the listener. Make sure it outlives the source object!

5. Attach ElementaryMediaStreamSource to HTMLMediaElement:

   html_media_element->SetSrc(elementary_media_stream_source.get());
   

   Note :

   When created, ElementaryMediaStreamSource is set to the ReadyState::kDetached state, preventing any operations on the source. Attaching it to HTMLMediaElement will change source's state to the ReadyState::kClosed state, allowing configuration of media tracks.

Adding ElementaryMediaTracks to ElementaryMediaStreamSource

Once ElementaryMediaStreamSource is created and attached to HTMLMediaElement, App should configure tracks that takes part in the media playback. Depending on its needs App can add up to one video track and up to one audio track to the source, as described below.

Preparing video track config

To prepare an ElementaryVideoStreamTrackConfig structure, fill it with an initial configuration of a video track:

samsung::wasm::ElementaryVideoStreamTrackConfig conf;
conf.mimeType = "video/mp4; codecs=\"hev1.1.6.L93.B0\""; // h265
conf.extradata = { /* codec extradata */ };
conf.width = 1920;
conf.height = 1080;
conf.framerateNum = 60;
conf.framerateDen = 1;
conf.decodingMode = samsung::wasm::DecodingMode::kHardware;

Preparing audio track config

To prepare an ElementaryAudioStreamTrackConfig structure, fill it with a configuration of an audio track:

samsung::wasm::ElementaryAudioStreamTrackConfig conf;
conf.mimeType = "audio/mp4; codecs=\"mp4a.40.2\""; // AAC
conf.extradata = { /* codec extradata */ };
conf.sampleFormat = samsung::wasm::SampleFormat::kPlanarF32;
conf.channelLayout = samsung::wasm::ChannelLayout::kStereo;
conf.samplesPerSecond = 48000;
conf.decodingMode = samsung::wasm::DecodingMode::kHardware;

Adding a track to the source

As soon as a track's configuration is ready, the track can be added to the source. This procedure is the same for both audio and video tracks.

1. Add a track to the source:

   auto track = elementary_media_stream_source->AddTrack(conf);
   

2. Create an object implementing ElementaryMediaTrackListener interface and associate it with the track. This step is optional, however events generated by the track are essential for implementing certain media player functionalities.

   1. First, define a custom implementation of ElementaryMediaTrackListener:

      class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
        // override virtual methods that App uses
      };
      

   2. Instantiate the custom listener and associate it with the ElementaryMediaTrack object:

      auto my_track_listener = std::make_unique<MyTrackListener>();
      track->SetListener(my_track_listener.get());
      

      Note :

      Please note that ElementaryMediaTrack::SetListener() method does not take ownership of the listener. Make sure that the listener outlives the track object!

Setting media duration

Duration of media should be set on the ElementaryMediaStreamSource instance during initialization:

auto content_duration = samsung::wasm::Seconds{42};
elementary_media_stream_source->SetDuration(content_duration);

Once set, the duration is not fixed. It can also be changed in runtime if playback scenario requires it.

Completing Media Player initialization

When App configures track layout, ElementaryMediaStreamSource can be initialized by putting it in ReadyState::kOpen and thus enabling playback.

1. Request ElementaryMediaStreamSource to open:

   using OperationResult = samsung::wasm::OperationResult;
   
   elementary_media_stream_source->Open([](OperationResult result) {
     if (result != OperationResult::kSuccess) {
       // handle error
     }
   });
   

   Running Open operation on the source immediately puts it in theReadyState::kOpenPending state. That means the source will enter the ReadyState::kOpen state as soon as possible, however this is not neccesarily the moment when Open operation finishes! For details see Starting playback after initialization paragraph below.

Starting Playback

Normal Latency mode

Normal Latency playback in WASM Player is meant for VOD scenarios. In this mode, Platform's Media Player buffers Elementary Media Packet data sent by App and ensures it plays smoothly.

Starting playback

Once ElementaryMediaStreamSource enters ReadyState::kOpen (thus opening ElementaryMediaTrack), App can start buffering media data. In Normal Latency mode, a number of packets needs to be buffered and processed by Platform before the playback can be started. App should observe HTMLMediaElement events (HTMLMediaElementListener::OnCanPlay()) to detect this moment or HTMLMediaElement autoplay feature.

To start playback in Normal Latency mode:

1. Configure Media Player as described in Initializing Media Player to work with WASM Player paragraph above.

2. Source enters ReadyState::kOpen and each track emits ElementaryMediaTrackListener::OnTrackOpen() event to notify that it can accept elementary media data.

   class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
     // ...
     void OnTrackOpen() override {
       // Start App's packet sending mechanism for the associated track.
     }
     // ...
   };
   

   Note :

   Alternatively, App can wait on ElementaryMediaStreamSourceListener::OnSourceOpen() event emitted by the source itself. OnTrackOpen events are generated before OnSourceOpen, so it's safe to trigger packet sending mechanism in this event handler too.

3. Build up internal Media Player data buffer by sending a number of packets. This is basically the same mechanism as described below in Sending Elementary Media Packets paragraph below.

4. When HTMLMediaElement is ready to start playback, it emits HTMLMediaElementListener::OnCanPlay() event. Handle the event and call HTMLMediaElement::Play().

   class MyMediaElementListener : public samsung::html::HTMLMediaElementListener {
     // ...
     void OnCanPlay() override {
       html_media_element->Play([](OperationResult result) {
         if (result != OperationResult::kSuccess) {
           // handle error
         }
       });
     }
     // ...
   };
   

See Sending Elementary Media Packets paragraph below for information regarding sending media data to Media Player.

Low Latency modes

Low Latency playback in WASM Player is meant for live streaming scenarios. In this mode Platform renders Elementary Media Packets immediately after App sends them. Therefore it is up to App to maintain pipeline clock and stream synchronization.

Starting playback

In Low Latency modes no buffering is required to start media playback. Media Player is ready to start playback at any moment, however WASM Player can accept packets only when HTMLMediaElement is in playing state.

When ElementaryMediaStreamSource::Open() is called, ElementaryMediaStreamSource is immediately set in ReadyState::kOpenPending state. Once App calls HTMLMediaElement::Play(), the source enters ReadyState::kOpen state and ElementaryMediaTracks open. From that point on Elementary Media Packets will be accepted and rendered immediately.

To start playback in Low Latency modes:

1. Configure Media Player as described in Initializing Media Player to work with WASM Player paragraph above.

2. When App is ready to start playback, call HTMLMediaElement::Play():

   html_media_element->Play([](OperationResult result) {
     if (result != OperationResult::kSuccess) {
       // handle error
     }
   });
   

3. Source enters ReadyState::kOpen and each track emits ElementaryMediaTrackListener::OnTrackOpen() event to notify it can accept elementary media data.

   class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
     // ...
     void OnTrackOpen() override {
       // Start App's packet sending mechanism for the associated track.
     }
     // ...
   };
   

   Note :

   Alternatively, App can wait on ElementaryMediaStreamSourceListener::OnSourceOpen() event emitted by the source itself. OnTrackOpen events are generated before OnSourceOpen, so it's safe to trigger packet sending mechanism in this event handler too.

Sending Elementary Media Packets

Elementary media stream playback with WASM Player relies on App sending Elementary Media Packets to the Platform. This is done via ElementaryMediaTrack objects, each representing a single media track.

Pipeline, clock and Mode

• In Normal Latency mode Platform buffers elementary media data, guaranteeing a smooth playback as long as enough data is buffered.
  Note :

  If Platform's internal media data buffer level is too low to maintain a steady playback, HTMLMediaElement pauses the playback. Media element resumes the playback automatically when enough data is buffered.

• In Low Latency modes Platform renders elementary media data immediately, allowing App to implement live streaming scenarios.
  Note :

  Stream running time represented by HTMLMediaElement::GetCurrentTime() advances according to timestamps of Elementary Media Packets sent by App.

AppendPacket() operation

App uses an ElementaryMediaTrack::AppendPacket() operation to send media data to Media Player. Remarks:

• Elementary Media Packets should be appended to each individual track in decoding order (i.e. sorted by decoding timestamp),
• appends do not need to maintain decoding order between tracks, however it is recommended they do (this does not apply in Low Latency mode, where rendering speed is entirely dependent on App and no internal buffering occurs),
• B-Frames should not be used in Low Latency mode.

A mechanism that sends Elementary Media Packets to WASM Player should follow the scheme below:

1. Prepare ElementaryMediaPacket structure for the next append. The packet to append is a packet with the lowest structure dts among packets that were not sent to WASM Player yet.

   samsung::wasm::ElementaryMediaPacket packet;
   packet.pts = /* presentation timestamp in samsung::wasm::Seconds */;
   packet.dts = /* decoding timestamp in samsung::wasm::Seconds */;
   packet.duration = /* packet duration in samsung::wasm::Seconds */;
   packet.is_key_frame = /* bool value indicating if packet is a keyframe */;
   packet.data = /* pointer to packet data payload */;
   packet.data_size = /* size of the data payload in bytes */;
   packet.session_id = /* current session id */
   

   Note :

   Usually all audio packets are keyframes.

Multithreaded Apps should use session_id. For details, please see Session id paragraph in Threading section.

App can set additional parameters if the packet in question is a video frame:

// Set either both or none; can be 0 unless resolution changed.
packet.width = /* 0 or width */;
packet.height = /* 0 or height */;

// Set either both or none; can be 0 unless resolution changed.
packet.framerate_num = /* 0 or value */;
packet.framerate_den = /* 0 or value */;

Video-only parameters come in pairs: .width and .height, .framerate_num and .framerate_den. Each pair is optional and doesn't have to be specified unless track parameter they describe changes from this packet onwards. See Runtime configuration change paragraph for more details on reconfiguring ElementaryMediaTrack.

2. Send the packet by appending it to the corresponding ElementaryMediaTrack:

   auto result = track->AppendPacket(packet);
   
   if (!result) {
     // handle error
   }
   

Managing amount of buffered data

In order to maintain a smooth playback, App should maintain a steady flow of Elementary Media Packets that will prevent underrun of Platform's media data buffer. As a rule of thumb, it's good to buffer a couple of seconds worth of packets for each track, with the following limitations:

• for clear (non-encrypted) content playback at most 64 MiB of data can be buffered,
• for DRM-protected content playback at most 10 MiB of data can be buffered,
• for audio track at most 768 KiB of data can be buffered,
• App shouldn't buffer more than 3 seconds of content in advance.

In order to keep track of amount of data that should be buffered, App should use ElementaryMediaStreamSourceListener::OnPlaybackPositionChanged() event. This event is emitted periodically to update current playback time and can be used to calculate amount of data to buffer:

// Controls how many packets should be buffered ahead of a current playback
// position.
constexpr samsung::wasm::Seconds kBufferAhead = Seconds{3.};

class MySourceListener : public samsung::wasm::ElementaryMediaStreamSourceListener {
  using Seconds = samsung::wasm::Seconds;
  // ...
  void OnPlaybackPositionChanged(Seconds new_time) override {
    // Buffer packets up to timestamp (new_time + kBufferAhead).
  }
  // ...
};

:::

ElementaryMediaStreamSourceListener::OnPlaybackPositionChanged() event is similar to HTMLMediaElementListener::OnTimeUpdate() event, however the one associated with HTMLMediaElement should not be used to control a data source.

Events associated with each of those two components are associated with a state of the respective component. States of ElementaryMediaStreamSource (data source) and HTMLMediaPlayer (Media Player control element) are not in sync. An example would be Seek operation: data source can be done seeking when Media Player is still in the middle of the Seek. Therefore HTMLMediaElementListener::OnTimeUpdate() is not reliable in the context of a data source.

:::

Threading considerations

Usually App will be sending Elementary Media Packets on a worker thread (see Threading paragraph below). However, ElementaryMediaStreamSourceListener events are delivered on the main thread (via main JS message loop). If triggering packet buffering involves thread synchronization, it's advised to limit amount of inter-thread communication that involves locking the main thread to a minimum.

Most of the events delivered from both ElementaryMediaStreamSource and HTMLMediaElement are scarce. The only exception are time updates, which are delivered on a regular basis, potentially with a high frequency. App should fine-tune how often such events are handled. Usually updating buffering mechanism once or twice per second is sufficient.

When to stop sending packets

App can send packets as long as ElementaryMediaTracks remain open and ElementaryMediaStreamSource is in ReadyState::kOpen state:

class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
   // ...
   void OnTrackClosed(ElementaryMediaTrack::CloseReason close_reason) override {
      // Stop App's packet sending mechanism for the associated track.
   }
   // ...
};

Platform can close tracks due to both App actions and the user interaction with the device. App should properly handle close event and expect tracks to close at any moment. The most notable occurrence that causes tracks to close is the user switching active application (see Multitasking section). Another common example would be the user performing Seek operation (see Seek section).

End of stream

When playback is coming to an end and all remaining Elementary Media Packets were buffered, App should mark each track as ended. It can be done by using one of 2 possible methods: AppendEndOfTrack() or AppendEndOfTrackAsync(). AppendEndOfTrack() is working synchronously - it returns a result when operation is finished (track was closed or an error occurred):

auto result = track->AppendEndOfTrack(session_id);

if (!result) {
   // handle error
}

AppendEndOfTrackAsync() method returns a result synchronously when end of track was appended and track starts closing (or an error occurred before that happened). All errors that occur when track is closing will be signalled via ElementaryMediaTrackListener::OnAppendError() event. For details, please see Handling async append errors section.

Multithreaded Apps should use session_id. For details, please see Session id paragraph in Threading section.

Seek

Seek operation allows the user to jump to a new playback position. It is performed either via HTMLMediaElement's interface when controls are enabled or programmatically by calling HTMLMediaElement::SetCurrentTime().

Seek algorithm

Whenever a new current time value is set:

1. If ElementaryMediaStreamSource is ReadyState::kOpen:

   1. ElementaryMediaStreamSource state is changed to ReadyState::kOpenPending,
      • ElementaryMediaStreamSourceListener::OnSourceOpenPending() is fired,
   2. ElementaryMediaTracks are closed,
      • ElementaryMediaTrackListener::OnTrackClosed() with CloseReason::kTrackSeeking is fired for each track,
   3. ElementaryMediaStreamSourceListner::OnSessionIdChanged() is fired for each track.

2. ElementaryMediaStreamSourceListner::OnSeek() is fired wit a new current time.

3. If ElementaryMediaStreamSource was ReadyState::kOpen prior Seek:

   1. ElementaryMediaTracks are open,
      • ElementaryMediaTrackListener::OnTrackOpen() is fired for each track,
   2. ElementaryMediaStreamSource state is changed to ReadyState::kOpen,
      • ElementaryMediaStreamSourceListener::OnSourceOpen() is fired,

Loop

If HTMLMediaElement has the loop property set to true, WASM Player automatically generates a Seek to 0s when playback reaches end of stream.

Multitasking

Multitasking happens when App is either hidden or shown on the device's screen (e.g. when the user switches active application). App should always properly handle Multitasking.

Suspend algorithm

When App is suspended (i.e. visibilitychange reports document.hidden change to true):

1. If ElementaryMediaStreamSource is ReadyState::kOpen:
   1. ElementaryMediaStreamSource state is changed to ReadyState::kOpenPending,
      • ElementaryMediaStreamSourceListener::OnSourceOpenPending() is fired,
   2. ElementaryMediaTracks are closed,
      • ElementaryMediaTrackListener::OnTrackClosed() with CloseReason::kSourceSuspended is fired for each track.
2. If HTMLMediaElement::IsPaused() attribute is false:
   1. HTMLMediaElement is paused.
      
      HTMLMediaElement::GetCurrentTime() is stored internally by WASM Player.
      
      (this step is omitted in Low Latency modes)

Resume algorithm

When App is resumed (i.e. visibilitychange reports document.hidden change to false):

1. If ElementaryMediaStreamSource was ReadyState::kOpen prior to suspend, an automatic Seek is performed to the stored HTMLMediaElement::GetCurrentTime().
   
   As a result, ElementaryMediaStreamSource is set to ReadyState::kOpen and each ElementaryMediaTrack is opened.
   
   (In Low Latency mode Seek is omitted; instead Source and Tracks are opened immediately)
2. If HTMLMediaElement::IsPaused() was false prior suspend:
   1. HTMLMediaElement is unpaused.

Supporting DRMs

WASM Player supports DRM-protected content playback. Supported formats are equivalent to those specified in EME section in Samsung Smart TV Media Specifications.

In order to set up encrypted media playback App should do as follow:

1. Prepare a DRMConfig structure:

   samsung::wasm::DRMConfig drm_config;
   drm_config.cdm = /* a samsung::wasm::ContentDecryptionModule value */;
   drm_config.encryption_mode = /* a samsung::wasm::EncryptionMode value */;
   drm_config.license_server /* an URL to a license server */;
   drm_config.init_data = /* DRM System-specific initialization data. */;
   
   // If video track is encrypted:
   drm_config.audio_mime_type = /* a mime of encrypted audio or empty string if track is clear */;
   drm_config.audio_robustness = /* a samsung::wasm::Robustness parameter for audio track */;
   
   // If video track is encrypted:
   drm_config.video_mime_type = /* a mime of encrypted video or empty string if track is clear */;
   drm_config.video_robustness = /* a samsung::wasm::Robustness parameter for video track */;
   

2. Create a MediaKey using the config created in the previous step and assign it to a corresponding ElementaryMediaTracks:

   using OperationResult = samsung::wasm::OperationResult;
   using MediaKey = samsung::wasm::MediaKey;
   
   // ...
   
   // Created MediaKey must be stored by App:
   MediaKey decryption_key;
   
   // ...
   
   auto result = samsung::wasm::MediaKey::SetupEncryption(drm_config, [](OperationResult result, MediaKey media_key) {
     if (result != OperationResult::kSuccess) {
       // handle error
       return;
     }
     // Store the key. Make sure it outlives tracks it is assigned to below!
     decryption_key = std::move(media_key);
   
     // if audio is encrypted...
     audio_track->SetMediaKey(&decryption_key)
   
     // if video is encrypted...
     video_track->SetMediaKey(&decryption_key)
   })
   

   Note :

   When creating a MediaKey instance, implementation automatically contacts the license server specified in the drm_config.license_server parameter.

3. When multimedia content is DRM-protected, App should follow normal usage guidelines for Elementary Media Stream source except for one difference: encrypted Elementary Media Packets are sent to ElementaryMediaTracks with an AppendEncryptedPacketAsync method:

   samsung::wasm::EncryptedElementaryMediaPacket encrypted_packet;
   // ...
   // Set all clear packet information as usual.
   // ...
   encrypted_packet.subsamples = /* subsample information: a vector of samsung::wasm::EncryptedSubsampleDescription */;
   encrypted_packet.key_id = /* id of a key that should be used to decrypt this packet */;
   encrypted_packet.initialization_vector = /* IV required to decrypt this packet */;
   encrypted_packet.encryption_mode = /* samsung::wasm::EncryptionMode for this packet */;
   
   auto result = track->AppendEncryptedPacketAsync(encrypted_packet);
   
   if (!result) {
     // handle error
   }
   

Handling async append errors

If AppendPacketAsync(), AppendEncryptedPacketAsync() or AppendEndOfTrackAsync() are used, App should detect async append errors using ElementaryMediaTrackListener. The OnAppendError() event should be handled in a custom implementation of ElementaryMediaTrackListener:

class MyMediaElementListener : public samsung::wasm::ElementaryMediaTrackListener {
  using OperationResult = samsung::wasm::OperationResult;
  // ...
  // override virtual OnAppendError method
  void OnAppendError(OperationResult result) override {
    // handle append error
  }
  // ...
};

Runtime configuration change

A playing stream can be reconfigured in the runtime, to an extent. Changes to the following parameters are supported in runtime:

Track type	Parameter
Video	Resolution
Video	framerate

Changing resolution

In order to change stream resolution, send a keyframe packet that has the new resolution set:

ElementaryMediaPacket packet;
// ...
packet.width = /* new width */;
packet.height = /* new height */;
// ...

Data payload of the packet must contain appropriate initialization data for the new configuration, depending on a codec that is currently in use.

Changing framerate

In order to change stream framerate, send a keyframe packet that has the new framerate set:

ElementaryMediaPacket packet;
// ...
packet.framerate_num = /* new framerate_num */;
packet.framerate_den = /* new framerate_den */;
// ...

Data payload of the packet must contain appropriate initialization data for the new configuration, depending on a codec that is currently in use.

Handling pipeline errors

A multimedia pipeline in Platform runs asynchronously. Elementary media packets are moved via pipeline and processed after AppendPacket() operation is finished. WASM Player verifies as much data as it's feasible when it's API is called, however some errors can still happen asynchronously (e.g. decoding errors).

App should detect asynchronous errors using HTMLMediaElementListener. The OnError() event should be handled in a custom implementation of HTMLMediaElementListener:

class MyMediaElementListener : public samsung::html::HTMLMediaElementListener {
  using MediaError = samsung::html::MediaError;
  // ...
  // override virtual OnError method
  void OnError(MediaError error_code, const char* error_msg) override {
    // handle media error
  }
  // ...
};

Threading

Media applications tend to make use of threads, to both speed up multimedia processing and free the main thread that is responsible for handling UI. While JavaScript lacks a convenient thread support, WebAssembly offers a full thread support: using threads with WASM Player is encouraged.

Common thread structure

Main application thread is a JS main thread that runs JS event loop. This is the thread that runs all JS events (this includes an UI/user interaction and App's JS code not running in WebWorkers) and can also execute WASM code. However, App shouldn't execute long running operations in WASM module on a main thread.

App should offload work to a number of worker threads to increase App's performance and ensure UI responsiveness. For a multimedia application, worker threads can be used to, for example:

• download data (e.g. when sockets are used),
• process content (e.g. demux containers),
• send and process Elementary Media Packets.

Thread affinity of listeners

All events emitted by listeners related to WASM Player (i.e. ElementaryMediaStreamSourceListener, ElementaryMediaTrackListener, HTMLMediaElementListener) are delivered via main JS thread.

Please note:

• event handlers shouldn't include any long-running operations; if any is necessary, it should be dispatched to a worker thread,
• excessive locking of a main JS thread is not advised and should be avoided whenever possible.

Session id

A session id is an ElementaryMediaTrack parameter associated with all ElementaryMediaPackets sent to a track between OnTrackOpen and OnTrackClose.

A session starts when track opens (OnTrackOpen) and lasts until it closes (OnTrackClose). All packets sent between those two events belong to a single session. When appending either a packet or an end of track to ElementaryMediaTrack, App should mark them with the current session_id value.

When session id changes, an ElementaryMediaTrackListener::OnSessionIdChanged() event is fired:

class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
   // ...
   void OnSessionIdChanged(uint32_t session_id) override {
      // Update stored session_id: propagate it to components managing
      // ElementaryMediaTracks.
   }
   // ...
};

Detecting available WASM Player features

Features supported by WASM Player can vary depending on Platform version. Availability of the features (and presence of the ElementaryMediaStreamSource class implementation itself) can be determined in runtime by checking the EmssVersionInfo structure entries:

// Query platform for WASM Player features.
auto features = samsung::wasm::EmssVersionInfo::Create();

if (features.has_emss |
 features.has_legacy_emss) {
  // Above condition evaluates to true if ElementaryMediaStreamSource data
  // source is available on the Platform.
  std::cout << "Platform supports WASM Player!" << std::endl;
}

Additionally, if WASM Player is not available on the Platform but source object is created, it's ElementaryMediaStreamSource::IsValid() method will always return false.

Getting detailed WASM Player version information

Using the EmssVersionInfo structure is the recommended way to check features supported by WASM Player on the current device. However, this method doesn't provide WASM Player version information directly. Detailed version information can be checked with Tizen TV WASM versioning APIs:

#include <algorithm>
#include <samsung/wasm/tizen_tv_api_info.h>

// ...

using TizenTVApiInfo = samsung::wasm::TizenTVApiInfo;

auto apis = samsung::wasm::GetAvailableApis();
auto api_iterator =
   std::find_if(apis.begin(), apis.end(), [](const TizenTVApiInfo& api) {
      return api.name == "ElementaryMediaStreamSource";
   });

if (api_iterator != apis.end()) {
   // WASM Player is available on the current device! api_iterator entries can
   // be checked for detailed version information.
}

The TizenTVApiInfo structure has following entries:

• name, as seen above. WASM Player is identified by it's main class name: ElementaryMediaStreamSource,
• version, which is a string in format: API_LEVEL.DETAILED_VERSION (e.g. 1.0),
• api_levels, a std::vector<uint32_t> of all API levels supported on the current device.

API Level identifies a single WASM Player API revision, which maps to EmssVersionInfo entries. Detailed version is internal implementation revision id.

Feature availability per Tizen version

WASM Player features available on the current device can be conveniently checked by looking up the EmssVersionInfo structure entries:

EmssVersionInfo.	Description	API Level
has_ultra_low_latency	WASM Player LatencyMode::kUltraLow mode is available on this device	5
has_low_latency_video_texture	WASM Player LatencyMode::kLow in conjunction with RenderingMode::kVideoTexture is available on this device	4
has_decoding_mode	DecodingMode::kSoftware and DecodingMode::kHardwareWithFallback are available on this device	3
has_video_texture	The device supports WASM Player Video Texture mode.	2
has_emss	WASM Player is available on the device.	1
has_legacy_emss	WASM Player is available on the device in the legacy version; please see below for notes on compatibility.	0

Note on compatibility

Presence of ElementaryMediaStreamSource can be determined by checking both has_emss and has_legacy_emss flags. Only one of those two flags can be true at a time.

Depending on the playback scenario App can be obliged to distinguish those two WASM Player versions, as has_legacy_emss one comes with certain limitations. When has_legacy_emss is true:

• seeking requires extra care, please see guidelines in note,
  Note :

  For has_legacy_emss, seeks should follow extra guidelines:

• seek bursts (i.e. multiple consecutive seeks occurring in close succession) should be avoided. Essentially, no new seek should start while HTMLMediaElement.seeking is true.
• App shouldn't call any ElementaryMediaTrack::AppendPacket()s after HTMLMediaElement::SetCurrentTime() is called. No packets should be sent to WASM Player until ElementaryMediaTrackListener::OnTrackOpen() event is fired after seek.

• HTMLMediaElement loop will be executed before playback position reaches duration.

Above affect only Normal Latency mode. Low Latency mode works the same on both versions.