Using Advanced Player Features

This topic covers using advanced Tizen WASM Player features such as multitasking and DRM, as well as detecting which WASM Player features are available on a device.

Detecting Available WASM Player Features

Features supported by the WASM Player and the support for the WASM Player itself can vary depending on the platform version. Availability of the features (and presence of the ElementaryMediaStreamSource class implementation) can be determined at runtime by checking the EmssVersionInfo structure entries:

// Query platform for WASM Player features
auto features = samsung::wasm::EmssVersionInfo::Create();

if (features.has_emss || features.has_legacy_emss) {
  // Above condition evaluates to true if ElementaryMediaStreamSource data
  // source is available on the platform
  std::cout << "Platform supports WASM Player!" << std::endl;
}

Additionally, if the WASM Player is not available on the platform but a source object is created, its ElementaryMediaStreamSource::IsValid() method always returns false.

Getting Detailed WASM Player Version Information

Use the EmssVersionInfo structure to check features supported by WASM Player on the current device. However, this method does not provide the WASM Player version information directly. Detailed version information can be checked with Tizen TV WASM versioning APIs:

#include <algorithm>
#include <samsung/wasm/tizen_tv_api_info.h>

// ...

using TizenTVApiInfo = samsung::wasm::TizenTVApiInfo;

auto apis = samsung::wasm::GetAvailableApis();
auto api_iterator =
   std::find_if(apis.begin(), apis.end(), [](const TizenTVApiInfo&amp; api) {
      return api.name == "ElementaryMediaStreamSource";
   });

if (api_iterator != apis.end()) {
   // WASM Player is available on the current device! api_iterator entries can
   // be checked for detailed version information.
}

The TizenTVApiInfo structure has the following entries:

• name, as seen above. The WASM Player is identified by its main class name: ElementaryMediaStreamSource.
• version, which is a string in format: API_LEVEL.DETAILED_VERSION (such as 1.0).
• api_levels, a std::vector<uint32_t> of all API levels supported on the current device.

The API level identifies a single WASM Player API revision, which maps to EmssVersionInfo entries. Detailed version is an internal implementation revision ID.

Features and Compatibility per Tizen Version

WASM Player features available on the current device can be conveniently checked by looking up the EmssVersionInfo structure entries:

EmssVersionInfo Entry	Description	API Level
has_ultra_low_latency	WASM Player LatencyMode::kUltraLow mode is available on this device	5
has_low_latency_video_texture	WASM Player LatencyMode::kLow in conjunction with RenderingMode::kVideoTexture is available on this device	4
has_decoding_mode	DecodingMode::kSoftware and DecodingMode::kHardwareWithFallback are available on this device	3
has_video_texture	The device supports WASM Player Video Texture mode.	2
has_emss	WASM Player is available on the device.	1
has_legacy_emss	WASM Player is available on the device in the legacy version; please see below for notes on compatibility.	0

Table 1: EmssVersionInfo Structure

Presence of ElementaryMediaStreamSource can be determined by checking both has_emss and has_legacy_emss flags. Only one of these two flags can be true at a time. Depending on the playback scenario, the application can be obliged to distinguish between these two WASM Player versions, since has_legacy_emss being true comes with certain limitations:

• Seek operations for has_legacy_emss require extra care, see the following guidelines:
  • Seek bursts (multiple consecutive seeks occurring in close succession) must be avoided. Essentially, no new seek can start while HTMLMediaElement.seeking is true.
  • The application must not call any ElementaryMediaTrack::AppendPacket()s after HTMLMediaElement::SetCurrentTime() is called. No packets can be sent to the WASM Player until ElementaryMediaTrackListener::OnTrackOpen() event is fired after the Seek.
• HTMLMediaElement loop is executed before the playback position reaches duration.

The above differences affect only the normal latency mode. Low latency modes work the same on both versions.

Multitasking

Multitasking happens when the application is either hidden from the screen or moved back to being shown on the device's screen (such as when the user switches the active application). The application needs to always properly handle Multitasking.

When the application is suspended (in other words, when visibilitychange reports document.hidden changing to true):

• If ElementaryMediaStreamSource is ReadyState::kOpen:
  • ElementaryMediaStreamSource state is changed to ReadyState::kOpenPending and ElementaryMediaStreamSourceListener::OnSourceOpenPending() is fired.
  • ElementaryMediaTracks are closed and ElementaryMediaTrackListener::OnTrackClosed() with CloseReason::kSourceSuspended is fired for each track.
• If the HTMLMediaElement::IsPaused() attribute is false, the HTMLMediaElement is paused and HTMLMediaElement::GetCurrentTime() is stored internally by WASM Player.
  
  (This step is omitted in low latency modes)
  When the application is resumed (in other words, when visibilitychange reports document.hidden changing to false):
• If ElementaryMediaStreamSource was ReadyState::kOpen prior to suspend, an automatic Seek is performed to the stored HTMLMediaElement::GetCurrentTime() value.
  
  As a result, ElementaryMediaStreamSource is set to ReadyState::kOpen and each ElementaryMediaTrack is opened.
  
  (In low latency modes, the Seek operation is omitted; instead the source and tracks are opened immediately.)
• If HTMLMediaElement::IsPaused() was false before playback being suspended, the HTMLMediaElement is unpaused.

Threading

Media applications tend to make use of threads, to both speed up multimedia processing and free the main thread that is responsible for handling the UI. While JavaScript lacks convenient thread support, WebAssembly offers full thread support, so using threads with the WASM Player is encouraged.

The main application thread is a JS main thread that runs a JS event loop. This is the thread that runs all JS events (this includes UI/user interaction and the application's JS code not running in WebWorkers) and that can also execute WASM code. However, the application must not execute long running operations in WASM module on the main thread.

The application must offload work to a number of worker threads to increase performance and ensure UI responsiveness. For a multimedia application, worker threads can be used to, for example:

• Download data (For example, when sockets are used)
• Process content (Such as. demux containers)
• Send and process Elementary Media Packets

Thread Affinity of Listeners

All events emitted by listeners related to the WASM Player (ElementaryMediaStreamSourceListener, ElementaryMediaTrackListener, and HTMLMediaElementListener) are delivered through the main JS thread.
Please note:

• Event handlers must not include any long-running operations; if one is necessary, dispatch it to a worker thread.
• Excessive locking of the main JS thread is not advised, and must be avoided whenever possible.

Session ID

A session ID is an ElementaryMediaTrack parameter associated with all ElementaryMediaPacket objects sent to a track between OnTrackOpen and OnTrackClose.

A session starts when the track opens (OnTrackOpen) and lasts until it closes (OnTrackClose). All packets sent between those two events belong to a single session. When appending either a packet or an end of track to ElementaryMediaTrack, the application must mark them with the current session_id value.

When session ID changes, an ElementaryMediaTrackListener::OnSessionIdChanged() event is fired:

class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener {
   // ...
   void OnSessionIdChanged(uint32_t session_id) override {
      // Update stored session_id: propagate it to components managing
      // ElementaryMediaTracks
   }
   // ...
};

Using DRM

Then WASM Player supports DRM-protected content playback. Supported formats are the same as those specified in the EME section in Media Specifications.

To set up encrypted media playback in the application:

• Prepare a DRMConfig structure:

  samsung::wasm::DRMConfig drm_config;
  drm_config.cdm = /* samsung::wasm::ContentDecryptionModule value */;
  drm_config.encryption_mode = /* samsung::wasm::EncryptionMode value */;
  drm_config.license_server /* URL to a license server */;
  drm_config.init_data = /* DRM system-specific initialization data */;
  
  // If audio track is encrypted:
  drm_config.audio_mime_type = /* MIME type of encrypted audio or empty string if track is clear */;
  drm_config.audio_robustness = /* samsung::wasm::Robustness parameter for audio track */;
  
  // If video track is encrypted:
  drm_config.video_mime_type = /* MIME type of encrypted video or empty string if track is clear */;
  drm_config.video_robustness = /* samsung::wasm::Robustness parameter for video track */;
  

• Create a MediaKey using the DRMConfig created in the previous step and assign it to the corresponding ElementaryMediaTracks:

  using OperationResult = samsung::wasm::OperationResult;
  using MediaKey = samsung::wasm::MediaKey;
  
  // ...
  
  // Created MediaKey must be stored by the application:
  MediaKey decryption_key;
  
  // ...
  
  auto result = samsung::wasm::MediaKey::SetupEncryption(drm_config, [](OperationResult result, MediaKey media_key) {
    if (result != OperationResult::kSuccess) {
      // Handle error
      return;
    }
    // Store the key. Make sure it outlives tracks it is assigned to below.
    decryption_key = std::move(media_key);
  
    // If audio is encrypted...
    audio_track->SetMediaKey(&decryption_key)
  
    // If video is encrypted...
    video_track->SetMediaKey(&decryption_key)
  })
  

  NoteWhen creating a MediaKey instance, implementation automatically contacts the license server specified in the drm_config.license_server parameter.

• When multimedia content is DRM-protected, the application needs to follow normal usage guidelines for Elementary Media Stream source except for one difference: encrypted Elementary Media Packets are sent to ElementaryMediaTracks with an AppendEncryptedPacketAsync method:

  samsung::wasm::EncryptedElementaryMediaPacket encrypted_packet;
  // ...
  // Set all clear packet information as usual
  // ...
  encrypted_packet.subsamples = /* Subsample information: a vector of samsung::wasm::EncryptedSubsampleDescription */;
  encrypted_packet.key_id = /* Id of a key that should be used to decrypt this packet */;
  encrypted_packet.initialization_vector = /* IV required to decrypt this packet */;
  encrypted_packet.encryption_mode = /* samsung::wasm::EncryptionMode for this packet */;
  
  auto result = track->AppendEncryptedPacketAsync(encrypted_packet);
  
  if (!result) {
    // Handle error
  }