This topic covers using advanced Tizen WASM Player features such as multitasking and DRM, as well as detecting which WASM Player features are available on a device.
Features supported by the WASM Player and the support for the WASM Player itself can vary depending on the platform version. Availability of the features (and presence of the ElementaryMediaStreamSource class implementation) can be determined at runtime by checking the EmssVersionInfo structure entries:
ElementaryMediaStreamSource
EmssVersionInfo
// Query platform for WASM Player features auto features = samsung::wasm::EmssVersionInfo::Create(); if (features.has_emss || features.has_legacy_emss) { // Above condition evaluates to true if ElementaryMediaStreamSource data // source is available on the platform std::cout << "Platform supports WASM Player!" << std::endl; }
Additionally, if the WASM Player is not available on the platform but a source object is created, its ElementaryMediaStreamSource::IsValid() method always returns false.
ElementaryMediaStreamSource::IsValid()
false
Use the EmssVersionInfo structure to check features supported by WASM Player on the current device. However, this method does not provide the WASM Player version information directly. Detailed version information can be checked with Tizen TV WASM versioning APIs:
#include <algorithm> #include <samsung/wasm/tizen_tv_api_info.h> // ... using TizenTVApiInfo = samsung::wasm::TizenTVApiInfo; auto apis = samsung::wasm::GetAvailableApis(); auto api_iterator = std::find_if(apis.begin(), apis.end(), [](const TizenTVApiInfo& api) { return api.name == "ElementaryMediaStreamSource"; }); if (api_iterator != apis.end()) { // WASM Player is available on the current device! api_iterator entries can // be checked for detailed version information. }
The TizenTVApiInfo structure has the following entries:
TizenTVApiInfo
name
version
API_LEVEL.DETAILED_VERSION
1.0
api_levels
std::vector<uint32_t>
The API level identifies a single WASM Player API revision, which maps to EmssVersionInfo entries. Detailed version is an internal implementation revision ID.
WASM Player features available on the current device can be conveniently checked by looking up the EmssVersionInfo structure entries:
EmssVersionInfo.
has_ultra_low_latency
LatencyMode::kUltraLow
has_low_latency_video_texture
LatencyMode::kLow
RenderingMode::kVideoTexture
has_decoding_mode
DecodingMode::kSoftware
DecodingMode::kHardwareWithFallback
has_video_texture
has_emss
has_legacy_emss
Presence of ElementaryMediaStreamSource can be determined by checking both has_emss and has_legacy_emss flags. Only one of these two flags can be true at a time. Depending on the playback scenario, the application can be obliged to distinguish between these two WASM Player versions, since has_legacy_emss being true comes with certain limitations:
true
HTMLMediaElement.seeking
ElementaryMediaTrack::AppendPacket()
HTMLMediaElement::SetCurrentTime()
ElementaryMediaTrackListener::OnTrackOpen()
HTMLMediaElement
The above differences affect only the normal latency mode. Low latency modes work the same on both versions.
Multitasking happens when the application is either hidden from the screen or moved back to being shown on the device's screen (such as when the user switches the active application). The application needs to always properly handle Multitasking.
When the application is suspended (in other words, when visibilitychange reports document.hidden changing to true):
visibilitychange
document.hidden
ReadyState::kOpen
ReadyState::kOpenPending
ElementaryMediaStreamSourceListener::OnSourceOpenPending()
ElementaryMediaTrack
ElementaryMediaTrackListener::OnTrackClosed()
CloseReason::kSourceSuspended
HTMLMediaElement::IsPaused()
HTMLMediaElement::GetCurrentTime()
When the application is resumed (in other words, when visibilitychange reports document.hidden changing to false):
Media applications tend to make use of threads, to both speed up multimedia processing and free the main thread that is responsible for handling the UI. While JavaScript lacks convenient thread support, WebAssembly offers full thread support, so using threads with the WASM Player is encouraged.
The main application thread is a JS main thread that runs a JS event loop. This is the thread that runs all JS events (this includes UI/user interaction and the application's JS code not running in WebWorkers) and that can also execute WASM code. However, the application must not execute long running operations in WASM module on the main thread.
The application must offload work to a number of worker threads to increase performance and ensure UI responsiveness. For a multimedia application, worker threads can be used to, for example:
All events emitted by listeners related to the WASM Player (ElementaryMediaStreamSourceListener, ElementaryMediaTrackListener, and HTMLMediaElementListener) are delivered through the main JS thread.
ElementaryMediaStreamSourceListener
ElementaryMediaTrackListener
HTMLMediaElementListener
Please note:
A session ID is an ElementaryMediaTrack parameter associated with all ElementaryMediaPacket objects sent to a track between OnTrackOpen and OnTrackClose.
ElementaryMediaPacket
OnTrackOpen
OnTrackClose
A session starts when the track opens (OnTrackOpen) and lasts until it closes (OnTrackClose). All packets sent between those two events belong to a single session. When appending either a packet or an end of track to ElementaryMediaTrack, the application must mark them with the current session_id value.
session_id
When session ID changes, an ElementaryMediaTrackListener::OnSessionIdChanged() event is fired:
ElementaryMediaTrackListener::OnSessionIdChanged()
class MyTrackListener : public samsung::wasm::ElementaryMediaTrackListener { // ... void OnSessionIdChanged(uint32_t session_id) override { // Update stored session_id: propagate it to components managing // ElementaryMediaTracks } // ... };
Note ElementaryMediaPackets marked with non-current session IDs are dropped by the platform. This causes AppendPacket() to return an OperationResult::kAppendIgnored error. However, this is not a fatal condition and does not impair playback in a properly written application.
ElementaryMediaPackets marked with non-current session IDs are dropped by the platform. This causes AppendPacket() to return an OperationResult::kAppendIgnored error. However, this is not a fatal condition and does not impair playback in a properly written application.
AppendPacket()
OperationResult::kAppendIgnored
Note This section does not apply in low latency modes, as they do not support DRM.
This section does not apply in low latency modes, as they do not support DRM.
Then WASM Player supports DRM-protected content playback. Supported formats are the same as those specified in the EME section in Media Specifications.
To set up encrypted media playback in the application:
Prepare a DRMConfig structure:
DRMConfig
samsung::wasm::DRMConfig drm_config; drm_config.cdm = /* samsung::wasm::ContentDecryptionModule value */; drm_config.encryption_mode = /* samsung::wasm::EncryptionMode value */; drm_config.license_server /* URL to a license server */; drm_config.init_data = /* DRM system-specific initialization data */; // If audio track is encrypted: drm_config.audio_mime_type = /* MIME type of encrypted audio or empty string if track is clear */; drm_config.audio_robustness = /* samsung::wasm::Robustness parameter for audio track */; // If video track is encrypted: drm_config.video_mime_type = /* MIME type of encrypted video or empty string if track is clear */; drm_config.video_robustness = /* samsung::wasm::Robustness parameter for video track */;
Create a MediaKey using the DRMConfig created in the previous step and assign it to the corresponding ElementaryMediaTracks:
MediaKey
using OperationResult = samsung::wasm::OperationResult; using MediaKey = samsung::wasm::MediaKey; // ... // Created MediaKey must be stored by the application: MediaKey decryption_key; // ... auto result = samsung::wasm::MediaKey::SetupEncryption(drm_config, [](OperationResult result, MediaKey media_key) { if (result != OperationResult::kSuccess) { // Handle error return; } // Store the key. Make sure it outlives tracks it is assigned to below. decryption_key = std::move(media_key); // If audio is encrypted... audio_track->SetMediaKey(&decryption_key) // If video is encrypted... video_track->SetMediaKey(&decryption_key) })
Note When creating a MediaKey instance, implementation automatically contacts the license server specified in the drm_config.license_server parameter.
When creating a MediaKey instance, implementation automatically contacts the license server specified in the drm_config.license_server parameter.
drm_config.license_server
When multimedia content is DRM-protected, the application needs to follow normal usage guidelines for Elementary Media Stream source except for one difference: encrypted Elementary Media Packets are sent to ElementaryMediaTracks with an AppendEncryptedPacketAsync method:
AppendEncryptedPacketAsync
samsung::wasm::EncryptedElementaryMediaPacket encrypted_packet; // ... // Set all clear packet information as usual // ... encrypted_packet.subsamples = /* Subsample information: a vector of samsung::wasm::EncryptedSubsampleDescription */; encrypted_packet.key_id = /* Id of a key that should be used to decrypt this packet */; encrypted_packet.initialization_vector = /* IV required to decrypt this packet */; encrypted_packet.encryption_mode = /* samsung::wasm::EncryptionMode for this packet */; auto result = track->AppendEncryptedPacketAsync(encrypted_packet); if (!result) { // Handle error }
Note AppendEncryptedPacketAsync() validates the packet and returns a result synchronously. However, packet decryption is executed asynchronously. If a decryption error occurs, it is signaled with an ElementaryMediaTrackListener::OnAppendError() event. For details, see Handling async append errors. To receive the decryption result synchronously, it is possible to use an AppendEncryptedPacket() operation instead. However, decrypting packets synchronously is not recommended: decryption is potentially a long running operation and blocking the append thread for extended periods of time can reduce video playback performance. Appends for the same track can be a mix of synchronous and asynchronous calls.
AppendEncryptedPacketAsync() validates the packet and returns a result synchronously. However, packet decryption is executed asynchronously. If a decryption error occurs, it is signaled with an ElementaryMediaTrackListener::OnAppendError() event. For details, see Handling async append errors.
AppendEncryptedPacketAsync()
ElementaryMediaTrackListener::OnAppendError()
To receive the decryption result synchronously, it is possible to use an AppendEncryptedPacket() operation instead. However, decrypting packets synchronously is not recommended: decryption is potentially a long running operation and blocking the append thread for extended periods of time can reduce video playback performance.
AppendEncryptedPacket()
Appends for the same track can be a mix of synchronous and asynchronous calls.