Main class of WASM Player. ElementaryMediaStreamSource acts as a data source for html::HTMLMediaElement. More...

#include <elementary_media_stream_source.h>

Public Types
enum	Mode { Mode::kNormal, Mode::kLowLatency, Mode::kVideoTexture }

enum	LatencyMode { LatencyMode::kNormal, LatencyMode::kLow, LatencyMode::kUltraLow }

enum	RenderingMode { RenderingMode::kMediaElement, RenderingMode::kVideoTexture }

enum	ReadyState { ReadyState::kDetached, ReadyState::kClosed, ReadyState::kOpenPending, ReadyState::kOpen, ReadyState::kEnded }

Public Member Functions
	ElementaryMediaStreamSource (Mode mode=Mode::kNormal)

	ElementaryMediaStreamSource (LatencyMode latency_mode, RenderingMode rendering_mode)

	ElementaryMediaStreamSource (const ElementaryMediaStreamSource &)=delete

	ElementaryMediaStreamSource (ElementaryMediaStreamSource &&)

ElementaryMediaStreamSource &	operator= (const ElementaryMediaStreamSource &)=delete

ElementaryMediaStreamSource &	operator= (ElementaryMediaStreamSource &&)

	~ElementaryMediaStreamSource ()

bool	IsValid () const

Result< ElementaryMediaTrack >	AddTrack (const ElementaryAudioTrackConfig &)

Result< void >	AddTrack (const ElementaryAudioTrackConfig &, std::function< void(OperationResult, ElementaryMediaTrack)> on_finished_callback)

Result< ElementaryMediaTrack >	AddTrack (const ElementaryVideoTrackConfig &)

Result< void >	AddTrack (const ElementaryVideoTrackConfig &, std::function< void(OperationResult, ElementaryMediaTrack)> on_finished_callback)

Result< void >	RemoveTrack (const ElementaryMediaTrack &)

Result< void >	Flush ()

Result< void >	Close (std::function< void(OperationResult)> on_finished_callback)

Result< void >	Open (std::function< void(OperationResult)> on_finished_callback)

Result< Seconds >	GetDuration () const

Result< void >	SetDuration (Seconds new_duration)

Result< Mode >	GetMode () const

Result< LatencyMode >	GetLatencyMode () const

Result< RenderingMode >	GetRenderingMode () const

Result< ReadyState >	GetReadyState () const

Result< void >	SetListener (ElementaryMediaStreamSourceListener *listener)

const char *	GetURL () const

Friends
class	html::HTMLMediaElement

Detailed Description

Main class of WASM Player. ElementaryMediaStreamSource acts as a data source for html::HTMLMediaElement.

ElementaryMediaStreamSource manages a set of ElementaryMediaTrack objects. ElementaryMediaPackets that media consists of are sent to WASM Player via individual tracks added to the source.

Usage overview:

Setup

Create an instance of html::HTMLMediaElement and attach an object implementing the html::HTMLMediaElementListener interface.
Create an instance of ElementaryMediaStreamSource and attach an object implementing the ElementaryMediaStreamSourceListener interface.
Associate media element with source using html::HTMLMediaElement::SetSrc(). This triggers source transition from ReadyState::kDetached to ReadyState::kClosed.
Add ElementaryMediaTracks that will provide data for the media stream using ElementaryMediaStreamSource::AddTrack().
- do not add tracks if they are not used (e.g. do not add audio track if stream doesn't have audio),
- attach an instance of ElementaryMediaTrackListener interface implementation to each track.
Set duration of the stream using ElementaryMediaStreamSource::SetDuration(). This step is skipped if low latency mode is used.
Call ElementaryMediaStreamSource::Open() to conclude setup. Setup is complete once source changes state from ReadyState::kClosed to ReadyState::kOpen.

Sending data for playback

ElementaryMediaStreamSource is a data source. It provides elementary media data for Media Player to play. html::MediaElement is a playback control element: it controls when media is to be played, paused, seeked, and so on.

Changing configuration of the source and sending data will cause HTMLMediaElement::ReadyState to change. Similarly, interacting with html::MediaElement will impact ElementaryMediaStreamSource. Application should listen to ElementaryMediaStreamSourceListener and all ElementaryMediaTrackListeners and react accordingly.

Most notably, changes in both source and track states will inform application when it should send elementary media data to Media Player for playback:

when the source enters ReadyState::kOpen (and each track receives ElementaryMediaTrackListener::OnTrackOpen()) Media Player is ready to accept data,
when the source leaves ReadyState::kOpen (and each track receives ElementaryMediaTrackListener::OnTrackClosed()) Media Player won't accept any more data.

Note: It's recommended to use ElementaryMediaTrackListener to track when data should be sent to Media Player.

A single ElementaryMediaPacket is sent to Media Player using the ElementaryMediaTrack::AppendPacket() method.

Sending data in the normal mode

When Mode::kNormal is used, App should track current playback position using ElementaryMediaStreamSourceListener::OnPlaybackPositionChanged(). Media Player uses buffering in this mode to ensure playback is smooth and App can use current playback position information to ensure packet buffer underrun doesn't happen:

for clear (non-encrypted) content playback at most 64 MiB of data can be buffered,
for DRM-protected content playback at most 10 MiB of data can be buffered,
for audio track at most 768 KiB of data can be buffered,
App can buffer up to 3 seconds of content ahead of the current playback position.

Sending data in the low latency mode

When Mode::kLowLatency is used, Media Player will render packets immediately after they are sent. No internal buffering happens in Media Player. Since this mode is intended for live sources, usually it's a good idea to hand packets to the source right after they are received by App.

See also: Detailed WASM Player Usage Guide on Samsung Developers

Definition at line 143 of file elementary_media_stream_source.h.

Member Enumeration Documentation

◆ LatencyMode

enum samsung::wasm::ElementaryMediaStreamSource::LatencyMode

strong

Defines modes in which ElementaryMediaStreamSource can operate. The mode is set in ElementaryMediaStreamSource's constructor and cannot be changed during its lifetime.

Enumerator

kNormal

This is a default mode, appropriate for most playback scenarios (most notably on-demand video playback).

Pipeline clock is controlled by the Platform when WASM Player works in normal latency mode. ElementaryMediaPackets will be buffered by Platform until they can be rendered according to their pts values.

Platform guarantees smooth media playback when working in this mode, provided packets are delivered to the Source on time (i.e. internal buffer overrun doesn't happen).

kLow

This mode is appropriate for low latency playback scenarios (live streaming).

Pipeline clock is controlled by the application when the player works in low latency mode. Source will render appended packets as soon as possible and won't perform any internal buffering. Pipeline clock is set according to the pts values of appended packets.

Remarks

the application is responsible for maintaining stream synchronization,
packets are rendered as soon as possible, so fps is dependent entirely on when packets are appended,
media element's time is calculated based on packets' pts values, so they should be set correctly,
packets should be tuned for low latency playback (e.g. B-frames are not allowed in video streams).

kUltraLow

This mode is appropriate for ultra low latency playback scenarios (live streaming games). It works in the same way as low-latency mode with following features:

disabled video enhancements,
enabled game mode.

Remarks: Same as in LatencyMode::kLow latency mode.

Definition at line 211 of file elementary_media_stream_source.h.

◆ Mode

enum samsung::wasm::ElementaryMediaStreamSource::Mode

strong

Defines modes in which ElementaryMediaStreamSource can operate. The mode is set in ElementaryMediaStreamSource's constructor and cannot be changed during its lifetime.

Deprecated:: ElementaryMediaStreamSource::Mode is deprecated, use ElementaryMediaStreamSource::LatencyMode and ElementaryMediaStreamSource::RenderingMode (and associated functions) instead.

Enumerator

kNormal

This is a default mode, appropriate for most playback scenarios (most notably on-demand video playback).

Pipeline clock is controlled by the Platform when WASM Player works in normal latency mode. ElementaryMediaPackets will be buffered by Platform until they can be rendered according to their pts values.

Platform guarantees smooth media playback when working in this mode, provided packets are delivered to the Source on time (i.e. internal buffer overrun doesn't happen).

kLowLatency

This mode is appropriate for low latency playback scenarios (live streaming).

Pipeline clock is controlled by the application when the player works in low latency mode. Source will render appended packets as soon as possible and won't perform any internal buffering. Pipeline clock is set according to the pts values of appended packets.

Remarks

the application is responsible for maintaining stream synchronization,
packets are rendered as soon as possible, so fps is dependent entirely on when packets are appended,
media element's time is calculated based on packets' pts values, so they should be set correctly,
packets should be tuned for low latency playback (e.g. B-frames are not allowed in video streams).

kVideoTexture

This mode makes WASM Player decode video packets into GL textures so that they can be rendered by App using OpenGL. Video is not displayed on the associated HTMLMediaElement.

Source will buffer packets until they can be rendered according to their pts values. Buffered packets will be decoded and put into GL textures. Application should constantly request for new pictures by calling ElementaryMediaTrack::FillTextureWithNextFrame() method. This method is called by Platform when the texture should be displayed, so the application should render the texture immediately. After the texture is rendered, it should be released with the ElementaryMediaTrack::RecycleTexture() method.

When this mode is set, Player behaves as in the kNormal mode in terms of available operations, stream synchronization and pipeline clock management.

Remarks: This mode is supported only on devices which have EmssVersionInfo::has_video_texture set to true.

Definition at line 154 of file elementary_media_stream_source.h.

◆ ReadyState

enum samsung::wasm::ElementaryMediaStreamSource::ReadyState

strong

Enumerates all possible states of ElementaryMediaStreamSource. Current ready state of EMSS can be retrieved with GetReadyState() method and its change is signaled by ElementaryMediaStreamSourceListener.

Attention: ElementaryMediaStreamSource and html::HTMLMediaElement states should not be confused. ElementaryMediaStreamSource represents a state of the source of data, while html::HTMLMediaElement represents a state of the multimedia player. As such, ElementaryMediaStreamSource will signal App whether or not multimedia pipeline needs data and can accept ElementaryMediaPackets. This is not in sync with the multimedia player state.

For example, a Source that is in the ReadyState::kEnded state can be associated with a still-playing Media Element. This will occur in Normal Latency mode when App signals end of all tracks but multimedia pipeline has buffered ElementaryMediaPackets that were not displayed yet.

Enumerator
kDetached	Not attached to `html::HTMLMediaElement`. This is the initial state of `ElementaryMediaStreamSource` object. It is also entered after detaching from `html::HTMLMediaElement`.
kClosed	Tracks are not configured and player is not initialized. Can't play in this state. `ElementaryMediaTrack` objects can be added to and removed from `ElementaryMediaStreamSource`. Track layout can be changed only in this state. This state will be entered when: `ElementaryMediaStreamSource` is attached to `html::HTMLMediaElement`, App requests it by calling `Close()`, unrecoverable playback error occurs (this will also cause `html::HTMLMediaElementListener::OnError()` to fire).
kOpenPending	`kOpen` state was requested, but pipeline state prevents entering it. State will change to `ReadyState::kOpen` as soon as possible. This state can be entered both from the `ReadyState::kClosed` state (when opening of Source is requested) and from the `ReadyState::kOpen` state (when pipeline can't accept ES data temporarily, for example during Seek).
kOpen	Player is fully initialized and `ElementaryMediaStreamSource` is ready to accept `ElementaryMediaPacket` data from App via `ElementaryMediaTrack` objects. When App finishes configuring tracks, it can request entering this state by calling `Open()`. This state will be entered when possible. When `ReadyState::kOpen` is the target state of a source, it's possible the `ReadyState::kOpenPending` state will be entered temporarily. This happens when `ElementaryMediaStreamSource` is opening or as a result of some operations (e.g. Seek).
kEnded	Stream has ended but multimedia pipeline remains initialized. Playback can still be restarted, for example by seek. This state is entered when App marks active `ElementaryMediaTrack` objects as ended. The `ReadyState::kEnded` state reverts to the `ReadyState::kOpen` state when multimedia pipeline resumes playback (e.g. due to Seek).

Definition at line 294 of file elementary_media_stream_source.h.

◆ RenderingMode

enum samsung::wasm::ElementaryMediaStreamSource::RenderingMode

strong

Defines modes in which ElementaryMediaStreamSource can operate. The mode is set in ElementaryMediaStreamSource's constructor and cannot be changed during its lifetime.

Enumerator

kMediaElement

This mode is appropriate for decoding video into HTMLMediaElement.

kVideoTexture

This mode makes WASM Player decode video packets into GL textures so that they can be rendered by App using OpenGL. Video is not displayed on the associated HTMLMediaElement.

Application should constantly request for new pictures by calling ElementaryMediaTrack::FillTextureWithNextFrame() method. This method is called by Platform when the texture should be displayed, so the application should render the texture immediately. After the texture is rendered, it should be released with the ElementaryMediaTrack::RecycleTexture() method.

Remarks: This mode is supported only on devices which have EmssVersionInfo::has_video_texture set to true.

Definition at line 256 of file elementary_media_stream_source.h.

Constructor & Destructor Documentation

◆ ElementaryMediaStreamSource() [1/4]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource ( Mode mode = Mode::kNormal )

explicit

Creates a source with the given mode. Mode cannot be changed during lifetime of the object.

Parameters

[in] mode Create a source that uses specified playback mode.

Deprecated:: Use ElementaryMediaStreamSource(LatencyMode, RenderingMode) instead.

◆ ElementaryMediaStreamSource() [2/4]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource	(	LatencyMode	latency_mode,
		RenderingMode	rendering_mode
	)

Creates a source with the given mode. Mode cannot be changed during lifetime of the object.

Parameters

[in]	latencyMode	Latency mode the player should use.
[in]	renderingMode	Specify render target of the player.

◆ ElementaryMediaStreamSource() [3/4]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource ( const ElementaryMediaStreamSource & )

delete

◆ ElementaryMediaStreamSource() [4/4]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource ( ElementaryMediaStreamSource && )

◆ ~ElementaryMediaStreamSource()

samsung::wasm::ElementaryMediaStreamSource::~ElementaryMediaStreamSource ( )

Member Function Documentation

◆ AddTrack() [1/4]

Result<ElementaryMediaTrack> samsung::wasm::ElementaryMediaStreamSource::AddTrack ( const ElementaryAudioTrackConfig & )

Adds an audio track to the source.

Parameters

[in] config A config describing track.

Returns: Result<ElementaryMediaTrack> with operation_result field set to OperationResult::kSuccess and a valid ElementaryMediaTrack object on success, otherwise a code describing the error.

Deprecated:: since version 3.0 This version has no support for reporting errors in case of not available hardware decoder. Please use AddTrack with callback instead.

Remarks

Tracks can only be added in kClosed state.
Only one track of each type can be held by the source at any given time.
Audio parameters cannot be changed during the lifetime of the track.

◆ AddTrack() [2/4]

Result<void> samsung::wasm::ElementaryMediaStreamSource::AddTrack	(	const ElementaryAudioTrackConfig &	,
		std::function< void(OperationResult, ElementaryMediaTrack)>	on_finished_callback
	)

Asynchronously adds an audio track to the source.

Parameters

[in]	config	A config describing track.
[in]	on_finished_callback	A callback notifying end of `AddTrack()`. The callback receives `OperationResult` informing of the result of the operation and `ElementaryMediaTrack` object - invalid in case of operation failure.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

Remarks

Tracks can only be added in kClosed state.
Only one track of each type can be held by the source at any given time.
Audio parameters cannot be changed during the lifetime of the track.

◆ AddTrack() [3/4]

Result<ElementaryMediaTrack> samsung::wasm::ElementaryMediaStreamSource::AddTrack ( const ElementaryVideoTrackConfig & )

Adds a video track to the source.

Parameters

[in] config A config describing track.

Returns: Result<ElementaryMediaTrack> with operation_result field set to OperationResult::kSuccess and a valid ElementaryMediaTrack object on success, otherwise a code describing the error.

Deprecated:: since version 3.0 This version has no support for reporting errors in case of not available hardware decoder. Please use AddTrack with callback instead.

Remarks

Tracks can only be added in kClosed state.
Only one track of each type can be held by the source at any given time.
As opposed to audio track, some video parameters can change during playback, notably resolution or framerate, by passing packets with new resolution/fps values.

◆ AddTrack() [4/4]

Result<void> samsung::wasm::ElementaryMediaStreamSource::AddTrack	(	const ElementaryVideoTrackConfig &	,
		std::function< void(OperationResult, ElementaryMediaTrack)>	on_finished_callback
	)

Asynchronously adds a video track to the source.

Parameters

[in]	config	A config describing track.
[in]	on_finished_callback	A callback notifying end of `AddTrack()`. The callback receives `OperationResult` informing of the result of the operation and `ElementaryMediaTrack` object - invalid in case of operation failure.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

Remarks

Tracks can only be added in kClosed state.
Only one track of each type can be held by the source at any given time.
As opposed to audio track, some video parameters can change during playback, notably resolution or framerate, by passing packets with new resolution/fps values.

◆ Close()

Result<void> samsung::wasm::ElementaryMediaStreamSource::Close ( std::function< void(OperationResult)> on_finished_callback )

Closes the source asynchronously. When the operation is done, the source will be in ReadyState::kClosed state and a callback passed as the argument will be called. During that time it's impossible to request another ready state change.

Parameters

[in] on_finished_callback A callback notifying end of Close(). The callback receives OperationResult informing of the result of the operation.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

◆ Flush()

Result<void> samsung::wasm::ElementaryMediaStreamSource::Flush ( )

Flushes internal packets' buffers, causing ElementaryMediaTracks belonging to this Source to drop all packets appended so far.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

◆ GetDuration()

Result<Seconds> samsung::wasm::ElementaryMediaStreamSource::GetDuration ( ) const

Returns the duration of the source.

Returns: Result<Seconds> with operation_result field set to OperationResult::kSuccess and a valid Seconds object representing duration of the source on success, otherwise a code describing the error.

◆ GetLatencyMode()

Result<LatencyMode> samsung::wasm::ElementaryMediaStreamSource::GetLatencyMode ( ) const

Returns the latency mode of the source, as set during object construction.

Returns: Result<ElementaryMediaStreamSource::LatencyMode> with operation_result field set to OperationResult::kSuccess and a valid LatencyMode representing mode of the source on success, otherwise a code describing the error.

◆ GetMode()

Result<Mode> samsung::wasm::ElementaryMediaStreamSource::GetMode ( ) const

Returns the mode of the source, as set during object construction.

Returns: Result<ElementaryMediaStreamSource::Mode> with operation_result field set to OperationResult::kSuccess and a valid Mode representing mode of the source on success, otherwise a code describing the error.

Deprecated:: Use GetLatencyMode() and GetRenderingMode() instead

◆ GetReadyState()

Result<ReadyState> samsung::wasm::ElementaryMediaStreamSource::GetReadyState ( ) const

Returns current ReadyState of the source.

Returns: Result<ReadyState> with operation_result field set to OperationResult::kSuccess and a valid ReadyState representing current ready state of the source on success, otherwise a code describing the error.

◆ GetRenderingMode()

Result<RenderingMode> samsung::wasm::ElementaryMediaStreamSource::GetRenderingMode ( ) const

Returns the rendering mode of the source, as set during object construction.

Returns: Result<ElementaryMediaStreamSource::RenderingMode> with operation_result field set to OperationResult::kSuccess and a valid RenderingMode representing mode of the source on success, otherwise a code describing the error.

◆ GetURL()

const char* samsung::wasm::ElementaryMediaStreamSource::GetURL ( ) const

Returns the source's URL received from URL.createObjectURL WebAPI. The URL is bound to the source's lifetime and will be revoked automatically.

Returns: c-style string containing the URL. Ownership is not transferred and the string is valid as long as this object is alive.

◆ IsValid()

bool samsung::wasm::ElementaryMediaStreamSource::IsValid ( ) const

Returns true if source instance is valid. This method should be called after constructor to ensure backend initialized the object properly. If source is invalid all method calls will fail.

Returns: true if source instance is valid, otherwise false.

◆ Open()

Result<void> samsung::wasm::ElementaryMediaStreamSource::Open ( std::function< void(OperationResult)> on_finished_callback )

Starts an asynchronous opening operation. When the operation is done, the source will be in ReadyState::kOpen state and a callback passed as the argument will be called. During that time it's impossible to request another ready state change.

Parameters

[in] on_finished_callback A callback notifying end of Open(). The callback receives OperationResult informing of the result of the operation.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

Remarks: This should be called when all tracks are added and configured, and the application is ready to provide data for the source.

◆ operator=() [1/2]

ElementaryMediaStreamSource& samsung::wasm::ElementaryMediaStreamSource::operator= ( const ElementaryMediaStreamSource & )

delete

◆ operator=() [2/2]

ElementaryMediaStreamSource& samsung::wasm::ElementaryMediaStreamSource::operator= ( ElementaryMediaStreamSource && )

◆ RemoveTrack()

Result<void> samsung::wasm::ElementaryMediaStreamSource::RemoveTrack ( const ElementaryMediaTrack & )

Removes a track from the source. After the operation removed track is still valid and can be re-added to the source again.

Parameters

[in] track Track to remove.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

Remarks

Tracks can only be removed in kClosed state.

◆ SetDuration()

Result<void> samsung::wasm::ElementaryMediaStreamSource::SetDuration ( Seconds new_duration )

Sets the duration of the source. Note that this operation is unavailable in Mode::kLowLatency mode.

Parameters

[in] new_duration Duration to be set.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

◆ SetListener()

Result<void> samsung::wasm::ElementaryMediaStreamSource::SetListener ( ElementaryMediaStreamSourceListener * listener )

Sets a listener to receive updates about EMSS's state changes. Only one listener can be set: setting another clears the previous one. Pass nullptr to reset the listener

Parameters

[in] listener Listener to be set or nullptr to unset the listener.

Warning: The ownership isn't transferred, and, as such, the listener must outlive the source.

Returns: Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

See also: ElementaryMediaStreamSourceListener

Friends And Related Function Documentation

◆ html::HTMLMediaElement

friend class html::HTMLMediaElement

friend

Definition at line 604 of file elementary_media_stream_source.h.

The documentation for this class was generated from the following file:

elementary_media_stream_source.h

Public Types

Public Member Functions

Friends

Detailed Description

Usage overview:

Setup

Sending data for playback

Sending data in the normal mode

Sending data in the low latency mode

Member Enumeration Documentation

◆ LatencyMode

◆ Mode

◆ ReadyState

◆ RenderingMode

Constructor & Destructor Documentation

◆ ElementaryMediaStreamSource() [1/4]

◆ ElementaryMediaStreamSource() [2/4]

◆ ElementaryMediaStreamSource() [3/4]

◆ ElementaryMediaStreamSource() [4/4]

◆ ~ElementaryMediaStreamSource()

Member Function Documentation

◆ AddTrack() [1/4]

◆ AddTrack() [2/4]

◆ AddTrack() [3/4]

◆ AddTrack() [4/4]

◆ Close()

◆ Flush()

◆ GetDuration()

◆ GetLatencyMode()

◆ GetMode()

◆ GetReadyState()

◆ GetRenderingMode()

◆ GetURL()

◆ IsValid()

◆ Open()

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ RemoveTrack()

◆ SetDuration()

◆ SetListener()

Friends And Related Function Documentation

◆ html::HTMLMediaElement