Tizen WASM Player
Tizen TV WebAssembly Media Player extension allowing for a low-level elementary media stream playback.
samsung::wasm::ElementaryMediaStreamSource Class Referencefinal

Main class of WASM Player. ElementaryMediaStreamSource acts as a data source for html::HTMLMediaElement. More...

#include <elementary_media_stream_source.h>

Public Types

enum  Mode {
  Mode::kNormal,
  Mode::kLowLatency,
  Mode::kVideoTexture
}
 
enum  ReadyState {
  ReadyState::kDetached,
  ReadyState::kClosed,
  ReadyState::kOpenPending,
  ReadyState::kOpen,
  ReadyState::kEnded
}
 

Public Member Functions

 ElementaryMediaStreamSource (Mode mode=Mode::kNormal)
 
 ElementaryMediaStreamSource (const ElementaryMediaStreamSource &)=delete
 
 ElementaryMediaStreamSource (ElementaryMediaStreamSource &&)
 
ElementaryMediaStreamSourceoperator= (const ElementaryMediaStreamSource &)=delete
 
ElementaryMediaStreamSourceoperator= (ElementaryMediaStreamSource &&)
 
 ~ElementaryMediaStreamSource ()
 
bool IsValid () const
 
Result< ElementaryMediaTrackAddTrack (const ElementaryAudioTrackConfig &)
 
Result< ElementaryMediaTrackAddTrack (const ElementaryVideoTrackConfig &)
 
Result< void > RemoveTrack (const ElementaryMediaTrack &)
 
Result< void > Flush ()
 
Result< void > Close (std::function< void(OperationResult)> on_finished_callback)
 
Result< void > Open (std::function< void(OperationResult)> on_finished_callback)
 
Result< SecondsGetDuration () const
 
Result< void > SetDuration (Seconds new_duration)
 
Result< ModeGetMode () const
 
Result< ReadyStateGetReadyState () const
 
Result< void > SetListener (ElementaryMediaStreamSourceListener *listener)
 
const char * GetURL () const
 

Friends

class html::HTMLMediaElement
 

Detailed Description

Main class of WASM Player. ElementaryMediaStreamSource acts as a data source for html::HTMLMediaElement.

ElementaryMediaStreamSource manages a set of ElementaryMediaTrack objects. ElementaryMediaPackets that media consists of are sent to WASM Player via individual tracks added to the source.

Usage overview:

Setup

  1. Create an instance of html::HTMLMediaElement and attach an object implementing the html::HTMLMediaElementListener interface.
  2. Create an instance of ElementaryMediaStreamSource and attach an object implementing the ElementaryMediaStreamSourceListener interface.
  3. Associate media element with source using html::HTMLMediaElement::SetSrc(). This triggers source transition from ReadyState::kDetached to ReadyState::kClosed.
  4. Add ElementaryMediaTracks that will provide data for the media stream using ElementaryMediaStreamSource::AddTrack().
    • do not add tracks if they are not used (e.g. do not add audio track if stream doesn't have audio),
    • attach an instance of ElementaryMediaTrackListener interface implementation to each track.
  5. Set duration of the stream using ElementaryMediaStreamSource::SetDuration(). This step is skipped if low latency mode is used.
  6. Call ElementaryMediaStreamSource::Open() to conclude setup. Setup is complete once source changes state from ReadyState::kClosed to ReadyState::kOpen.

Sending data for playback

ElementaryMediaStreamSource is a data source. It provides elementary media data for Media Player to play. html::MediaElement is a playback control element: it controls when media is to be played, paused, seeked, and so on.

Changing configuration of the source and sending data will cause HTMLMediaElement::ReadyState to change. Similarly, interacting with html::MediaElement will impact ElementaryMediaStreamSource. Application should listen to ElementaryMediaStreamSourceListener and all ElementaryMediaTrackListeners and react accordingly.

Most notably, changes in both source and track states will inform application when it should send elementary media data to Media Player for playback:

Note
It's recommended to use ElementaryMediaTrackListener to track when data should be sent to Media Player.

A single ElementaryMediaPacket is sent to Media Player using the ElementaryMediaTrack::AppendPacket() method.

Sending data in the normal mode

When Mode::kNormal is used, App should track current playback position using ElementaryMediaStreamSourceListener::OnPlaybackPositionChanged(). Media Player uses buffering in this mode to ensure playback is smooth and App can use current playback position information to ensure packet buffer underrun doesn't happen:

  • for clear (non-encrypted) content playback at most 64 MiB of data can be buffered,
  • for DRM-protected content playback at most 10 MiB of data can be buffered,
  • for audio track at most 768 KiB of data can be buffered,
  • App can buffer up to 3 seconds of content ahead of the current playback position.

Sending data in the low latency mode

When Mode::kLowLatency is used, Media Player will render packets immediately after they are sent. No internal buffering happens in Media Player. Since this mode is intended for live sources, usually it's a good idea to hand packets to the source right after they are received by App.

See also
Detailed WASM Player Usage Guide on Samsung Developers

Definition at line 142 of file elementary_media_stream_source.h.

Member Enumeration Documentation

◆ Mode

Defines modes in which ElementaryMediaStreamSource can operate. The mode is set in ElementaryMediaStreamSource's constructor and cannot be changed during its lifetime.

Enumerator
kNormal 

This is a default mode, appropriate for most playback scenarios (most notably on-demand video playback).

Pipeline clock is controlled by the Platform when WASM Player works in normal latency mode. ElementaryMediaPackets will be buffered by Platform until they can be rendered according to their pts values.

Platform guarantees smooth media playback when working in this mode, provided packets are delivered to the Source on time (i.e. internal buffer overrun doesn't happen).

kLowLatency 

This mode is appropriate for low latency playback scenarios (live streaming).

Pipeline clock is controlled by the application when the player works in low latency mode. Source will render appended packets as soon as possible and won't perform any internal buffering. Pipeline clock is set according to the pts values of appended packets.

Remarks
  • the application is responsible for maintaining stream synchronization,
  • packets are rendered as soon as possible, so fps is dependent entirely on when packets are appended,
  • media element's time is calculated based on packets' pts values, so they should be set correctly,
  • packets should be tuned for low latency playback (e.g. B-frames are not allowed in video streams).
kVideoTexture 

This mode makes WASM Player decode video packets into GL textures so that they can be renderer by App using OpenGL. Video is not displayed on the associated HTMLMediaElement.

Source will buffer packets until they can be rendered according to their pts values. Buffered packets will be decoded and put into GL textures. Application should constantly request for new pictures by calling ElementaryMediaTrack::FillTextureWithNextFrame() method. This method is called by Platform when the texture should be displayed, so the application should render the texture immediately. After the texture is rendered, it should be released with the ElementaryMediaTrack::RecycleTexture() method.

When this mode is set, Player behaves as in the kNormal mode in terms of available operations, stream synchronization and pipeline clock management.

Remarks
This mode is supported only on devices which have EmssVersionInfo::has_video_texture set to true.

Definition at line 147 of file elementary_media_stream_source.h.

◆ ReadyState

Enumerates all possible states of ElementaryMediaStreamSource. Current ready state of EMSS can be retrieved with GetReadyState() method and its change is signaled by ElementaryMediaStreamSourceListener.

Attention
ElementaryMediaStreamSource and html::HTMLMediaElement states should not be confused. ElementaryMediaStreamSource represents a state of the source of data, while html::HTMLMediaElement represents a state of the multimedia player. As such, ElementaryMediaStreamSource will signal App whether or not multimedia pipeline needs data and can accept ElementaryMediaPackets. This is not in sync with the multimedia player state.

For example, a Source that is in the ReadyState::kEnded state can be associated with a still-playing Media Element. This will occur in Normal Latency mode when App signals end of all tracks but multimedia pipeline has buffered ElementaryMediaPackets that were not displayed yet.

Enumerator
kDetached 

Not attached to html::HTMLMediaElement. This is the initial state of ElementaryMediaStreamSource object. It is also entered after detaching from html::HTMLMediaElement.

kClosed 

Tracks are not configured and player is not initialized. Can't play in this state. ElementaryMediaTrack objects can be added to and removed from ElementaryMediaStreamSource. Track layout can be changed only in this state.

This state will be entered when:

kOpenPending 

kOpen state was requested, but pipeline state prevents entering it. State will change to ReadyState::kOpen as soon as possible.

This state can be entered both from the ReadyState::kClosed state (when opening of Source is requested) and from the ReadyState::kOpen state (when pipeline can't accept ES data temporarily, for example during Seek).

kOpen 

Player is fully initialized and ElementaryMediaStreamSource is ready to accept ElementaryMediaPacket data from App via ElementaryMediaTrack objects.

When App finishes configuring tracks, it can request entering this state by calling Open(). This state will be entered when possible.

When ReadyState::kOpen is the target state of a source, it's possible the ReadyState::kOpenPending state will be entered temporarily. This happens when ElementaryMediaStreamSource is opening or as a result of some operations (e.g. Seek).

kEnded 

Stream has ended but multimedia pipeline remains initialized. Playback can still be restarted, for example by seek.

This state is entered when App marks active ElementaryMediaTrack objects as ended. The ReadyState::kEnded state reverts to the ReadyState::kOpen state when multimedia pipeline resumes playback (e.g. due to Seek).

Definition at line 218 of file elementary_media_stream_source.h.

Constructor & Destructor Documentation

◆ ElementaryMediaStreamSource() [1/3]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource ( Mode  mode = Mode::kNormal)
explicit

Creates a source with the given mode. Mode cannot be changed during lifetime of the object.

Parameters
[in]modeCreate a source that uses specified playback mode.

◆ ElementaryMediaStreamSource() [2/3]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource ( const ElementaryMediaStreamSource )
delete

◆ ElementaryMediaStreamSource() [3/3]

samsung::wasm::ElementaryMediaStreamSource::ElementaryMediaStreamSource ( ElementaryMediaStreamSource &&  )

◆ ~ElementaryMediaStreamSource()

samsung::wasm::ElementaryMediaStreamSource::~ElementaryMediaStreamSource ( )

Member Function Documentation

◆ AddTrack() [1/2]

Result<ElementaryMediaTrack> samsung::wasm::ElementaryMediaStreamSource::AddTrack ( const ElementaryAudioTrackConfig )

Adds an audio track to the source.

Parameters
[in]configA config describing track.
Returns
Result<ElementaryMediaTrack> with operation_result field set to OperationResult::kSuccess and a valid ElementaryMediaTrack object on success, otherwise a code describing the error.
Remarks
  • Tracks can only be added in kClosed state.
  • Only one track of each type can be held by the source at any given time.
  • Audio parameters cannot be changed during the lifetime of the track.

◆ AddTrack() [2/2]

Result<ElementaryMediaTrack> samsung::wasm::ElementaryMediaStreamSource::AddTrack ( const ElementaryVideoTrackConfig )

Adds a video track to the source.

Parameters
[in]configA config describing track.
Returns
Result<ElementaryMediaTrack> with operation_result field set to OperationResult::kSuccess and a valid ElementaryMediaTrack object on success, otherwise a code describing the error.
Remarks
  • Tracks can only be added in kClosed state.
  • Only one track of each type can be held by the source at any given time.
  • As opposed to audio track, some video parameters can change during playback, notably resolution or framerate, by passing packets with new resolution/fps values.

◆ Close()

Result<void> samsung::wasm::ElementaryMediaStreamSource::Close ( std::function< void(OperationResult)>  on_finished_callback)

Closes the source asynchronously. When the operation is done, the source will be in ReadyState::kClosed state and a callback passed as the argument will be called. During that time it's impossible to request another ready state change.

Parameters
[in]on_finished_callbackA callback notifying end of Close(). The callback receives OperationResult informing of the result of the operation.
Returns
Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

◆ Flush()

Result<void> samsung::wasm::ElementaryMediaStreamSource::Flush ( )

Flushes internal packets' buffers, causing ElementaryMediaTracks belonging to this Source to drop all packets appended so far.

Returns
Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

◆ GetDuration()

Result<Seconds> samsung::wasm::ElementaryMediaStreamSource::GetDuration ( ) const

Returns the duration of the source.

Returns
Result<Seconds> with operation_result field set to OperationResult::kSuccess and a valid Seconds object representing duration of the source on success, otherwise a code describing the error.

◆ GetMode()

Result<Mode> samsung::wasm::ElementaryMediaStreamSource::GetMode ( ) const

Returns the mode of the source, as set during object construction.

Returns
Result<ElementaryMediaStreamSource::Mode> with operation_result field set to OperationResult::kSuccess and a valid Mode representing mode of the source on success, otherwise a code describing the error.

◆ GetReadyState()

Result<ReadyState> samsung::wasm::ElementaryMediaStreamSource::GetReadyState ( ) const

Returns current ReadyState of the source.

Returns
Result<ReadyState> with operation_result field set to OperationResult::kSuccess and a valid ReadyState representing current ready state of the source on success, otherwise a code describing the error.

◆ GetURL()

const char* samsung::wasm::ElementaryMediaStreamSource::GetURL ( ) const

Returns the source's URL received from URL.createObjectURL WebAPI. The URL is bound to the source's lifetime and will be revoked automatically.

Returns
c-style string containing the URL. Ownership is not transferred and the string is valid as long as this object is alive.

◆ IsValid()

bool samsung::wasm::ElementaryMediaStreamSource::IsValid ( ) const

Returns true if source instance is valid. This method should be called after constructor to ensure backend initialized the object properly. If source is invalid all method calls will fail.

Returns
true if source instance is valid, otherwise false.

◆ Open()

Result<void> samsung::wasm::ElementaryMediaStreamSource::Open ( std::function< void(OperationResult)>  on_finished_callback)

Starts an asynchronous opening operation. When the operation is done, the source will be in ReadyState::kOpen state and a callback passed as the argument will be called. During that time it's impossible to request another ready state change.

Parameters
[in]on_finished_callbackA callback notifying end of Open(). The callback receives OperationResult informing of the result of the operation.
Returns
Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.
Remarks
This should be called when all tracks are added and configured, and the application is ready to provide data for the source.

◆ operator=() [1/2]

ElementaryMediaStreamSource& samsung::wasm::ElementaryMediaStreamSource::operator= ( const ElementaryMediaStreamSource )
delete

◆ operator=() [2/2]

ElementaryMediaStreamSource& samsung::wasm::ElementaryMediaStreamSource::operator= ( ElementaryMediaStreamSource &&  )

◆ RemoveTrack()

Result<void> samsung::wasm::ElementaryMediaStreamSource::RemoveTrack ( const ElementaryMediaTrack )

Removes a track from the source. After the operation removed track is still valid and can be re-added to the source again.

Parameters
[in]trackTrack to remove.
Returns
Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.
Remarks
  • Tracks can only be removed in kClosed state.

◆ SetDuration()

Result<void> samsung::wasm::ElementaryMediaStreamSource::SetDuration ( Seconds  new_duration)

Sets the duration of the source. Note that this operation is unavailable in Mode::kLowLatency mode.

Parameters
[in]new_durationDuration to be set.
Returns
Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.

◆ SetListener()

Result<void> samsung::wasm::ElementaryMediaStreamSource::SetListener ( ElementaryMediaStreamSourceListener listener)

Sets a listener to receive updates about EMSS's state changes. Only one listener can be set: setting another clears the previous one. Pass nullptr to reset the listener

Parameters
[in]listenerListener to be set or nullptr to unset the listener.
Warning
The ownership isn't transferred, and, as such, the listener must outlive the source.
Returns
Result<void> with operation_result field set to OperationResult::kSuccess on success, otherwise a code describing the error.
See also
ElementaryMediaStreamSourceListener

Friends And Related Function Documentation

◆ html::HTMLMediaElement

friend class html::HTMLMediaElement
friend

Definition at line 438 of file elementary_media_stream_source.h.


The documentation for this class was generated from the following file: