top

NaCl Player API How-To

This article presents usage of the NaCl Player API on Tizen Samsung Smart TV.

API Overview

NaCl Player offers multiple usage scenarios that cover several playback modes depending on the source of the multimedia data. This includes a playback from a URL multimedia resource and a playback using elementary stream packets with a demuxer provided by the application. Both modes support a playback of a DRM-protected content. The API allows to control the player (MediaPlayer ) and source of its playback (MediaDataSource) which are URLDataSource and ESDataSource. The API also delivers listener classes which allow receiving certain events from the player.

NaCl Player also supports subtitles. These can be:

  • internal - embedded in media content,
  • external - loaded from an external location.

Either way they are parsed by NaCl Player and the application is notified when and what subtitle text should be displayed.

A playback from the URL resource is done by using URLDataSource and is the simpler one out of the two cases. In this mode NaCl Player (thus a platform) handles a content downloading, demuxing and a playback. The simplified use case scenario for URLDataSource is presented below:

  1. Create a media player and custom event listener objects.
  2. Create a URLDataSource object configured to playback a specified URL address.
  3. Wait for NaCl player to signalize that data buffering is complete.
  4. Start a playback.

A playback using elementary stream packets is done by using ESDataSource. In this mode an application that uses NaCl Player takes responsibility for the content downloading, demuxing and optionally custom decryption (custom DRM) to elementary stream packets. NaCl Player handles a playback of elementary streams in this scenario. This mode is suitable for more advanced tasks, as it is more versatile and gives the application wider control over the player. The simplified use case scenario for ESDataSource is presented below:

  1. Create a media player and custom event listener objects.
  2. Create an ESDataSource object configured to handle a number of elementary streams.
  3. [optional] Configure DRM protection system (if a platform DRM is used by the application).
  4. Send elementary stream packets to NaCl Player, until a completion of data buffering is signalized.
  5. Start a playback.
  6. Buffer subsequent elementary stream packets to keep the playback going.

Above usage scenarios list only generalized key actions that the application must execute to playback media content. More detailed use case scenarios (which point to specific parts of the NaCl Player API) can be found in the Use Cases article.

For more details about the API and its classes and methods please refer to the NaCl Player documentation.

The NaCl Player API description and usage code snippets can be found in the following sections.

Tizen privileges

An application using NaCl Player may be required to specify particular Tizen privileges in the Setting config.xml for TV file to achieve a playback.

Usage of DRM systems with NaCl Player requires specific privileges in the configuration file. Additionally, some privileges not required directly by NaCl Player are likely to be required by an application in most common player-related usage scenarios.

A list of privileges related to NaCl Player can be found below.

Tizen Privilege Description
http://developer.samsung.com/privilege/drmplay Required for a PlayReady content playback.
http://tizen.org/privilege/internet Required for an application to access content on the Internet.

Initialization

There are two main classes in NaCl Player that must be initialized to achieve a playback:

  • MediaPlayer - a representation of the platform player which provides a functionality to control the player.
  • MediaDataSource:
    • an abstract representation of a data source. MediaPlayer receives data for playing from the data source.
    • the application uses a specialization of this class (i.e. URLDataSource or ESDataSource). Each specialization has a different initialization procedure (see the MediaDataSource section for details).

Creation of MediaPlayer is parameter-free and can be done as presented below:

// a managing class holds a pointer to the player
std::shared_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
 
// create the player object
player_ = std::make_shared<Samsung::NaClPlayer::MediaPlayer>();

The initialized MediaDataSource object has to be attached to the MediaPlayer object:

// a managing class holds a pointer to the data source
std::shared_ptr<Samsung::NaClPlayer::MediaDataSource> data_source_ =
  std::make_shared</* MediaDataSource-derived type */>(/* args... */);
 
// an application-defined function initializes the MediaDataSource object
InitializeDataSource(data_source_);
 
// attach the initialized data source
player_->AttachDataSource(*data_source_);

The prepared player can begin the playback but it has no way to inform an application about occurring events (e.g. information that content is loaded and play can be proceeded). In order to inform the application about certain events MediaPlayer uses listeners. The application needs to override a listener class and register it in MediaPlayer to receive events. More information about listeners can be obtained in the Listeners subsection.

Listeners

Listeners are created in order to notify the application about certain player-related events. To receive events from the player the application should:

  1. specialize one of the Listener classes,
  2. register the specialized class in the MediaPlayer object,
  3. react to the virtual method associated with the event. The method that will be called by the MediaPlayer object on the registered Listener interface.

Important information about listeners in NaCl Player:

  • Listeners don’t provide a default event handler implementation. Events must be handled specified by the application or they are lost.
  • Only one Listener of each type can be registered. Registering a new listener automatically unregisters the previous one.
  • Listeners doesn’t have to be unregistered before destruction of the MediaPlayer object.

Listeners available in MediaPlayer , including an implementation stub, purpose and usage context, are described below.

Registering and deregistering listeners

The below code snippets use …Listener as the listener name. It should be replaced with one of the listeners described in the following subsections.

A listener object must be registered in the MediaPlayer object before it can receive events.

// a managing class holds pointers
std::shared_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
std::shared_ptr<My...Listener> events_listener_;
 
// create a listener and register it in the player
events_listener_ = std::make_shared<My...Listener>();
player_->Set...Listener(events_listener_);

The listener can be deregistered from the MediaPlayer object:

// the application can resign from receiving events by deregistering a listener
// of the given type from the player
player_->Set...Listener(nullptr);
Note

The MediaPlayer object holds a shared reference to the listener. The application code doesn’t need to store this reference and doesn’t need to deregister the listener from the player manually. The listener will be dereferenced automatically when the MediaPlayer object is destroyed.

MediaEventsListener

Overriding a MediaEventsListener class allows the application to receive playback-related notifications from NaCl Player.

A sample listener can be implemented as follows:

class MyMediaEventsListener : public Samsung::NaClPlayer::MediaEventsListener {
 public:
  // This event is delivered periodically during a playback.
  // @param time A current playback time.
  void OnTimeUpdate(Samsung::NaClPlayer::TimeTicks time) override {
    // Handle the event...
  }

  // This event is delivered when a playback has finished.
  void OnEnded() override {
    // Handle the event...
  }

  // This event is delivered when a player error occured.
  void OnError(Samsung::NaClPlayer::MediaPlayerError) override {
    // Handle the event...
  }
};

A listener object can be registered in the player object using the following method:

player_->SetMediaEventsListener(media_event_listener_);

BufferingListener

Overriding a BufferingListener class allows the application to receive buffering-related notifications from NaCl Player.

A sample listener can be implemented as follows:

class MyBufferingListener : public Samsung::NaClPlayer::BufferingListener {
 public:
  // This event is delivered when NaCl player starts buffering data. It is
  // delivered in both URLDataSource and ESDataSource scenarios.
  void OnBufferingStart() override {
    // Handle the event...
  }
 
  // This event indicates a percentage of data buffered so far. It is delivered
  // only in the URLDataSource scenario.
  void OnBufferingProgress(uint32_t percent) override {
    // Handle the event...
  }
 
  // This event is delivered when NaCl player finishes buffering data. It is
  // delivered in both URLDataSource and ESDataSource scenarios.
  void OnBufferingComplete() override {
    // Handle the event...
  }
};

Specific meaning of particular events varies slightly depending on the used data source. Details are provided below.

The events’ details for the URLDataSource scenario:

  • The OnBufferingStart event is delivered when NaCl Player internal implementation starts buffering data.
    Most notably, this event is delivered when a data source is attached to the MediaPlayer object and during seek. In these cases playback is not possible until the OnBufferingComplete event occurs.
  • The OnBufferingProgress event can be used to update UI with information regarding the percentage of the buffering completed so far.
  • The OnBufferingComplete event is delivered when NaCl Player internal implementation finishes buffering data.
    This means a playback can be started for a newly attached data source (the MediaPlayer::Play() method can be called) or the playback can continue after a seek operation (no action required).

The events’ details for the ESDataSource scenario:

  • The OnBufferingStart event is delivered when the NaCl Player data buffer is filled and the application should start sending elementary stream packets to NaCl Player.
    Most notably, this event is delivered when a data source is attached to the MediaPlayer object. Playback is not possible until the buffer is filled and the OnBufferingComplete event occurs.
  • The OnBufferingComplete event is delivered when the NaCl Player data buffer has received enough data.
    This means a playback can be started for a newly attached data source (the MediaPlayer::Play() method can be called).
  • The OnBufferingProgress event is not delivered in this scenario.
Note

A MediaPlayer::Play() method call cannot be issued when buffering is in progress.

A listener object can be registered in the player object using the following method:

player_->SetBufferingListener(buffering_listener_);

DRMListener

Overriding a DRMListener class is required in a DRM-protected playback scenario. The application will receive protection system requests and notifications through DRMListener. Requests must be handled to properly decrypt the protected content.

A sample listener can be implemented as follows:

class MyDRMListener : public Samsung::NaClPlayer::DRMListener {
 public:
  DRMListener(const std::shared_ptr<Samsung::NaClPlayer::MediaPlayer>& player)
      : player_(player) {
  }
 
  // Called when DRM init data is loaded. This event is informative.
  void OnInitdataLoaded(
      Samsung::NaClPlayer::DRMType drm_type, uint32_t init_data_size,
      const void *init_data) override {
  }
 
  // A license is required to playback protected content.
  // @param request Contains a challenge data that should be used to obtain a
  //   license.
  void OnLicenseRequest(uint32_t request_size, const void *request) override {
    // 1. connect to a license server and download the license:
    std::string license;
    // ...fill the license...
 
    // 2. install the obtained license:
    if (auto player = player_.lock()) {
      player->SetDRMSpecificData(
          DRMType_*, DRMOperation_InstallLicense,
          license.size(), license.data());
    }
  }
 private:
  std::weak_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
};

A license request will be generated when there is a protected content that needs to be decrypted and no license is already installed that can decrypt the content. If there are multiple tracks protected by the same license, it should be installed only once. For more details regarding using DRMs with NaCl Player, please refer to the DRM section.

A listener object can be registered in the player object using the following method:

player_->SetDRMListener(drm_listener_);

SubtitleListener

Overriding a SubtitleListener class allows the application to receive subtitle update notifications from NaCl Player.

NaCl Player acts as a parser of subtitle files that can be attached as a track to media content. When a playback is in progress, NaCl Player will use a SubtitleListener-derived object to notify the application when a subtitle text should be displayed. The listener will be notified only about subtitles associated with the currently selected text track.

Important

Please note that NaCl Player does not display subtitles itself!
Subtitles must be displayed by the application basing on the subtitle events.

A sample listener can be implemented as follows:

class SubtitleListener : public Samsung::NaClPlayer::SubtitleListener {
 public:
  // A subtitle string "text" should be displayed for a "duration" seconds.
  void OnShowSubtitle(
      Samsung::NaClPlayer::TimeDelta duration, const char *text) override {
    // Show the subtitle...
  }
};

A listener object can be registered in the player object using the following method:

player_->SetSubtitleListener(subtitle_listener_);

MediaDataSource

URLDataSource

URLDataSource allows the playback from the given URL address of media content.

To create a URLDataSource object an application needs to call it’s constructor and attach it to the MediaPlayer object. See the Initialization section for a MediaPlayer object creation.

// a managing class holds pointers
std::shared_ptr<Samsung::NaClPlayer::MediaDataSource> data_source_;
std::shared_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
 
// create a media data source for the given URL address and attach it to the player
data_source_ = std::make_shared<Samsung::NaClPlayer::URLDataSource>(media_content_url);
player_->AttachDataSource(*data_source_);

ESDataSource

ESDataSource allows to playback from the elementary stream packets demuxed from the media content.

To create and fully use an ESDataSource object an application needs to:

  1. Call the ESDataSource constructor.
  2. Create an ElementaryStreamListener-derived object.
  3. Add ElementaryStreams with listeners to the media data source.
  4. Configure each newly added ElementaryStreams
  5. Attach ESDataSource to the player.

Creating ESDataSource

The ESDataSource must be instantiated and attached to the MediaPlayer object.

// a managing class holds pointers
std::shared_ptr<Samsung::NaClPlayer::MediaDataSource>  es_data_source_;
std::shared_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
std::shared_ptr<Samsung::NaClPlayer::AudioElementaryStream> audio_stream_;
std::shared_ptr<Samsung::NaClPlayer::VideoElementaryStream> video_stream_;
std::shared_ptr<ESListener> audio_listener_;
std::shared_ptr<ESListener> video_listener_;
 
// create media data source
es_data_source_ = std::make_shared<Samsung::NaClPlayer::ESDataSource>();

Definitions of the objects used in the code snippet are provided below.

ElementaryStreams objects need to be created before a data source can be attached to the player. Each ElementaryStreams requires ElementaryStreams to be registered to a stream, as described below.

Implementing ElementaryStreamListener

The ElementaryStreamListener must be instantiated and registered in the newly created ElementaryStream object. This is done while adding the ElementaryStream object to the ESDataSource object.

Definition of the sample class specializing ElementaryStreamListener is provided below

class ESListener : public Samsung::NaClPlayer::ElementaryStreamListener {
 public:
  // This event is delivered when NaCl Player needs ES packets.
  // @param bytes_max Tells how many bytes the player can accept.
  void OnNeedData(int32_t bytes_max) {
    // Sending packets...
  }

  // This event is delivered when NaCl Player ES packets buffer is full.
  // Appending ES packets should be stopped after this event.
  void OnEnoughData() {
    // Stop sending packets...
  }

  // This event is delivered when NaCl Player performed seek operation.
  // @param new_position After this event ES packets should be delivered
  //   according to the given time.
  void OnSeekData(Samsung::NaClPlayer::TimeTicks new_position) {
    // Adjust media content position from which packets are being sent...
  }
}

Adding Elementary Streams

Each elementary stream must be added to the ESDataSourceobject and associated with an ElementaryStreamListener object, as shown below.

// add VideoElementaryStream
video_listener_ = std::make_shared<ESListener>();
video_stream_ = std::make_shared<Samsung::NaClPlayer::VideoElementaryStream>();
es_data_source_->AddStream(*video_stream, video_listener_);
 
// add AudioElementaryStream
audio_listener_ = std::make_shared<ESListener>();
audio_stream_ = std::make_shared<Samsung::NaClPlayer::AudioElementaryStream>();
es_data_source_->AddStream(*audio_stream, audio_listener_);

Each ElementaryStream object needs to be configured. The required configuration vary depending on the stream type and must be concluded with a call to the ElementaryStream::InitializeDone() method. The configuration procedure is described in details in the following subsections.

Configuring an audio stream

To configure AudioElementaryStream all the below fields should be set. Please refer to AudioConfig for the parameters’ description. After the configuration is done, a call to the ElementaryStream::InitializeDone() method is required.

AudioConfig audio_config;
// fill audio_config with proper information
 
audio_stream_->SetAudioCodecType(audio_config.codec_type);
audio_stream_->SetAudioCodecProfile(audio_config.codec_profile);
audio_stream_->SetSampleFormat(audio_config.sample_format);
audio_stream_->SetChannelLayout(audio_config.channel_layout);
audio_stream_->SetBitsPerChannel(audio_config.bits_per_channel);
audio_stream_->SetSamplesPerSecond(audio_config.samples_per_second);
audio_stream_->SetCodecExtraData(audio_config.extra_data.size(),
                                 &audio_config.extra_data.front());
audio_stream_->InitializeDone();

Configuring a video stream

To configure VideoElementaryStream all the below fields should be set. Please refer to VideoConfig for the parameters’ description. After the configuration is done, a call to the ElementaryStream::InitializeDone() method is required.

VideoConfig video_config;
// fill video_config with proper information
 
video_stream_->SetVideoCodecType(video_config.codec_type);
video_stream_->SetVideoCodecProfile(video_config.codec_profile);
video_stream_->SetVideoFrameFormat(video_config.frame_format);
video_stream_->SetVideoFrameSize(video_config.size);
video_stream_->SetFrameRate(video_config.frame_rate);
video_stream_->SetCodecExtraData(video_config.extra_data.size(),
                                 &video_config.extra_data.front());
video_stream_->InitializeDone();

Attaching the media data source and appending packets

When ElementaryStream-derived objects are added to the data source and configured, the ESDataSource object can be attached to the MediaPlayer object. From that moment on, the application can start appending ESPackets and receive Elementary Stream events.

player_->AttachDataSource(*es_data_source_);
 
// after receiving audio and video NeedData events, the application
// can append packets
audio_stream_->AppendPacket(es_pkt);
video_stream_->AppendPacket(es_pkt);

A playback can be started after the MediaPlayer object signalizes readiness by the BufferingListener::OnBufferingComplete() call.

DRM

NaCl Player can handle multimedia content protected by DRM systems. Details regarding handling DRM content vary depending on the data source used.

Important

In order to enable a PlayReady-protected content playback an application must specify the http://developer.samsung.com/privilege/drmplay privilege in it’s configuration file.

DRM in URLDataSource

In the URLDataSource playback scenario a DRM is configured automatically by the platform. This is done basing on the DRM configuration stored in a file the URL address points to (e.g. a DASH manifest file containing protected tracks information and a DRM protection configuration).

DRM in ESDataSource

Preparing for DRM-protected content playback

In the ESDataSource playback scenario a DRM must be configured by the application. The DRM configuration must be done during a stream configuration procedure (see the ESDataSourcesubsection for details). This can be summarized with the following steps:

  1. Create an object that derives from the DRMListener class and register it in NaCl Player using the MediaPlayer::SetDRMListener() method.
  2. Call the ElementaryStream::SetDRMInitData() method during the configuration of either an AudioElementaryStream or a VideoElementaryStream object.
  3. React to the DRMListener::OnLicenseRequest() event by installing a proper license by the means of MediaPlayer::SetDRMSpecificData(DRMType_*, DRMOperation_InstallLicense, drm_data_size, drm_data).

Alternatively, a license may be pre-installed by the application without waiting for the OnLicenseRequest call. This case will not be discussed here in detail, however it might be handy in some usage scenarios. For a pre-installed license, only step (2) from the above list is required.

A DRM Listener

A DRMListener main role is to listen for the OnLicenseRequest event and react to it by installing a license that allows a playback of the DRM-protected content. A listener can be declared as follows:

class MyDRMListener : public Samsung::NaClPlayer::DRMListener {
 public:
  DRMListener(const std::shared_ptr<Samsung::NaClPlayer::MediaPlayer>& player)
      : player_(player) {
  }
  
  // Mainly an informative role, informs that DRM protection data was loaded.
  void OnInitdataLoaded(
      Samsung::NaClPlayer::DRMType drm_type, uint32_t init_data_size,
      const void *init_data) override;
  
  // Signalizes that a license is required to playback the current content.
  void OnLicenseRequest(uint32_t request_size, const void *request) override;
  
 private:
  // OnLicenseRequest will call this method when it downloads a license.
  void InstallLicense(std:: string license);
 
  std::uint32_tweak_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
};

DRMListener must be instantiated and registered in the MediaPlayer object. This should be done right after the MediaPlayer object is created.

// a managing class holds pointers
std::shared_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
std::shared_ptr<MyDRMListener> drm_listener_;
 
// create a listener and register it in the player
drm_listener_ = std::make_shared<MyDRMListener>(player_);
player_->SetDRMListener(drm_listener_);

For more details about NaCl Player listeners please refer to the Listeners section.

Elementary stream DRM information

A stream should be configured to use DRM at the same time when other stream parameters (such as resolution, sampling frequency, codec, etc.) are configured. This is done right after an Elementary Streamobject is added to the ESDataSource object.

// a managing class holds pointers
std::shared_ptr<Samsung::NaClPlayer::MediaPlayer> player_;
std::shared_ptr<Samsung::NaClPlayer::ESDataSource> es_data_source_;
// ...
std::shared_ptr<Samsung::NaClPlayer::ElementaryStreamListener> video_listener_;
 
// some method that does ESDataSource configuration
void PlayerManager::ConfigureStreams(...) {
  // ...
  // add and configure a video elementary stream
  video_listener_ = std::make_shared<ESListener>();
  Samsung::NaClPlayer::VideoElementaryStream video_stream;
  es_data_source_->AddStream(video_stream, video_listener_);
  // configure VideoStream parameters: codec, frame format, etc.
  // ...
  video_stream.SetDRMInitData(encryption_type, init_data_size, init_data_ptr);
  video_stream.InitializeDone();
 
  // Repeat the above procedure for an audio elementary stream...
  // ...
}

For more details about the elementary stream configuration please refer to the MediaDataSource section.

The OnLicenseRequest event

If a license required for a playback of the current DRM-protected content is not available, DRMListener will be notified with the OnLicenceRequest event. The application should download the license from a license server and pass it to NaCl Player.

// A method that downloads a license asynchronously and then calls a
// license_downloaded_cb functor.
void DownloadLicense(
    std::string license_server_url, uint32_t request_size, const void *request,
    std::function<std::string> license_downloaded_cb);
 
void MyDRMListener::OnLicenseRequest(
    uint32_t request_size, const void *request) {
  // This method must download a proper license from the license server.
  // - License server URL address is known by the application
  // - "request" and "request_size" describe challenge data required to
  //   obtain a license.
  // For the best application responsiveness, a license download should be
  // done in the asynchronous manner:
  DownloadLicense("a license server URL", request_size, request, std::bind(
      &MyDRMListener::InstallLicense, this, std::placeholders::_1));
}
 
void MyDRMListener::InstallLicense(std::string license) {
  int32_t result = player_->SetDRMSpecificData(
      Samsung::NaClPlayer::DRMType_*,
      Samsung::NaClPlayer::DRMOperation_InstallLicense,
      license.size(), license.data());
  if (result < Samsung::NaClPlayer::ErrorCodes::Success) {
    // ...handle the error...
  }
}
Note

A license download method is out of the scope of this tutorial. However, please note that the download itself as well as license installation should be done asynchronously. Such approach is beneficial for the application responsiveness!

Important

A license server URL address is known by the application. Its origin depends on the usage scenario (e.g. a DASH with CENC stores the license server URL address in a manifest file).

Interacting with a DRM system

NaCl Player allows interaction with a DRM system. Various DRM-related operations can be issued using the MediaPlayer::SetDRMSpecificData() method.

Please refer to the Doxygen documentation for the list of all supported operations.

Important

DRM operation availability depends on the DRM system used. Some DRM systems may support only a subset of the NaCl Player DRM operations.