ImportantDue to NaCl deprecation by the Chromium project, Tizen TV will continue its support for NaCl only until 2021-year products. Meanwhile, Tizen TV will start focusing on high-performance, cross-browser WebAssembly from 2020-year products.
This topic describes in detail how to develop a media player application using the NaCl Player API. The topic uses the Native Player sample application as an example.
The Native Player sample application implements some of the main NaCl (Native Client) Player API use cases. The sample demonstrates the media data flow: data download, the demuxing process, and displaying the content on the TV.
The sample application is designed in a modular and extensible way. Its source code is released under the permissive MIT license, allowing you to reuse and customize the code to meet your own needs. You can access the latest Native Player source code from the Native Player GitHub repository.
ImportantThis application uses API that is now deprecated. Use raw pointers instead of shared_ptrs.
Native Player Architecture
The NaCl Player API is used by the NaCl module. All player logic is implemented in the NaCl module, including user authorization, downloading and demuxing (parsing) media content, decrypting DRM-protected content, and passing demuxed elementary stream packets to the NaCl Player.
The following figure shows the main structural layers, the components within the layers, and the data flow of a typical NaCl Player application:
External data for the NaCl Player application is stored in:
Media server, which stores the media content.
DRM server, which provides information needed to decrypt DRM-protected content.
Application is installed and run by the user. It consists of 2 components:
HTML5 component, which implements the UI and handles user interaction. The component has been omitted from the figure for clarity.
Native Client module component, which is responsible for the NaCl Player application logic:
Communicator handles communication with the HTML5 component, authorizing the user's access to the requested content.
Controller interfaces with the NaCl Player by, for example, requesting data download and appending media packets to the NaCl Player.
Data provider downloads external data in chunks from the media server.
Data demuxer processes media content and passes the packets to the controller:
Downloaded content is demuxed into elementary stream packets. Clear content is not encrypted and can be parsed directly into elementary stream packets.
DRM-protected content is decrypted (custom DRM by the Native Client module, and platform-supported DRM by the platform).
Platform
The platform contains the implementation of the player, which is responsible for performing the functions called by the application, such as decoding packets, playing audio, and rendering video on the screen.
Playing from URL Data Sources
The sample application supports MPEG4 media playback from a URL data source, with external or internal subtitles. You can show or hide external subtitles, and you can show, hide, and select internal subtitle tracks.
In this scenario, the platform is responsible for downloading and demuxing content. The application is responsible only for playback control, using the controller and communicator components.
In the "inc/player/url_player/url_player_controller.h" file, manage playback from the URL data source using the UrlPlayerController class. It aggregates all the necessary objects:
Define the listeners in the "src/player/player/player_listeners.cc" file:
Implement the SubtitleListener class to handle subtitle events. It forwards the received subtitle text and duration to the JavaScript application component, which is responsible for displaying the subtitle:
Implement the MediaEventsListener class to handle player and playback events.
The OnBufferingComplete() event notifies the JavaScript application component that the NaCl Player has buffered enough data to start playback:
void MediaBufferingListener::OnBufferingComplete() {
LOGM("Event: Buffering complete! Now you may play.");
if (auto message_sender = message_sender_.lock()) {
message_sender->BufferingCompleted();
}
// ...
}
In the "src/player/url_player/url_player_controller.cc" file, initialize and register the listeners to a MediaPlayer object:
void UrlPlayerController::InitPlayer(const std::string& url,
const std::string& subtitle,
const std::string& encoding) {
// For external subtitles, the subtitle argument points to the location of the subtitle file
// For internal subtitles, the subtitle argument must be an empty string
// ...
player_ = make_shared<MediaPlayer>();
// Initialize listeners and register them to the player
listeners_.player_listener =
make_shared<MediaPlayerListener>(message_sender_);
listeners_.buffering_listener =
make_shared<MediaBufferingListener>(message_sender_);
listeners_.subtitle_listener =
make_shared<SubtitleListener>(message_sender_);
player_->SetMediaEventsListener(listeners_.player_listener);
player_->SetBufferingListener(listeners_.buffering_listener);
player_->SetSubtitleListener(listeners_.subtitle_listener);
Register external subtitles to the player.
To check whether an external subtitle URL has been defined, and pass the address to the player:
// Register external subtitles source, if defined
if (!subtitle.empty()) {
text_track_ = MakeUnique<TextTrackInfo>();
int32_t ret = player_->AddExternalSubtitles(subtitle, encoding, *text_track_);
// ...
}
When all the player objects are set up, initialize and construct a URLDataSource object to handle a given URL address, and attach it to the player:
Retrieve information about the available internal and external text tracks using the UrlPlayerController::PostTextTrackInfo() function, and send the track list to the HTML5 application component for showing in the UI:
void UrlPlayerController::PostTextTrackInfo() {
int32_t ret = player_->GetTextTracksList(text_track_list_);
if (ret == ErrorCodes::Success) {
LOG("GetTextTrackInfo called successfully");
message_sender_->SetTextTracks(text_track_list_);
} else {
LOG_ERROR("GetTextTrackInfo call failed, code: %d", ret);
}
}
When the player receives the OnBufferingComplete() event, start playback:
During playback, the application receives SubtitleListener::OnShowSubtitle() events containing subtitle text and timing information. To display the subtitles, pass the subtitle event data to the HTML5 application component.
Playing from Elementary Stream Data Sources
The sample application supports MPEG-DASH media playback from elementary stream data sources. It demonstrates the following use cases:
Playing clear MPEG-DASH content with external subtitles
Playing PlayReady DRM-encrypted MPEG-DASH content
In these scenarios, the application is responsible for downloading and demuxing content, and controlling the media player.
Data Provider
To play MPEG-DASH content, the DashManifest class uses the libdash library to parse a DASH manifest file and download the appropriate media file or media chunk.
The DashManifest class is also responsible for extracting Content Protection DRM initialization information included in the DASH manifest, through the ContentProtectionVisitor class.
In the sample application, the "src/dash" directory contains the data provider implementation.
Data Demuxer
In the sample application "src/demuxer" directory, the FFMpegDemuxer class implements the StreamDemuxer interface. The StreamDemuxer interface supports clear and encrypted elementary stream content. It demuxes the media content into elementary stream packets.
The FFMpegDemuxer class uses the ffmpeg library. It extracts the elementary stream configuration and protection data, such as PSSH box and DRM initialization data, from the media content.
To implement the demuxer:
When data must be parsed, call the StreamDemuxer::Parse() function:
NoteThe parsed packets are delivered to the application using the es_data_callback_() function, which was registered during the StreamDemuxer class initialization.
Convert the FFMpeg elementary packet (AVPacket) into an elementary stream packet:
unique_ptr<ElementaryStreamPacket> FFMpegDemuxer::MakeESPacketFromAVPacket(
AVPacket* pkt) {
auto es_packet = MakeUnique<ElementaryStreamPacket>(pkt->data, pkt->size);
AVStream* s = format_context_->streams[pkt->stream_index];
// Set ES packet information
es_packet->SetPts(ToTimeTicks(pkt->pts, s->time_base) + timestamp_);
es_packet->SetDts(ToTimeTicks(pkt->dts, s->time_base) + timestamp_);
es_packet->SetDuration(ToTimeTicks(pkt->duration, s->time_base));
es_packet->SetKeyFrame(pkt->flags == 1);
AVEncInfo* enc_info = reinterpret_cast<AVEncInfo*>(
av_packet_get_side_data(pkt, AV_PKT_DATA_ENCRYPT_INFO, NULL));
if (!enc_info) return es_packet;
// Packet encrypted
es_packet->SetKeyId(enc_info->kid, kKidLength);
es_packet->SetIv(enc_info->iv, enc_info->iv_size);
for (uint32_t i = 0; i < enc_info->subsample_count; ++i) {
es_packet->AddSubsample(enc_info->subsamples[i].bytes_of_clear_data,
enc_info->subsamples[i].bytes_of_enc_data);
}
return es_packet;
}
MPEG-DASH Content Playback
To implement MPEG-DASH content playback:
In the "inc/player/es_dash_player/es_dash_player_controller.h" file, manage playback from the elementary stream data source using the ESDashPlayerController class. It aggregates all the necessary objects:
In the "src/player/es_dash_player/es_dash_player_controller.cc" file, initialize the DASH parser.
The application must handle DRM configuration. Because DRM-specific data is not part of the DASH standard, it cannot be parsed by the libdash library. The DRM-specific data is stored in the ContentProtection DASH manifest section. The ContentProtectionVisitor class passes the DRM-specific data to the DashManifest::ParseMPD() function.
In the "src/player/es_dash_player/stream_manager.cc" file, initialize the StreamManager object.
It manages elementary streams by downloading the appropriate representation, demuxing it into elementary stream packets, and sending the packets to the player for playback.
bool StreamManager::Impl::Initialize(
unique_ptr<MediaSegmentSequence> segment_sequence,
shared_ptr<ESDataSource> es_data_source,
std::function<void(StreamType)> stream_configured_callback,
DRMType drm_type,
std::shared_ptr<ElementaryStreamListener> listener) {
// ...
// dd a stream to ESDataSource
if (stream_type_ == StreamType::Video) {
auto video_stream = make_shared<VideoElementaryStream>();
result = es_data_source->AddStream(*video_stream, listener);
elementary_stream_ =
std::static_pointer_cast<ElementaryStream>(video_stream);
} else if (stream_type_ == StreamType::Audio) {
auto audio_stream = make_shared<AudioElementaryStream>();
result = es_data_source->AddStream(*audio_stream, listener);
elementary_stream_ =
std::static_pointer_cast<ElementaryStream>(audio_stream);
}
// ...
// Initialize stream demuxer - create and set stream configs listeners
if (!InitParser()) {
LOG_ERROR("Failed to initialize parser or config listeners");
return false;
}
return ParseInitSegment();
}
Manage Your Cookies
We use cookies to improve your experience on our website and to show you relevant
advertising. Manage you settings for our cookies below.
Essential Cookies
These cookies are essential as they enable you to move around the website. This
category cannot be disabled.
Company
Domain
Samsung Electronics
.samsungdeveloperconference.com
Analytical/Performance Cookies
These cookies collect information about how you use our website. for example which
pages you visit most often. All information these cookies collect is used to improve
how the website works.
Company
Domain
LinkedIn
.linkedin.com
Meta (formerly Facebook)
.samsungdeveloperconference.com
Google Inc.
.samsungdeveloperconference.com
Functionality Cookies
These cookies allow our website to remember choices you make (such as your user name, language or the region your are in) and
tailor the website to provide enhanced features and content for you.
Company
Domain
LinkedIn
.ads.linkedin.com, .linkedin.com
Advertising Cookies
These cookies gather information about your browser habits. They remember that
you've visited our website and share this information with other organizations such
as advertisers.
Company
Domain
LinkedIn
.linkedin.com
Meta (formerly Facebook)
.samsungdeveloperconference.com
Google Inc.
.samsungdeveloperconference.com
Preferences Submitted
You have successfully updated your cookie preferences.