Playback Using AVPlay

This topic describes how your application can play media content using the AVPlay API.

Related Info

AVPlay API

In general, common media formats can be played using the HTML5 video and audio elements. To play media content with features not supported in HTML5, such as adaptive streaming, additional subtitle formats, and 4K UHD video, use the Samsung Product AVPlay API.

For information on the supported media containers, streaming format and DRM combinations, and network protocols, see the General Specifications.

AVPlay Life-cycle

An AVPlay instance has various states, which determine the actions you can take. You must also pay attention to the state limitations for the AVPlay API methods.

The following table lists the possible AVPlay instance states.

State	Description
NONE	Before the instance is created, or after it is removed with the `close()` method.
IDLE	After the instance is created by opening the media with the open() method.
READY	During and after media buffering. In this state you can, for example, use the `getDuration()` method to retrieve the duration of the media.
PLAYING	When playback is in progress. In this state you can, for example, use the `getCurrentTime()` method to retrieve the current time of the media.
PAUSED	When playback is paused with the `pause()` method.

Table 1. AVPlay instance states

The following figure shows the state changes and life-cycle of an AVPlay instance, and the playback control operations that drive state transitions.

Figure 1. AVPlay instance life-cycle

Prerequisites

To enable your application to use the functionalities of the AVPlay API:

To use the AVPlay API, include the WebAPI library in the “index.html” file:

<head>
...
<script type="text/javascript" src="$WEBAPIS/webapis/webapis.js"></script>
...
</head>

To access external network resources through the application, set the appropriate content security policy (CSP) in the “config.xml” file.
Important Since 2015 models, the http://developer.samsung.com/privilege/avplay privilege is not used in TV Web applications.

Playing Media

To play media using the AVPlay API:

Create an element for media playback, using an object element with the "application/avplayer" type attribute:

var objElem = document.createElement('object');
objElem.type = 'application/avplayer';

/*
//Adjust the size and position of the media display area 
//by changing the CSS style attribute
objElem.style.left = 100 + 'px';
objElem.style.top = 200 + 'px';
objElem.style.width = 600 + 'px';
objElem.style.height = 400 + 'px';
*/

//Append the object element to your document
document.body.appendChild(objElem);

Open a media file using the open() method.

The AVPlay API supports absolute local paths and remote network URIs. Relative local paths are not supported.
```
webapis.avplay.open('yourMediaURI');
```
After the media file is successfully opened, the AVPlay instance is created in the IDLE state.

Define the event handlers using the setListener() method. The following table lists the available event handlers.

Event Handler	Description
`onbufferingstart()`	Media data buffering has started.
`onbufferingprogress()`	Media data buffering progress percentage.
`onbufferingcomplete()`	Media data buffering has completed.
`onstreamcompleted()`	Media playback has completed.
`oncurrentplaytime()`	Current playback time in the PLAYING state, in milliseconds.
`onerror()`	An error has occurred during media playback.
`onevent()`	Some other event is received by the player.
`onsubtitlechange()`	Subtitle text has changed.
`ondrmevent()`	DRM information is detected by the player.

Table 2. Event methods

var listener = {
onbufferingstart: function() {
	console.log("Buffering start.");
},

onbufferingprogress: function(percent) {
	console.log("Buffering progress data : " + percent);
},

onbufferingcomplete: function() {
	console.log("Buffering complete.");
},
onstreamcompleted: function() {
	console.log("Stream Completed");
	webapis.avplay.stop();
},

oncurrentplaytime: function(currentTime) {
	console.log("Current playtime: " + currentTime);
},

onerror: function(eventType) {
	console.log("event type error : " + eventType);
},

onevent: function(eventType, eventData) {
	console.log("event type: " + eventType + ", data: " + eventData);
},

onsubtitlechange: function(duration, text, data3, data4) {
	console.log("subtitleText: " + text);
},
ondrmevent: function(drmEvent, drmData) {
	console.log("DRM callback: " + drmEvent + ", data: " + drmData);
}
};

webapis.avplay.setListener(listener);

For video media, set the media display area using the setDisplayRect() method.

The setDisplayRect() method takes 4 parameters: the position from the left side of the screen, the position from the top of the screen, the width, and the height. For the purpose of the setDisplayRect() method parameters, the TV screen resolution is always treated as 1920x1080 px, regardless of the application resolution or viewport size.

If the application resolution is 1920x1080 px, simply set the same position values as defined in the object element style attribute. Otherwise, you must calculate the values for the setDisplayRect() method parameters.
```
// For example,  video positon is 
// left: 100 px / top: 200 px / width: 600 px / height: 400 px

// Case 1: Application resolution 1920x1080 px
webapis.avplay.setDisplayRect(100,200,600,400);

// Case 2: Other application resolution

// Base resolution of avplay
var avplayBaseWidth = 1920;

// Calculate ratio to base resolution
var ratio = avplayBaseWidth / window.document.documentElement.clientWidth;

// Convert rectangle to base resolution
var newLeft = 100 * ratio;
var newTop = 200 * ratio;
var newWidth = 600 * ratio;
var newHeight = 400 * ratio;

webapis.avplay.setDisplayRect(newLeft,newTop,newWidth,newHeight); 
```
Note : If you use the setDisplayRect() method to change the media display area size or position during media playback, the object element style attribute must also be changed correspondingly.

By default, video is displayed full screen within the media display area. To fit the video to the media display area:
```
webapis.avplay.setDisplayMethod('PLAYER_DISPLAY_MODE_LETTER_BOX')
```
Prepare the media for playback, synchronously or asynchronously:
- To prepare the media synchronously, use the prepare() method.
  
  Because preparation can involve fetching and decoding media data, the prepare() method can take a long time to execute. Do not call it from your application’s UI thread, as it causes the UI to hang until execution is finished.
```
webapis.avplay.prepare();
```
- To prepare the media asynchronously, use the prepareAsync() method.
  
  This method starts preparing in the background and returns immediately. When the media is finished preparing, the successCallback() method is invoked.
```
var successCallback = function() {
console.log('The media has finished preparing');
}

var errorCallback = function() {
console.log('The media has failed to prepare');
}
webapis.avplay.prepareAsync(successCallback,errorCallback);
```
When the AVPlay instance starts preparing the media, the onbufferingstart() event handler is invoked, and the AVPlay instance enters the READY state.
To manage starting and stopping playback:
1. Start playback using the play() method:
```
webapis.avplay.play(); 
```
  When playback starts, the oncurrentplaytime() event handler is invoked, and the AVPlay instance enters the PLAYING state.
2. Pause playback using the pause() method:
```
webapis.avplay.pause();
```
The AVPlay instance enters the PAUSED state. Playback can be resumed using the play() method.
1. When playback is complete, stop the player:
```
webapis.avplay.stop();
```
The AVPlay instance enters the IDLE state. The instance retains its configuration, such as the media URI, display area, and event methods. To replay the media, you can simply call the prepare() and play() methods again.

Note : Failure to stop playback after the media has finished playing causes the media player window to show the final frame continuously.
To remove the AVPlay instance, call the close() method.

Managing Media Playback

Using the AVPlay API, you can implement functionalities such as jumping to a specific time in the media, adjusting the playback rate, and switching audio tracks.

Media Seek

You can jump to a specific time in the media based on an absolute time or relative time:

To jump based on an absolute time, use the seekTo() method:

Retrieve the current playback time using the getCurrentTime() method.
Set the time to jump to. The new time must be a positive value, which is less than the duration of the media.

  //Media seek during playback
  var successCallback = function() {
    console.log('Media seek successful');
  }

  var errorCallback = function() {
    console.log('Media seek failed');
  }

  //Jump forward by 5000 ms
  var currentTime = webapis.avplay.getCurrentTime();
  var newTime = currentTime + 5000;

  webapis.avplay.seekTo(newTime,successCallback,errorCallback);

You can also set the start position for media playback by calling the seekTo() method when the AVPlay instance is in the IDLE state.

  //Change playback start time
  var successCallback = function() {
    console.log('Media seek successful');
  }

  var errorCallback = function() {
    console.log('Media seek failed');
  }
  webapis.avplay.open('yourMediaURI');
  ...
  //Move the playback start time to 10 minutes
  webapis.avplay.seekTo(600000,successCallback,errorCallback);
  ...
  webapis.avplay.prepare();

To jump based on a relative time, use the jumpForward() and jumpBackward() methods:

//Media seek during playback
var successCallback = function() {
  console.log('Media seek successful');
}

var errorCallback = function() {
  console.log('Media seek failed');
}

//Case 1 Fast-forward by 5000 ms
webapis.avplay.jumpFoward(5000,successCallback,errorCallback);

//Case 2 Rewind by 5000 ms
webapis.avplay.jumpBackward(5000,successCallback,errorCallback);

Note : You must check that the value of the parameter is within the valid range, based on the current playback time.

Playback Rate Control (Trick Play)

You can control the playback rate of the media using the setSpeed() method to set a multiplier for the playback rate. Positive parameter values play the media forwards, while negative values cause the media to play in reverse.

For example, to play the media at double speed:

webapis.avplay.setSpeed(2);

The maximum playback rate depends on the streaming format.

Streaming Format	Playback Rate Range
Smooth Streaming	-16x ~ +16x
HTTP(S)	-8x ~ +8x
MPEG-DASH	-16x ~ +16x
HTTP Live Streaming (HLS)	Since 2017 models: -16x ~ +16x for content with "EXT-X-I-FRAME-STREAM_INF" tag only

Table 3. Playback rates for streaming formats

Audio Track Switching

To switch between multiple audio tracks during media playback:

Retrieve the audio track list.

The getTotalTrackInfo() method of the AVPlay API can only be called when the AVPlay instance is in the READY, PLAYING, or PAUSED states. It returns an array of objects for all tracks in the media.

Each object has 3 properties:
- type: The AUDIO value identifies the track as an audio track.
- index: The index number for the track.
- extra_info: Additional information about the track, such as language information and number of channels.
```
var totalTrackInfo = webapis.avplay.getTotalTrackInfo();

for (var i=0; i<totalTrackInfo.length;i++)
{
  if (totalTrackInfo.type =='AUDIO')
  {
    console.log('Find audio track.');
    console.log('audio track index is ' + totalTrackInfo.index);
    console.log('audio track language is ' + totalTrackInfo.extra_info.language);
  }
}
```
Select the audio track.

Call the setSelectTrack() method with the track type and index number as parameters:
```
// Select audio track with index 2
webapis.avplay.setSelectTrack('AUDIO',2);
```

DRM Contents Playback Sequence

You can play drm contents as setting drm properties for each drm case properly using setDrm() method in IDLE state.

And, as using ondrmevent() callback function, you can handle errors caused by incorrect drm settings or environments, and in the PlayReady GenChallenge case or Widevine Modular case, you can get the challenge data from AVPlay and get the license data from the license server through the corresponding the challenge data.

Below figures are sequence diagram for each drm case, you can refer to the detailed information at the link.

PlayReady getrights case

This method is driven by the AVPlay creation of challenge data and acquiring license data directly from license server and performing install it self. If license server require addition properties for acquiring license data(e.g. HTTP header, User agent, Cookie, etc.), application should call setDrm("PLAYREADY", "SetProperties", json_object) method with setting additional data corresponding to key and value of json object as the example code below. To execute this case, either set the value for "GetChallenge" into false or do not set "GetChallenge" property for "SetProperties" drmOperation in setDrm() avplay api.

Sample code

function drmEventCallback(event, data) {
    if(data.name == "DrmError") {
        // error handling
    }
}

function prepareCallback() {
    webapi.avplay.play();
}

var json = {
    "DeleteLicenseAfterUse" : true,
  
    // Optional field
    "Cookie":"Your Cookie",
    "LicenseServer" : "License server url",
    "HttpHeader" : "HTTP Header which license server required.",
    "UserAgent" : "Your User Agent"
};
var properties = JSON.stringify(json);

webapi.avplay.open(url);
webapi.avplay.setListener({ondrmevent:drmEventCallback});
webapi.avplay.setDrm("PLAYREADY", "SetProperties", properties);
webapi.avplay.setDisplayRect(0, 0, 1920, 1080);
webapi.avplay.prepareAsync(prepareCallback);

webapi.avplay.stop();
webapi.avplay.close();

Sequence Diagram

PlayReady genchallenge case (Since Tizen 2018)

AVPlay passes the challenge data to the ondrmevent() callback function. The application should acquire and install the license data using setDrm("PLAYREADY", "InstallLicense", license_data) directly from the license server by using jquery.ajax() method, XMLHttpReuqest class, etc. To execute this case, "GetChallenge" property for "SetProperties" drmOperation should set to true via setDrm() AVPlay API.

Sample code

function drmEventCallback(event, data) {
    // Challenge case
    if(data.name == "Challenge") {
        // request license data from license server (via HTTP POST)
        $.ajax({
            url: license_server_url,
            data: atob(data.challenge),
            headers: {
                "Content-Type": "text/xml",
            },
            type: 'POST',
            dataType: 'text',
            cache: false,
            processData: false
        }).success(function(msg, status) {
            webapi.avplay.setDrm("PLAYREADY", "InstallLicense", btoa(msg));
        });
    } else if(data.name == "DrmError") {
        // error handling
    }
}

function prepareCallback() {
    webapi.avplay.play();
}

var json = {
    "DeleteLicenseAfterUse" : true,
    "GetChallenge" : true
};
var properties = JSON.stringify(json);

webapi.avplay.open(url);
webapi.avplay.setListener({ondrmevent:drmEventCallback});
webapi.avplay.setDrm("PLAYREADY", "SetProperties", properties);
webapi.avplay.setDisplayRect(0, 0, 1920, 1080);
webapi.avplay.prepareAsync(prepareCallback);

webapi.avplay.stop();
webapi.avplay.close();

Sequence Diagram

Verimatrix case

Sample code

function drmEventCallback(event, data) {
    if(data.name == "DrmError") {
        // error handling
    }
}

function prepareCallback() {
    webapi.avplay.play();
}

var json = {
    "CompanyName" : YourCompany,
    "IPTV" : public2.example.verimatrix.com,
    Web : public-ott-nodrm.example.verimatrix.com:8080
};
var properties = JSON.stringify(json);

webapi.avplay.open(url);
webapi.avplay.setListener({ondrmevent:drmEventCallback});
webapi.avplay.setDrm("VERIMATRIX", "SetProperties", properties);
webapi.avplay.setDisplayRect(0, 0, 1920, 1080);
webapi.avplay.prepareAsync(prepareCallback);

webapi.avplay.stop();
webapi.avplay.close();

Sequence Diagram

Widevine Modular case

As with PlayReady genchallenge scenario, the application should acquire license data as challenge data to the license server and install using setDrm("WIDEVINE_CDM", "widevine_license_data", license_data).

When sending a request, the application should set the license server information (wvJson.license_server). And if necessary, the application can set the extra information (such as wvJson.wvheader).

Sample code

function drmEventCallback(event, data) {
    // Challenge case
    if(data.name == "Challenge") {
        // request license data from license server (via HTTP POST)
        var message = atob(data.challenge); // The challenge data is base64 encoded type.   

        // <--The application should follow the server interface guide when acquiring license data -->
        var xmlhttp = new XMLHttpRequest();
        xmlhttp.responseType = "arraybuffer";
        xmlhttp.open("POST", wvJson.license_server, true);
        xmlhttp.setRequestHeader("pallycon-customdata", wvJson.wvheader);

        xmlhttp.onload = function(e) {
            if (this.status == 200) {
                if(this.response) {
                    var licenseParam = data.session_id + "PARAM_START_POSITION" + btoa(this.response) + "PARAM_START_POSITION";
                    var licenseDataLen = b64.length;
                    webapi.avplay.setDrm("WIDEVINE_CDM", "widevine_license_data", licenseParam);
                }
            }
        };
        xmlhttp.send(message);
      //<-- server interface guide -->
    } else if(data.name == "DrmError") {
        // error handling
    }
}

function prepareCallback() {
    webapi.avplay.play();
}

var json = {
    "AppSession" : YourAppSession,
    "DataType" : "matroska_webm" // The application should set DataType when the value is 'matroska_webm'.
};
var properties = JSON.stringify(json);

webapi.avplay.open(url);
webapi.avplay.setListener({ondrmevent:drmEventCallback});
webapi.avplay.setDrm("WIDEVINE_CDM", "SetProperties", properties);
webapi.avplay.setDisplayRect(0, 0, 1920, 1080);
webapi.avplay.prepareAsync(prepareCallback);

webapi.avplay.stop();
webapi.avplay.close();

Sequence Diagram

State Limitations for AVPlay API Methods

The following table lists the AVPlay API methods and their valid states. Unless otherwise specified, calling the method does not change the state of the AVPlay instance.

Method	Valid States	Notes
`getVersion()`	any	-
`open()`	NONE, IDLE	After a successful call during a valid state, the instance enters the IDLE state.
`prepare()`	IDLE, READY	After a successful call during a valid state, the instance enters the READY state.
`prepareAsync()`	IDLE, READY
`play()`	READY, PLAYING, PAUSED	After a successful call during a valid state, the instance enters the PLAYING state.
`pause()`	PLAYING, PAUSED	After a successful call during a valid state, the instance enters the PAUSED state.
`stop()`	any	After a successful call, the instance enters the IDLE state.
`close()`	any	After a successful call, the instance is removed and enters the NONE state.
`setDisplayRect()`	IDLE, READY, PLAYING, PAUSED	-
`seekTo()`	IDLE, READY, PLAYING, PAUSED
`jumpForward()`	READY, PLAYING, PAUSED
`jumpBackward()`	READY, PLAYING, PAUSED
`getState()`	any
`getDuration()`
`getCurrentTime()`
`setListener()`
`setSpeed()`	READY, PLAYING, PAUSED
`setDrm()`	NONE, IDLE	NONE state is supported for Verimatrix "GetUID" operation only
`setExternalSubtitlePath()`	IDLE, READY, PLAYING, PAUSED	-
`setSubtitlePosition()`	PLAYING, PAUSED
`setSilentSubtitle()`	IDLE, READY, PLAYING, PAUSED
`setDisplayMethod()`	IDLE, READY, PLAYING, PAUSED
`setSelectTrack()`	READY, PLAYING, PAUSED	READY state is supported for Smooth Streaming only
`getTotalTrackInfo()`		READY state is supported for synchronous prepare method only
`getCurrentStreamInfo()`		-
`setStreamingProperty()`	IDLE
`getStreamingProperty()`	READY, PLAYING, PAUSED
`setBufferingParam()`	IDLE
`suspend()`	READY, PLAYING, PAUSED
`restore()`	PLAYING, PAUSED

Table 4. AVPlay API method state limitations