PoseEstimation API

To use Samsung Product API,

<script type="text/javascript" src="$WEBAPIS/webapis/webapis.js"></script>

Should be loaded in index.html

This module defines the functions of pose estimation using AIFW that are provided as the Tizen Samsung Product API.

Since : 9.0

Product : TV

Summary of Interfaces and Methods

Interface

 Method
PoseEstimationKeypoints
PoseEstimationManagerObject
PoseEstimationManager

 DOMString getVersion();
PoseEstimationHandle create();
void init(PoseEstimationHandle handle);
PoseEstimationKeypoints[] run(PoseEstimationHandle handle, DOMString url, Array roi);
PoseEstimationKeypoints[] runWithBuffer(PoseEstimationHandle handle, DOMString buffer, Array roi);
void deinit(PoseEstimationHandle handle);
void destroy(PoseEstimationHandle handle);

1. Type Definitions

1.1 KeyPonitId

Specifies the pose estimation keypoint.

enum KeyPonitId {
  "NOSE",
  "LEFT_EYE",
  "RIGHT_EYE",
  "LEFT_EAR",
  "RIGHT_EAR",
  "LEFT_SHOULDER",
  "RIGHT_SHOULDER",
  "LEFT_ELBOW",
  "RIGHT_ELBOW",
  "LEFT_WRIST",
  "RIGHT_WRIST",
  "LEFT_HIP",
  "RIGHT_HIP",
  "LEFT_KNEE",
  "RIGHT_KNEE",
  "LEFT_ANKLE",
  "RIGHT_ANKLE",
  "CNT"
};

The following values are supported

  • NOSE : nose keypoint
  • LEFT_EYE : left_eye keypoint
  • RIGHT_EYE : right_eye keypoint
  • LEFT_EAR : left_ear keypoint
  • RIGHT_EAR : right_ear keypoint
  • LEFT_SHOULDER : left_shoulder keypoint
  • RIGHT_SHOULDER : right_shoulder keypoint
  • LEFT_ELBOW : left_elbow keypoint
  • RIGHT_ELBOW : right_elbow keypoint
  • LEFT_WRIST : left_wrist keypoint
  • RIGHT_WRIST : right_wrist keypoint
  • LEFT_HIP : left_hip keypoint
  • RIGHT_HIP : right_hip keypoint
  • LEFT_KNEE : left_knee keypoint
  • RIGHT_KNEE : right_knee keypoint
  • LEFT_ANKLE : left_ankle keypoint
  • RIGHT_ANKLE : right_ankle keypoint
  • CNT : count of keypoints.

1.2 PoseEstimationHandle

Defines the handle of the pose estimation.
It is created through create(), and the created handle is invalid after destroy() is called.
The Pose estimation API identifies the Estimation created through the PoseEstimationHandle parameter.

typedef DOMString PoseEstimationHandle;

2. Interfaces

2.1 PoseEstimationKeypoints

Specifies the key point information of pose estimation.

[NoInterfaceObject] interface PoseEstimationKeypoints {
  readonly attribute double x;
  readonly attribute double y;
  readonly attribute double confidence;
  readonly attribute KeyPonitId keypoint;
};

Attributes

  • readonly double x
    Indicates x-coordinate of keypoint
  • readonly double y
    Indicates y-coordinate of keypoint
  • readonly double confidence
    Indicates confidence score of keypoint
  • readonly KeyPonitId keypoint
    Indicates id of keypoint

2.2 PoseEstimationManagerObject

Defines a WebApi object instance of the Tizen Samsung Product API.
The webapis.poseestimation object enables access to Pose Estimation API functionality.

[NoInterfaceObject] interface PoseEstimationManagerObject {
  readonly attribute PoseEstimationManager poseestimation;
};

WebApi implements PoseEstimationManagerObject;

Attributes

2.3 PoseEstimationManager

Provides methods for Pose Estimation functionalities.

[NoInterfaceObject] interface PoseEstimationManager {
  DOMString getVersion();
  PoseEstimationHandle create();
  void init(PoseEstimationHandle handle);
  PoseEstimationKeypoints[] run(PoseEstimationHandle handle, DOMString url, Array roi);
  PoseEstimationKeypoints[] runWithBuffer(PoseEstimationHandle handle, DOMString buffer, Array roi);
  void deinit(PoseEstimationHandle handle);
  void destroy(PoseEstimationHandle handle);
};

Methods

getVersion

Retrieves the plugin version number.

DOMString getVersion();

Return Value :

  • DOMString : Plugin version

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.

Code Example :

try {
  let value = webapis.poseestimation.getVersion();
  console.log("version value = " + value);
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

create

Create a new instance of pose estimation. You should call the method first to use pose estimation.

PoseEstimationHandle create();

Return Value :

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.
    • with error type UnknownError, for any other error.

Code Example :

try {
  let handle = webapis.poseestimation.create();
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

init

Initialize pose estimation. e.g loading a model, etc.

void init(PoseEstimationHandle handle);

Parameters :

  • handle : handle of pose estimation

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.
    • with error type TypeMismatchError, if the type of the parameters passed into the function is incorrect.
    • with error type UnknownError, for any other error.

Code Example :

try {
  let handle = webapis.poseestimation.create();
  webapis.poseestimation.init(handle); //you should call init() before calling run().
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

run

Run pose estimation with a content URL to get the keypoint information from it.

PoseEstimationKeypoints[] run(PoseEstimationHandle handle, DOMString url, Array roi);

Parameters :

  • handle : handle of pose estimation
  • url : Content URL,It should be an absolute local path of a PNG image
  • roi : the body roi information of input image
    roi[0] = x coordinate of roi, value should be 0~1
    roi[1] = y coordinate of roi, value should be 0~1
    roi[2] = width of roi, value should be 0~1
    roi[3] = height of roi, value should be 0~1

Return Value :

  • PoseEstimationKeypoints[] : a array which include all keypoints info, see the description of PoseEstimationKeypoints

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.
    • with error type TypeMismatchError, if the type of the parameters passed into the function is incorrect.
    • with error type InvalidValuesError, if any input parameter contains an invalid value.
    • with error type UnknownError, for any other error.

Code Example :

try {
  let handle = webapis.poseestimation.create();
  webapis.poseestimation.init(handle);
  let roi = [0.406667, 0.144097, 0.136667, 0.697571];
  let outputs = webapis.poseestimation.run(handle, "/opt/media/USBDriveA1/images/high_knees.png", roi); //you should call run() after calling init()
  console.log("key points number : " + outputs.length);
  for (let i = 0; i < outputs.length; i++) {
    console.log("KeyPonit[" + i + "]： keypoint = " + outputs[i].keypoint + ", confidence = " + outputs[i].confidence + ", coordinate =(" + outputs[i].x + "," + outputs[i].y + ")");
  }
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

runWithBuffer

Run pose estimation with a base64 encoded buffer to get the keypoint information from it.

PoseEstimationKeypoints[] runWithBuffer(PoseEstimationHandle handle, DOMString buffer, Array roi);

Parameters :

  • handle : handle of pose estimation
  • buffer : It should be base64 encoded buffer data of a PNG image file
  • roi : the body roi information of input image
    roi[0] = x coordinate of roi, value should be 0~1
    roi[1] = y coordinate of roi, value should be 0~1
    roi[2] = width of roi, value should be 0~1
    roi[3] = height of roi, value should be 0~1

Return Value :

  • PoseEstimationKeypoints[] : a array which include all keypoints info, see the description of PoseEstimationKeypoints

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.
    • with error type TypeMismatchError, if the type of the parameters passed into the function is incorrect.
    • with error type InvalidValuesError, if any input parameter contains an invalid value.
    • with error type UnknownError, for any other error.

Code Example :

try {
  let handle = webapis.poseestimation.create();
  webapis.poseestimation.init(handle);
  let roi = [0.406667, 0.144097, 0.136667, 0.697571];
  let outputs = webapis.poseestimation.runWithBuffer(handle, "/9j/4AAQSkZJRgABAQAAAQABAAD//gA7Q1JFQVRPU...", roi); //you should call runWithBuffer() after calling init()
  console.log("key points number : " + outputs.length);
  for (let i = 0; i < outputs.length; i++) {
    console.log("KeyPonit[" + i + "]： keypoint = " + outputs[i].keypoint + ", confidence = " + outputs[i].confidence + ", coordinate =(" + outputs[i].x + "," + outputs[i].y + ")");
  }
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

deinit

Deinitialize pose estimation. If you have called init function, you should call deinit before terminating your application.

void deinit(PoseEstimationHandle handle);

Parameters :

  • handle : handle of pose estimation

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.
    • with error type TypeMismatchError, if the type of the parameters passed into the function is incorrect.
    • with error type UnknownError, for any other error.

Code Example :

try {
  let handle = webapis.poseestimation.create();
  webapis.poseestimation.init(handle);
  let roi = [0.406667, 0.144097, 0.136667, 0.697571];
  webapis.poseestimation.run(handle, "/opt/media/USBDriveA1/images/high_knees.png", roi);
  webapis.poseestimation.deinit(handle); //you should call deinit() after using pose estimation.
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

destroy

Destroy an instance of pose estimation. If you have called create function, you should call destroy before terminating your application.

void destroy(PoseEstimationHandle handle);

Parameters :

  • handle : handle of pose estimation

Exceptions :

  • WebAPIException
    • with error type NotSupportedError, if this API is called for products which don't support pose estimation.
    • with error type TypeMismatchError, if the type of the parameters passed into the function is incorrect.
    • with error type UnknownError, for any other error.

Code Example :

try {
  let handle = webapis.poseestimation.create();
  webapis.poseestimation.init(handle);
  let roi = [0.406667, 0.144097, 0.136667, 0.697571];
  webapis.poseestimation.run(handle, "/opt/media/USBDriveA1/images/high_knees.png", roi);
  webapis.poseestimation.deinit(handle);
  webapis.poseestimation.destroy(handle);
} catch (error) {
  console.error("error code = " + error.code + ", error name = " + error.name + ", error message = " + error.message);
}

3. Full WebIDL

module PoseEstimation {
  enum KeyPonitId {
    "NOSE",
    "LEFT_EYE",
    "RIGHT_EYE",
    "LEFT_EAR",
    "RIGHT_EAR",
    "LEFT_SHOULDER",
    "RIGHT_SHOULDER",
    "LEFT_ELBOW",
    "RIGHT_ELBOW",
    "LEFT_WRIST",
    "RIGHT_WRIST",
    "LEFT_HIP",
    "RIGHT_HIP",
    "LEFT_KNEE",
    "RIGHT_KNEE",
    "LEFT_ANKLE",
    "RIGHT_ANKLE",
    "CNT"
  };

  typedef DOMString PoseEstimationHandle;

  [NoInterfaceObject] interface PoseEstimationKeypoints {
    readonly attribute double x;
    readonly attribute double y;
    readonly attribute double confidence;
    readonly attribute KeyPonitId keypoint;
  };

  [NoInterfaceObject] interface PoseEstimationManagerObject {
    readonly attribute PoseEstimationManager poseestimation;
  };

  WebApi implements PoseEstimationManagerObject;

  [NoInterfaceObject] interface PoseEstimationManager {
    DOMString getVersion();
    PoseEstimationHandle create();
    void init(PoseEstimationHandle handle);
    PoseEstimationKeypoints[] run(PoseEstimationHandle handle, DOMString url, Array roi);
    PoseEstimationKeypoints[] runWithBuffer(PoseEstimationHandle handle, DOMString buffer, Array roi);
    void deinit(PoseEstimationHandle handle);
    void destroy(PoseEstimationHandle handle);
  };

};