top

Definition of Actions by Input Methods

An application can be controlled by various input methods, such as Smart Touch Control, standard button-type remote control, mouse, voice recognition, and motion control.

If an application supports mouse, voice recognition, or motion control features, the supported input methods are displayed on the application icon in the Apps Homescreen. Also, the application must inform the user about supported input methods when they open it.

Definition of Smart Touch Control Actions

All applications should be optimized to enable the use of the Smart Touch Control.

Definition of Button Actions

Unlike traditional remote controls, the Smart Touch Control only includes a small number of buttons. The touchpad can be used to move the focus.

The Smart Touch control layout is shown in Figure 3-1. Depending on the region, colored buttons may be provided instead of the four shortcut buttons under the touchpad. Voice and Guide buttons may not be assigned, depending on the region and TV model. The four shortcut buttons may also vary depending on the region. Among the buttons provided, those listed in Table 3-1 can be used to control the application. When the user provides button inputs, the actions described in the action specifications are performed.

Smart Touch Control

Figure 3-1. Smart Touch Control

Buttons Actions
Channel UP/DOWN [TV Status] Return to TV channel and change channel after closing the currently running application
[Outside Input Status] Display a Message (Not Available)
[if PIP screen is included] Change the channel on the PIP screen
Volume UP/DOWN [With No Sound]
NR (Not Response)
[With Sound (Play Music/Clips)]
Display TV Volume OSD
MUTE [With No Sound]
NR (Not Response)
[With Sound (Play Music/Clips)]
Display TV Mute OSD
VOICE Enable voice recognition
MORE Displays the OSD to provide shortcuts to access numbers not provided on the remote control (0-9), Tools, Color, Playback control buttons and other major features of the TV.
Color
(RED (A) / GREEN (B) / YELLOW (C) / BLUE (D))
[With no Assigned Functions] NR (Not Response)
[With Assigned functions] Work the assigned function.
SMART HUB Exit the current application and go to the On TV Homescreen of Smart Hub
RETURN Restore to the previous screen(upper step)
* Depending on the application type, a confirmation popup message may be displayed to ensure that the button was pressed on purpose. (e.g. game applications)
EXIT(Press & hold the RETURN button for at least 2 seconds) [General Status] Close application
[In case that the popup window is open] Close the popup Window.
* Depending on the application type, a confirmation popup message may be displayed to ensure that the button was pressed on purpose. E.g.) game applications
ENTER (Click touchpad) [With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Work the assigned function.
E.g.) Selecting item, Playing video

Table 3-1. Buttons used to control applications

Unlike conventional remote controls, the Tools, Color and Playback control buttons are grouped into a single button (MORE). Therefore, a process of calling the OSD and selecting the button to enter must be added to generate a button input. This means that assigning frequently used features to color buttons or playback control buttons is not recommended. Pressing a color button also immediately exits the OSD, so should not be used for features that require repetitive inputs. It is recommended that the application be designed to provide sufficient usability with basic 4-direction focus movement and ENTER actions.

As shown in Figure 3-1, color buttons are only provided in certain regions. Therefore, globally launched applications must be created with this factor in mind.

Button Inputs using OnScreen Remote

In addition to the physical buttons shown in Figure 3-1 and described in Table 3-1, several other buttons that are provided on the standard long bar-type remote controls can be accessed via the TV screen. When the MORE button is pressed on the Smart Touch Control, the OnScreen Remote, as shown in Figure 3-2, is displayed at the bottom left of the screen.

Figure 3-2. OnScreen Remote

Figure 3-2. OnScreen Remote

The OnScreen Remote consists of 3 panels. By default, the panel with the color buttons and playback control buttons shown in Figure 3-1 is displayed first. The layout of the three panels is as shown below.

Figure 3-3. OnScreen Remote

Figure 3-3. OnScreen Remote

The user can switch to the previous or next screen by moving the focus, or by dragging their finger along the raised horizontal line on the touchpad to select the button needed to control the application. (Refer to Chapter 3-2: Definition of Touchpad Actions) Refer to Table 3-2 for OnScreen Remote buttons used to control an application and Action Specifications for Inputs.

Buttons Actions
Number (0~9) [With No Assigned Functions] NR (No Response)
[With Assigned Functions] Perform the assigned function. (e.g. IME)
Color
(RED (A) / GREEN (B) / YELLOW (C) / BLUE (D))
[With no Assigned Functions] NR (No Response)
[With Assigned functions] Perform the assigned function.
Playback
Enter
[With No Assigned Functions] NR (No Response)
[With Assigned Functions] Perform the assigned function.
TOOLS [With No Assigned Functions] NR (No Response)
[With Assigned Functions] Perform the assigned function.
(e.g. Options, call Additional Functions)
INFO [With No Assigned Functions] NR (No Response)
[With Assigned Functions] Perform the assigned function.

Table 3-2. OnScreen Remote buttons used to control applications

Number Input

When a number entry is required, the user can opt to bring up a number pad just by selecting the input field, and not necessarily pressing the MORE button to bring up the On Screen Remote first. When the user clicks the touchpad while the focus is over the input field, a number pad will automatically be displayed on the screen, as shown in the scenario in Figure 3-4.

The user can move the focus by using the left/right buttons at the bottom. It is recommended that the use of the ENTER button follows the following action specification.

  • When multiple input fields that require only number inputs exist consecutively, pressing the ENTER button moves the focus to the next field.
  • When the ENTER button is pressed in the last field, it automatically closes the input field and the number pad.
Figure 3-4. Automatic Number Pad Pop-up

Figure 3-4. Automatic Number Pad Pop-up

However, when a series of number input fields and text input fields exist, a keypad (Figure 4-9. Number Input Only) must be provided for number inputs. Refer to Chapter 4. IME Categorization for Text Inputs.

Definition of Touchpad Actions

The touchpad on the Smart Touch Control includes the following events.

Click(Press & Release) click Clicking the touchpad
Drag click Dragging the finger on the touchpad or along the protruded horizontal and vertical ridges for a certain distance
Flick click Lightly flicking the finger on the touchpad or on the protruded horizontal and vertical ridges.
Press & Hold click Pressing and holding the touchpad for longer than a certain amount of time (e.g. 2 seconds)
Press & Move click Moving the finger on the touchpad while pressing down

Table 3-3. Basic Touch Actions of Touchpad

By touching and moving the finger on touchpad, the Smart Touch Control can be used to control the focus in a 4-Direction. The ENTER action is achieved by clicking the area of the touchpad that is not designated as an area for specific buttons.

  • 4-Direction Focus Move

    4-Direction Focus Move is achieved by performing dragging or flicking actions on the touchpad. The distance that the focus moves can be determined by the speed of the action.

  • Scrolling the Screen Horizontally & Vertically

    On the top and the two sides of the touchpad are three long, bar-shaped, raised lines. The user can scroll the screen horizontally or vertically by dragging or lightly flicking their fingers on these lines. When the user drags their finger left or right along the top line, the screen scrolls to the left or right accordingly. When the user drags their finger up or down along the left or right lines, the screen scrolls up or down accordingly.

    The drag/flick actions on the lines can be used to perform continuous scrolling by pixels or to switch the screen by panels/pages. Refer to the example below.

    Figure 3-5 shows the example of dragging the finger left and right along the top ridge. As the finger moves, the screen scrolls in the direction shown in the figure. When the finger is dragged from left to right, the page located on the right of the screen scrolls into the center. When the finger is dragged from right to left, the page located on the left of the screen scrolls into the center.

    Figure 3-5. Horizontal scrolling of a continuous list using dragging actions on the raised line

    Figure 3-5. Horizontal scrolling of a continuous list using dragging actions on the raised line

    When applied to a list that automatically switches pages when a drag action is performed, the page should automatically snap to the next page when the dragging distance has exceeded approximately 30% of the area of the page and the finger is released.

    Figure 3-6 is an example of vertically scrolling a web browser by dragging the finger vertically along the left/right ridges.

    Figure 3-6. Scrolling a web browser screen by using dragging actions on the raised line

    Figure 3-6. Scrolling a web browser screen by using dragging actions on the raised line

    Figure 3-7 shows an example of lightly flicking the finger on the raised horizontal line. This action quickly scrolls the page.

    Figure 3-7. Switching pages to the left/right by using a flick action on the raised line

    Figure 3-7. Switching pages to the left/right by using a flick action on the raised line

    If the flick action above occurs on a continuous list as shown in Figure 3-7, the amount of scrolling generated by the flick depends on the flicking speed.

  • Holding and Moving an Object

    This action can be implemented if the application being used supports Pointing Mode actions. Hold & Move can be performed by placing and pressing down on the touchpad to hold the pointer, and then dragging the finger on the touchpad surface without releasing the finger. It can be used to hold and move an object or to pan the background. However, due to the limited space on the touchpad, using it for editing actions that require holding and moving an object over a long distance may lower its usability, and the range of application must be limited accordingly.

Definition of Standard Remote Control Actions

The buttons listed below are those most commonly used in applications. They may have slightly different functions or labels, depending on the product or the model of the remote control.

Buttons that are also provided on the exterior of Smart Touch Controls

Remote Control Buttons Actions
Power TV power On/Off
Volume UP/DOWN [With No Sound]
NR (Not Response)
[With Sound (Play Music/Clips)]
Display TV Volume OSD
MUTE [With No Sound]
NR (Not Response)
[With Sound (Play Music/Clips)]
Display TV Volume OSD
0~9 (Number) [With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Work the assigned function. (e.g. Entering Date of Birth or PIN)
TOOLS [With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Work the Assigned Function.
ex) Options, Additional Functions Calling
INFO [With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Work the assigned function.
SMART HUB Exit the current application and goes to the On TV Homescreen of Smart Hub
RETURN Return to the previous screen (up one step)
* Depending on the application type, a confirmation popup message may be displayed to
ensure that the button was pressed on purpose. (e.g. game applications)
EXIT [General Status] Close application
[In case that the popup window is open] Close the popup Window.
* Depending on the application type, a confirmation popup message may be displayed to
ensure that the button was pressed on purpose. (ex. game applications)
Four Directions [With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Work the assigned function.
ex) Moving the Focus, Adjusting Option Values
Enter(ENTER) [With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Work the assigned function.
ex) Selecting items, Playing videos
Color
(RED (A) / GREEN (B) /
YELLOW (C) / BLUE (D))
[With no Assigned Functions] NR (Not Response)
[With Assigned functions] Work the assigned function.
Playback
Playback
[With No Assigned Functions] NR (Not Response)
[With Assigned Functions] Perform the assigned function.
Channel UP/DOWN [If PIP screen is not included] NR (No Response)
[If PIP screen is included] The channel displayed on the PIP screen changes
[Outside Input Status] Display a Message (Not Available)
VOICE Run voice recognition

Table 3-4. Conventional remote control buttons used to control applications

Voice Recognition

The basic TV functions and applications can be controlled by simply talking or using preset voice commands. However, applications can only be controlled using preset voice commands. You can use preset voice commands to control applications in all countries and languages that support voice recognition, with restrictions. Voice recognition is supported in 26 languages, including Korean and American English.

Basic Voice Recognition Policy

Voice recognition is enabled when the user presses the voice recognition button on the remote control or says a start command to the TV's microphone.

Figure 3-8. Screen layout when using voice recognition

Figure 3-8. Screen layout when using voice recognition

As shown in Figure 3-8, the voice recognition screen shows a voice recognition status indicator icon and a guide message or brief help text. In the Help area, guide messages and commands are shown in quotation marks (""). All readable commands must be marked by quotation marks.

The Help area shows the most important available phrases and commands for the current screen in order of priority. There can be up to 5 pages and 15 phrases or commands. This Help area closes automatically when the user says "Close Help" or when a voice recognition command has been recognized.

If there are more than 15 possible phrases or commands, the user can check them in the View All screen. Extra commands that don't fit in the Help area are still available for use as long as they are registered within the application.

If no key commands are defined for the current screen, only the basic control commands are displayed in the Help area. Table 3-5 below shows the commands that are always available by default. These can be used without requiring additional registration within the application.

위로 / 아래로 Focus Up / Focus Down
왼쪽으로 / 오른쪽으로 Focus Left / Focus Right
선택하기 Select
종료 Exit

Table 3-5. Basic Voice Recognition Commands

It is possible to display the application's own Guide Messages and Commands without the predefined Guide Messages and Help. However, this is not recommended.

Figure 3-9. View All Voice Commands

Figure 3-9. View All Voice Commands

When the user says “Show All”, all the available commands for the current feature are shown on the TV Basics tab, as shown in Figure 3-9. The applications are always listed in order of priority, then in the order of supporting application, followed by commands that are always available.

Figure 3-10. Systems response to the users speech

Figure 3-10. System's response to the user's speech

If a specific response other than running a basic feature is required, System Response is displayed on the screen. As shown in Figure 3-10, user phrases or commands are shown on the left and the system response is shown on the right of the screen. In countries that support it, the system response can also be read out loud using TTS.

Cases that require system responses are as follows.

  • When providing an option to select the function
  • When additional information is required
  • When informing the user of errors
Figure 3-11. Types of System Responses

Figure 3-11. Types of System Responses

Depending on the situation, system responses can be provided with a simple list, a selectable list, or a voice command list. If an additional selection by the user is required, instructions will be displayed on the right-hand side of the central icon with available voice commands. When additional choices are displayed, any voice commands other than the ones given cannot be used.

Figure 3-12. Example of Showing a Selectable Lis

Figure 3-12. Example of Showing a Selectable List

Also, if a choice has to be made from an unspecified list, it is recommended that the application shows a list of options below the Guide Message, or numbers choices when an different UI is used, as shown in Figure 3-12. The user must be able to select the number or bring up the entire list using voice commands. If a list is numbered, the method of selecting an option by saying "First" or "Number 1" is supported, but just reading out the number (one, two, three..) is not supported.

Command Writing Rules

New commands must be written according to the following rules.

  1. Applications must use commands written by Samsung.
  2. By adding the world "Home" after the name of the application (e.g., Skype (X)-> Skype Home), you can call the first page of the current application.
  3. Only commands that are currently available must be displayed on the screen.
    1. When displaying multiple commands in a row, each command must be separated by a comma and a space. (e.g., Play, Stop)
  4. For loanwords, abbreviations and symbols that have more than one correct way of reading them, only one must be displayed and used.
    1. Loanwords must be displayed in the language of origin. (ex : TV (X) -> TV (In Korean)
    2. Only one way of reading abbreviations must be set and displayed as they are pronounced. (e.g., STB (X) -> Set-top Box, MLB (X) -> MLB (In Korean)
    3. Symbols may not be displayed. (ex : TV & Movie (X) -> TV and Movie)
      * When displaying symbols is inevitable, it must be written out as they are pronounced. (e.g. : # -> Pound)

Motion Control

Motion control can be used to control the TV and applications without using a remote control or any other input device.

Basic Motion Control Policy

Motion control is activated when the user makes a predefined gesture in front of the TV's camera. Once activated, the pointer and on screen menu are displayed on the screen. Whether or not to use the on screen menu can be decided depending on the characteristics of each application. When the BACK button is not used, the application must provide a gesture for exiting the application using motion recognition. The composition and rules of the on screen menu displayed for each situation are as follows.

OSD Menu Description Situations for Usage
Live TV Non-live TV
Animated Motion Guide A guide to currently available gestures X O
Volume Control OSD menu for volume control O
(When the user needs to control volume)
Channel Control OSD menu for channel control O
(When the user needs to control channel)
Smart Hub Closes current application and goes to the On TV Homescreen of Smart Hub O X
Back Restores to the previous screen (go back one step) X O
Recommended
(History)
Shows Recommended content
(Shows the History if Recommended content is not provided)
O O
OnScreen Remote Displays the OSD menu for providing shortcut buttons for accessing numbers not provided on the remote control (0-9), Tools, Color, Playback control buttons, and other major features of the TV. O O
Source Displays the Source List O O
Power Off Power off TV O O

Table 3-6. Whether or not to use the OSD menu for each situation

1) Watching Live TV

When motion recognition is activated while watching Live TV, an OSD menu for motion recognition is displayed on the screen as shown in Figure 3-13.

Figure 3-13. When motion control is activated on the Live TV screen

Figure 3-13. When motion control is activated on the Live TV screen

2) Watching Non-live TV

When motion recognition is activated while watching Non-live TV, an on screen menu for motion control is displayed on the screen as shown in Figure 3-14 and automatically disappears after a certain period of time. The on screen menu that corresponds to each area can be brought up by placing the pointer at the corner of the screen for a certain period of time. When the pointer is moved away from the on screen menu area while the it is being displayed, the on screen menu closes after a certain period of time.

Figure 3-14. When motion control is activated on the Non-live TV screen

Figure 3-14. When motion control is activated on the Non-live TV screen

Pointer Control using Motion Control

When motion control is activated, a pointer is displayed on the screen, as shown in Figure 3-15, and the pointer can be used to select an item. When controlling an application using motion control, the user must be able to select and run features. For UIs that immediately switch category when the focus is moved using a remote control, using motion control can delay the switching of the category. Therefore, a method of moving the pointer over an item and selecting it is recommended for such UIs.

Figure 3-9. View All Voice Commands

Figure 3-15. Areas selectable by a pointer

Pointers with different shapes can be designed and applied in each application. However, they must be designed so that they give the user visual confirmation that items have been selected. The maximum size of a pointer is 96 x 96 pixels.

Definition of Available Gesture Sets and Actions

Gestures that are not in the provided gesture set cannot be created or registered. The functions of each gesture can be redefined depending on the characteristics of each application. However, the gesture set below can be used in all situations within Samsung Smart TV. Therefore, it is recommended that gestures be used for the same functions within applications to provide a uniform user experience.
※ Two-handed gestures cannot be used on TVs that use an external camera. (Thumbs-up can be used.)

Category Gesture Actions
Default Gestures move pointer Move Move pointer
grab run Grab Run (Continuous actions may be provided when an item that can be run continuously is grabbed and held.)
slap Slap Change item or page
Extended Gestures grab expanding Grabbing & Expanding/Contracting
with two hands
Zoom in/Zoom out
grab rotating Grabbing & Rotating
with two hands
Rotate
grab move Grab & Move Pan
Special Gesture thumb up Thumb-up Defined according to application characteristics
(e.g., Using the Like feature in the Facebook App)
Activating and Back Gestures waving a hand Waving a hand Activate motion control
waving two hands Waving two hands Activate two-handed motion control
counterclockwise rotation Counterclockwise rotation Back

Table 3-7. Gesture Set

Slap Gestures

The user may use the slap gesture of moving the finger left and right, as if turning a page, to move to the previous/next item or page. The visual feedback of the slap gesture can be redefined depending on the characteristics of each application.

Figure 3-9. View All Voice Commands

Figure 3-16. Moving to the next item using slap gestures

Two-handed Gestures

When two hands are recognized, two pointers, following the left and right hands respectively, are displayed on the screen. Each hand can perform grab gestures independently. Functions specialized to two-handed gestures, such as zooming in/out and rotating, can be performed by moving the two hands closer together or further apart after performing a grab gesture. The functions of two-handed gestures can be redefined according to the characteristics of each application.

Figure 3-17. Onscreen feedback when zooming in/out

Figure 3-17. Onscreen feedback when zooming in/out

Thumbs-up

When motion control is not activated, the user can use the thumbs-up gesture by making a fist and raising the thumb. When the user holds a thumbs-up gesture for 0.5 seconds, a visual feedback appears on the screen. If the hand is lowered while the visual feedback is being displayed, the function that corresponds to the thumbs-up gesture is not executed. After displaying the visual feedback, a popup window appears informing the user that the function has been performed successfully. '※ However, the thumbs-up gesture cannot be used if motion control itself is enabled.

Figure 3-18. Onscreen feedback for the thumbs-up function

Figure 3-18. Onscreen feedback for the thumbs-up function

Mouse

When a mouse is connected to the TV, the mouse pointer is displayed on the screen and the left click can be used to select items. When controlling an application using a mouse, the user must be able to select items and execute functions, as well as exit the application using the mouse.

Items that can be clicked using a mouse are as follows.

  • All items that can be performed (executed) with the Enter button
  • Scroll Indicator on lists
  • Tracking the Scrollbar/Slider/Playbar
  • Key Guide

Basic Mouse Policy

When the pointer is rolled over a selectable item, the focus follows the pointer.

Figure 3-19. When the pointer is rolled over a selectable item

Figure 3-19. When the pointer is rolled over a selectable item

When the pointer is not over a selectable item, the focus remains at the last selectable item. When a 4-direction input is received, the focus begins moving from the current position.
Figure 3-20. When the pointer is moved away from a selectable item

Figure 3-20. When the pointer is moved away from a selectable item

Definition of item and pointer status depending on pointer positions

The status of items and pointers depending on pointer positions are as follows.

Item Status
item default

Default

Default status when the pointer or focus is not displayed
item focus

Pointer Rollover

Status when the pointer is rolled over a selectable item
item rollover

4-direction Focus

The status when the focus remains at the last selectable item when the pointer is over an item that is not selectable in Pointing Control Mode

Table 3-8. Item Status

Pointer Status
default cursor Default status of the pointer
When the pointer is positioned over an empty area without a selectable object
rollover cursor When the pointer is positioned over a selectable object
A focus (highlight) is shown on the item and the pointer graphics are changed.
not selected cursor When the pointer is positioned over an object that is not selectable (Identical to default pointer status)
press cursor When the left button is pressed

Table 3-9. Pointer Status

Mouse Button Control & Scrolling

If there is a Tools menu available for the current screen, it can be called by using the right mouse button. In a screen with no Tools menu, using the right mouse button has no effect.

Figure 3-21. Screens showing a Tools menu

Figure 3-21. Screens showing a Tools menu

The scroll wheel of the mouse is used for vertical scrolling.

Factors common to both Motion Control and using a Mouse

Provides an indicator for scrolling a list

When using motion or mouse control, an indicator for scrolling must be provided. Pressing the list indicator will scroll the list in the marked direction. Clicking & holding the indicator will perform continuous scrolling.

Figure 3-22. Examples of indicators provided for a vertical list

Figure 3-22. Examples of indicators provided for a vertical list

Figure 3-9. View All Voice Commands

Figure 3-23. Examples of indicators provided for a horizontal list

Scroll bar/Slider/Play bar

Left-clicking over UI items such as a scroll bar, slider, or play bar jumps to the clicked position.

Figure 3-24. Example of a slider

Figure 3-24. Example of a slider