Filter
-
Content Type
-
Category
Mobile/Wearable
Visual Display
Digital Appliance
Platform
Recommendations
Filter
Learn Code Lab
codelabimplement flex mode on an unreal engine game objective learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface jni overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps when the phone is partially folded, it will go into flex mode apps will reorient to fit the screen, letting you watch videos or play games without holding the phone for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard set up your environment you will need the following epic games launcher with unreal engine 4 or later visual studio or any source code editor samsung galaxy foldable device galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available create and set up your project after launching unreal engine from the epic games launcher, follow the steps below to start your project in the select or create new project window, choose games as a new project category and click next select third person as template, then click next to proceed noteyou can implement flex mode on any template or existing projects and use this code lab activity as a reference in the project settings window, set the following type of project c++ target platform mobile / tablet performance characteristics scalable 3d or 2d real-time raytracing raytracing disabled include an additional content pack no starter content project name tutorial_project click create project wait for the engine to finish loading and open the unreal editor once the project is loaded, go to edit > project settings > platforms > android click the configure now button if the project is not yet configured for the android platform then, proceed with the following apk packaging and build settings a apk packaging set target sdk version to 30 set orientation to full sensor change the maximum supported aspect ratio to 2 8 aspect ratio of galaxy z fold3 in decimal to prevent black bars from appearing on the cover display leave it if your game does not need to use the cover display enable use display cutout region?, to prevents black bars at the edge of the main screen otherwise, leave it unchecked b build disable support opengl es3 1 and enable support vulkan notecurrently, there is a problem with opengl es and the split-screen system being investigated the only option right now is to turn off opengl es and use vulkan instead enable native resize event the resize event of a game when switching between displays is disabled in the engine by default however, this behavior can be easily enabled by setting android enablenativeresizeevent=1 in the deviceprofile currently, the only way to create a profile for foldable devices is by creating a specific rule for each device to save time in this code lab, enable the native resize event for all android devices instead locate and open the tutorial_project > config folder in file explorer inside the config folder, create a new folder named android create a new file called androiddeviceprofiles ini and open this file in a text editor, such as visual studio copy below deviceprofile code to the newly created androiddeviceprofiles ini file [deviceprofiles] +deviceprofilenameandtypes=android,android [android deviceprofile] devicetype=android baseprofilename= +cvars=r mobilecontentscalefactor=1 0 +cvars=slate absoluteindices=1 +cvars=r vulkan delayacquirebackbuffer=2 +cvars=r vulkan robustbufferaccess=1 +cvars=r vulkan descriptorsetlayoutmode=2 ; don't enable vulkan by default specific device profiles can set this cvar to 0 to enable vulkan +cvars=r android disablevulkansupport=1 +cvars=r android disablevulkansm5support=1 ; pf_b8g8r8a8 +cvars=r defaultbackbufferpixelformat=0 +cvars=android enablenativeresizeevent=1 ; previewallowlistcvars and previewdenylistcvars are arrays of cvars that are included or excluded from being applied in mobile preview ; if any previewallowlistcvars is set, cvars are denied by default previewallowlistcvars=none this is a copy of the default android deviceprofile from the existing basedeviceprofiles ini file but with the enabled nativeresizeevent console variable cvars notethis step is not required when you only want to implement flex mode yet, it's recommended, to allow applications to run seamlessly from main to cover display without stretching and squashing the game, by enabling the nativeresizeevent create a new plugin and import the foldablehelper foldablehelper is a java file that you can use in different projects it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device flat for normal mode and half-opened for flex mode , window size, and orientation of the fold on the screen download the foldablehelper java file here foldablehelper java 5 64 kb to import the foldablehelper java file to the project, follow the steps below go to edit > plugins in the unreal editor click the new plugin button and select blank to create a blank plugin in the name field, type foldables_tutorial and click the create plugin button in file explorer, locate and open tutorial_project > plugins folder go to plugins > foldables_tutorial > source> foldables_tutorial > private and create a new folder called java copy the foldablehelper java file into java folder open the tutorial_project sln file in visual studio in the same private folder path, add a new filter called java right-click on the java filter and click add > existing item locate the foldablehelper java file, then click add to include this java file in the build modify java activity to use foldablehelper unreal plugin language upl is a simple xml-based language created by epic games for manipulating xml and returning strings using upl, you can utilize the foldablehelper java file by modifying the java activity and related gradle files as follows in visual studio, right-click on source > foldables_tutorial folder, then click add > new item > web > xml file xml create an xml file called foldables_tutorial_upl xml ensure that the file location is correct before clicking add in the newly created xml file, include the foldablehelper java file in the build by copying the java folder to the build directory <root xmlns android="http //schemas android com/apk/res/android"> <prebuildcopies> <copydir src="$s plugindir /private/java" dst="$s builddir /src/com/samsung/android/gamedev/foldable" /> </prebuildcopies> set up the gradle dependencies in the build gradle file by adding the following in the xml file <buildgradleadditions> <insert> dependencies { implementation filetree dir 'libs', include ['* jar'] implementation "org jetbrains kotlin kotlin-stdlib-jdk8 1 6 0" implementation "androidx core core 1 7 0" implementation "androidx core core-ktx 1 7 0" implementation "androidx appcompat appcompat 1 4 0" implementation "androidx window window 1 0 0" implementation "androidx window window-java 1 0 0" } android{ compileoptions{ sourcecompatibility javaversion version_1_8 targetcompatibility javaversion version_1_8 } } </insert> </buildgradleadditions> next, modify the gameactivity <gameactivityimportadditions> <insert> <!-- package name of foldablehelper --> import com samsung android gamedev foldable foldablehelper; </insert> </gameactivityimportadditions> <gameactivityoncreateadditions> <insert> foldablehelper init this ; </insert> </gameactivityoncreateadditions> <gameactivityonstartadditions> <insert> foldablehelper start this ; </insert> </gameactivityonstartadditions> <gameactivityonstopadditions> <insert> foldablehelper stop ; </insert> </gameactivityonstopadditions> </root> gameactivityimportadditions adds the com samsung android gamedev foldable foldablehelper into the gameactivity with the existing imports gameactivityoncreateadditions adds the code to the oncreate method inside the gameactivity gameactivityonstartadditions adds the code to the onstart method inside the gameactivity gameactivityonstopadditions adds the code to the onstop method inside the gameactivity save the xml file then, ensure that the engine uses the upl file by modifying the foldables_tutorial build cs script, located in the same folder as the foldables_tutorial_upl xml file after the dynamicallyloadedmodulenames addrange call, add the following if target platform == unrealtargetplatform android { additionalpropertiesforreceipt add "androidplugin", moduledirectory + "\\foldables_tutorial_upl xml" ; } this means that the game engine will use the upl file if the platform is android otherwise, the foldablehelper won’t work implement a storage struct the next thing to implement is a struct, the native version of java’s foldablelayoutinfo class to store the data retrieved from the java code using a struct, do the following in content browser of unreal editor, right-click on c++ classes > add/import content then, click new c++ class select none for the parent class and click next name the new class as foldablelayoutinfo assign it to the foldables_tutorial plugin then, click create class delete the created foldablelayoutinfo cpp file and only keep its header file in the header file called foldablelayoutinfo h, set up a struct to store all needed data from the windowmanager #pragma once #include "core h" enum efoldstate { undefined_state, flat, half_opened }; enum efoldorientation { undefined_orientation, horizontal, vertical }; enum efoldocclusiontype { undefined_occlusion, none, full }; struct ffoldablelayoutinfo { efoldstate state; efoldorientation orientation; efoldocclusiontype occlusiontype; fvector4 foldbounds; fvector4 currentmetrics; fvector4 maxmetrics; bool isseparating; ffoldablelayoutinfo state efoldstate undefined_state , orientation efoldorientation undefined_orientation , occlusiontype efoldocclusiontype undefined_occlusion , foldbounds -1, -1, -1, -1 , currentmetrics -1, -1, -1, -1 , maxmetrics -1, -1, -1, -1 , isseparating false { } }; implement jni code to implement jni, create a new c++ class with no parent and name it foldables_helper assign the class to the same plugin, then modify the c++ header and source files as follows in the created header file foldables_helper h , include foldablelayoutinfo h #include "foldablelayoutinfo h" then, declare a multicast_delegate to serve as a listener for passing the data from the java implementation to the rest of the engine declare_multicast_delegate_oneparam fonlayoutchangeddelegate, ffoldablelayoutinfo ; lastly, set up the methods and member variables class foldables_tutorial_api ffoldables_helper { public static void init ; static bool haslistener; static fonlayoutchangeddelegate onlayoutchanged; }; moving to the source file foldables_helper cpp , set up the definitions for the methods and member variables created in the header file bool ffoldables_helper haslistener = false; fonlayoutchangeddelegate ffoldables_helper onlayoutchanged; void ffoldables_helper init { haslistener = true; } now, in the same source file, create the native version of the onlayoutchanged function created in the foldablehelper java file since the java onlayoutchanged function only works on android, surround the function with an #if directive to ensure that it compiles only on android #if platform_android #endif within this directive, copy the code below to use the jni definition of the java onlayoutchanged function extern "c" jniexport void jnicall java_com_samsung_android_gamedev_foldable_foldablehelper_onlayoutchanged jnienv * env, jclass clazz, jobject jfoldablelayoutinfo { create the ffoldablelayoutinfo to store the data retrieved from java ffoldablelayoutinfo result; retrieve the field ids of the foldablelayoutinfo and rect objects created in the java file //java foldablelayoutinfo field ids jclass jfoldablelayoutinfocls = env->getobjectclass jfoldablelayoutinfo ; jfieldid currentmetricsid = env->getfieldid jfoldablelayoutinfocls, "currentmetrics", "landroid/graphics/rect;" ; jfieldid maxmetricsid = env->getfieldid jfoldablelayoutinfocls, "maxmetrics", "landroid/graphics/rect;" ; jfieldid hingeorientationid = env->getfieldid jfoldablelayoutinfocls, "hingeorientation", "i" ; jfieldid stateid = env->getfieldid jfoldablelayoutinfocls, "state", "i" ; jfieldid occlusiontypeid = env->getfieldid jfoldablelayoutinfocls, "occlusiontype", "i" ; jfieldid isseparatingid = env->getfieldid jfoldablelayoutinfocls, "isseparating", "z" ; jfieldid boundsid = env->getfieldid jfoldablelayoutinfocls, "bounds", "landroid/graphics/rect;" ; jobject currentmetricsrect = env->getobjectfield jfoldablelayoutinfo, currentmetricsid ; //java rect object field ids jclass rectcls = env->getobjectclass currentmetricsrect ; jfieldid leftid = env->getfieldid rectcls, "left", "i" ; jfieldid topid = env->getfieldid rectcls, "top", "i" ; jfieldid rightid = env->getfieldid rectcls, "right", "i" ; jfieldid bottomid = env->getfieldid rectcls, "bottom", "i" ; retrieve the current windowmetrics and store it in the ffoldablelayoutinfo as an fintvector4 // currentmetrics int left = env->getintfield currentmetricsrect, leftid ; int top = env->getintfield currentmetricsrect, topid ; int right = env->getintfield currentmetricsrect, rightid ; int bottom = env->getintfield currentmetricsrect, bottomid ; // store currentmetrics rect to fvector4 result currentmetrics = fintvector4{ left, top, right, bottom }; do the same for the other variables in the java foldablelayoutinfo // maxmetrics jobject maxmetricsrect = env->getobjectfield jfoldablelayoutinfo, maxmetricsid ; left = env->getintfield maxmetricsrect, leftid ; top = env->getintfield maxmetricsrect, topid ; right = env->getintfield maxmetricsrect, rightid ; bottom = env->getintfield maxmetricsrect, bottomid ; //store maxmetrics rect to fvector4 result maxmetrics = fintvector4{ left, top, right, bottom }; int hingeorientation = env->getintfield jfoldablelayoutinfo, hingeorientationid ; int state = env->getintfield jfoldablelayoutinfo, stateid ; int occlusiontype = env->getintfield jfoldablelayoutinfo, occlusiontypeid ; bool isseparating = env->getbooleanfield jfoldablelayoutinfo, isseparatingid ; // store the values to an object for unreal result orientation = tenumasbyte<efoldorientation> hingeorientation + 1 ; result state = tenumasbyte<efoldstate> state + 1 ; result occlusiontype = tenumasbyte<efoldocclusiontype> occlusiontype + 1 ; result isseparating = isseparating; // boundsrect jobject boundsrect = env->getobjectfield jfoldablelayoutinfo, boundsid ; left = env->getintfield boundsrect, leftid ; top = env->getintfield boundsrect, topid ; right = env->getintfield boundsrect, rightid ; bottom = env->getintfield boundsrect, bottomid ; // store maxmetrics rect to fvector4 result foldbounds = fintvector4{ left, top, right, bottom }; broadcast the result via the onlayoutchanged delegate for use in the engine if ffoldables_helper haslistener { ue_log logtemp, warning, text "broadcast" ; ffoldables_helper onlayoutchanged broadcast result ; } } create a player controller and two ui states this section focuses on adding a player controller and creating two user interface ui states for flat and flex modes these objects are needed for the flex mode logic implementation following are the steps to add a player controller and create two ui states add a new player controller blueprint in content browser, go to content > thirdpersoncpp and right-click on blueprints > add/import content > blueprint class pick player controller as its parent class rename it as flexplayercontroller notethe flexplayercontroller added is generic and can be replaced by your custom player controller in an actual project add a new c++ class with actor component as its parent class name it as foldables_manager and assign it to the foldables_tutorial plugin click the create class button open the flexplayercontroller blueprint by double-clicking it click open full blueprint editor attach the actor component to the flexplayercontroller in the left pane, click add component, then find and select the foldables_manager next, create a pair of userwidget classes for the ui layouts needed flat mode ui for the full screen or normal mode; and flex mode ui for split-screen in add c++ class window, select the show all classes checkbox find and pick userwidget as the parent class then, click next name the new user widget as flatui and attach it to the plugin click next repeat the process but name the new user widget as flexui you might get an error when trying to compile stating that the userwidget is an undeclared symbol to fix this, open the foldables_tutorial build cs file, and in the publicdependencymodulenames addrange call, add "inputcore" and "umg" to the list create a pair of blueprints made from subclasses of these two user widgets right-click on content and create a new folder called foldui inside the newly created folder, right-click to add a new blueprint class in all classes, choose flatui and click the select button rename the blueprint as bp_flatui in the same folder, repeat the process but choose the flexui class and rename the blueprint as bp_flexui double-click on bp_flatui and bp_flexui, then design your two uis like below to visualize switching between flat and flex mode flat ui flex ui notethis code lab activity does not cover the steps on how to create or design ui if you want to learn about how to create your own game design in unreal engine 4, refer to unreal motion graphics ui designer guide implement the flex mode logic after creating the flexplayercontroller and the two ui states bp_flatui and bp_flexui , you can now implement flex mode logic in the foldables_manager open the foldables_manager h and include the necessary c++ header files #pragma once #include "coreminimal h" #include "components/actorcomponent h" #include "engine h" #include "flatui h" #include "flexui h" #include "foldables_helper h" #include "foldables_manager generated h" remove the line below to save a little bit of performance as this component doesn't need to tick public // called every frame virtual void tickcomponent float deltatime, eleveltick ticktype, factorcomponenttickfunction* thistickfunction override; set up the functions needed in foldables_manager the constructor, a function to create the ui widgets the implementation of onlayoutchanged delegate public // sets default values for this component's properties ufoldables_manager ; void createwidgets ; protected // called when the game starts virtual void beginplay override; protected void onlayoutchanged ffoldablelayoutinfo foldablelayoutinfo ; then, set up the variables needed references to the flat and flex ui classes references to the flat and flex ui objects mark the pointers as uproperty to ensure that garbage collection does not delete the objects they point to tsubclassof<uuserwidget> flatuiclass; tsubclassof<uuserwidget> flexuiclass; uproperty class uflatui* flatui; uproperty class uflexui* flexui; finally, define a new private function restoreflatmode , to disable flex mode and return to normal mode private void restoreflatmode ; moving over to foldables_manager cpp, implement the constructor using the constructorhelpers, find the ui classes and set the variables to store these classes also, set the bcanevertick to false to prevent the component from ticking and remove the code block of tickcomponent function // sets default values for this component's properties ufoldables_manager ufoldables_manager { primarycomponenttick bcanevertick = false; static constructorhelpers fclassfinder<uflatui> flatuibpclass text "/game/foldui/bp_flatui" ; static constructorhelpers fclassfinder<uflexui> flexuibpclass text "/game/foldui/bp_flexui" ; if flatuibpclass succeeded { flatuiclass = flatuibpclass class; } if flexuibpclass succeeded { flexuiclass = flexuibpclass class; } } next, set up the beginplay function to link the delegate to the onlayoutchanged function, to initialize the foldables_helper, and to create the widgets ready for use in the first frame // called when the game starts void ufoldables_manager beginplay { super beginplay ; ffoldables_helper onlayoutchanged adduobject this, &ufoldables_manager onlayoutchanged ; ffoldables_helper init ; createwidgets ; } set up the createwidgets function to create the widgets using the ui classes acquired in the constructor add the flatui widget to the viewport, assuming the app opens in normal mode until it receives the foldablelayoutinfo void ufoldables_manager createwidgets { flatui = createwidget<uflatui> aplayercontroller* getowner , flatuiclass, fname text "flatui" ; flexui = createwidget<uflexui> aplayercontroller* getowner , flexuiclass, fname text "flexui" ; flatui->addtoviewport ; } afterward, create the onlayoutchanged function, which will be called via a delegate inside this function, check whether the device’s folded state is half_opened if so, check whether the orientation of the fold is horizontal void ufoldables_manager onlayoutchanged ffoldablelayoutinfo foldablelayoutinfo { //if state is now flex if foldablelayoutinfo state == efoldstate half_opened { if foldablelayoutinfo orientation == efoldorientation horizontal { notefor this third person template, splitting the screen vertically isn’t ideal from a user experience ux point of view for this code lab activity, split the screen only on the horizontal fold top and bottom screen if you want it vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis you must also ensure that you have a flex ui object for the vertical layout if the device is both on flex mode and horizontal fold, change the viewport to only render on the top screen using the normalized position of the folding feature then in an asynctask on the game thread, disable the flatui and enable the flexui however, if the device is on normal mode, then return to flat ui using restoreflatmode function //horizontal split float foldratio = float foldablelayoutinfo foldbounds y / foldablelayoutinfo currentmetrics w - foldablelayoutinfo currentmetrics y ; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizex = 1 0f; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizey = foldratio; asynctask enamedthreads gamethread, [=] { if flatui->isinviewport { flatui->removefromparent ; } if !flexui->isinviewport { flexui->addtoviewport 0 ; } } ; } else { restoreflatmode ; } } else { restoreflatmode ; } } reverse the flex mode implementation logic to create the restoreflatmode function by setting the viewport to fill the screen, then disable the flexui and enable the flatui void ufoldables_manager restoreflatmode { gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizex = 1 0f; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizey = 1 0f; asynctask enamedthreads gamethread, [=] { if !flatui->isinviewport { flatui->addtoviewport 0 ; } if flexui->isinviewport { flexui->removefromparent ; } } ; } set up a game mode and attach the flexplayercontroller the game mode defines the game rules, scoring, and any game-specific behavior set up the game mode in unreal editor by creating a blueprint class in the content > thirdpersoncpp > blueprints folder pick game mode base as the parent class and rename it as flexgamemode double-click on flexgamemode in the drop-down menu next to player controller class, choose the flexplayercontroller lastly, go to edit > project settings > project > maps & modes and select flexgamemode as the default gamemode build and run the app go to edit > package project > android to build the apk ensure that the android development environment for unreal is already set up to your computer noteif the build failed due to corrupted build tools, consider downgrading the version to 30 or lower using the sdk manager or, simply rename d8 bat to the name of the missing file dx bat in sdk path > build-tools > version number directory and, in lib folder of the same directory, rename d8 jar to dx jar after packaging your android project, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement flex mode in your unreal engine game app by yourself! if you're having trouble, you may download this file flex mode on unreal complete code 20 16 mb to learn more, visit www developer samsung com/galaxy-z www developer samsung com/galaxy-gamedev
Learn Code Lab
webcode lab code lab is an education platform that engages anyone to understand, through a series of topics, the entirety of the sdks and tools powered by samsung. all tags all tags watch face studio optimize your watch face design 30 min start watch face studio apply motion graphics to your watch face using tag expressions and gyro effects 30 mins start in-app purchase implement in-app subscriptions using samsung iap 30 mins start watch face studio design a watch face with customizable edge complication slots and digital clock 30 mins start in-app purchase add samsung in-app purchase service to your app 30 mins start watch face studio get creative with weather data in watch face studio 30 mins start sdc24 health build a health app with steps from samsung health and its connected wearables 30 mins start sdc24 health access rich sleep data from samsung health measured by galaxy wearables 30 mins start sdc24 smartthings create a smartthings edge driver for an iot bulb 30 mins start sdc24 smartthings develop a smartthings find-compatible device 30 mins start sdc24 smartthings test edge drivers using smartthings test suite 30 mins start sdc24 health research stack establish a health research system using samsung health research stack 30 mins start sdc24 samsung pay samsung wallet integrate samsung pay web checkout with merchant sites 30 mins start sdc24 samsung pay samsung wallet integrate samsung pay sdk flutter plugin into merchant apps for in-app payment 30 mins start sdc24 samsung wallet utilize add to samsung wallet service for digital cards 30 mins start sdc24 samsung wallet verify your id with samsung wallet 30 mins start sdc24 automotive create an android automotive operating system (aaos) app with payments via samsung checkout 30 mins start watch face studio apply gyro effects to a watch face using watch face studio 20 mins start smartthings matter: create a virtual device and make an open source contribution 25 mins start smartthings matter: build a matter iot app with smartthings home api 25 mins start galaxy z develop a widget for flex window 25 mins start samsung pay samsung wallet integrate in-app payment into merchant apps using samsung pay sdk 30 mins start gamedev optimize game performance with adaptive performance in unity 30 mins start gamedev galaxy z implement flex mode into a unity game 30 mins start watch face studio customize styles of a watch face with watch face studio 30 mins start watch face studio galaxy z customize flex window using good lock plugin on watch face studio 20 mins start health measure skin temperature on galaxy watch 20 mins start health transfer heart rate data from galaxy watch to a mobile device 30 mins start watch face studio design a watch face using mask and moon phase tags 30 mins start bixby smartthings control a smart bulb 30 mins start watch face studio apply conditional lines on watch faces 20 mins start health measure blood oxygen level on galaxy watch 30 mins start health measure blood oxygen level and heart rate on galaxy watch 40 mins start galaxy z implement multi-window picture-in-picture on a video player 20 mins start samsung blockchain transfer erc20 token with blockchain app 45 mins start galaxy ar emoji gamedev use ar emoji on games and 3d apps 60 mins start gamedev galaxy z implement flex mode on an unreal engine game 120 mins start smartthings integrate iot devices into the smartthings ecosystem 45 mins start watch face studio create a watch face using tag expressions 60 mins start galaxy z implement flex mode on a video player 30 mins start galaxy z implement app continuity and optimize large screen ui of a gallery app 40 mins start galaxy z configure an app to enable copy and paste in multi-window 30 mins start galaxy z configure an app to enable drag and drop in multi-window 30 mins start samsung blockchain develop a secure blockchain app 40 mins start samsung blockchain develop a blockchain shopping app 40 mins start
web
foldables and large screens new opportunities in the mobile experience design guidelines design guidelines boost your apps’ value withfoldable & large screen optimization your galaxy becomes simpler, more convenient and better optimized with one ui. one ui provides meaningful innovations and improves your everyday life, ensuring that you adapt in the ever-changing world. also, one ui continues to evolve to bring your joyful experiences to life. code labs all tags all tags galaxy z develop a widget for flex window 25 mins start gamedev galaxy z implement flex mode into a unity game 30 mins start watch face studio galaxy z customize flex window using good lock plugin on watch face studio 20 mins start galaxy z implement multi-window picture-in-picture on a video player 20 mins start gamedev galaxy z implement flex mode on an unreal engine game 120 mins start samsung internet galaxy z develop a camera web app on foldables 30 mins start galaxy z implement app continuity and optimize large screen ui of a gallery app 40 mins start galaxy z configure an app to enable drag and drop in multi-window 30 mins start galaxy z implement flex mode on a video player 30 mins start rich ui optimized for large screens! foldable devices can provide richer experiences than phones. learn how to optimize your app's ui for large screen devices in our newly updated one ui guide. read more if you don’t have any galaxy z devices try remote test lab! remote control test lab provides a foldable device for testing app even when you don’t onw one. check out the video tutorial on the remote test lab to max out your knowledge! video thumbanil blogs go to blog search this wide range of blog articles for tips and valuable know-how in developing apps related to foldable & large screen optimization. galaxy z tech docs get detailed info on the galaxy z (foldable) in the following tech documents. overview the new form factor is not the only thing notable in foldable phones. it opens new opportunities in the mobile experience. app continuity app continuity refers to the capability of an app to seamlessly restore the user state when it receives a configuration change. multi-window the ability to have multiple windows open simultaneously benefits from larger screen real estate. flex mode when your phon is partially folded, it will go into flex mode. apps will reorient to fit the screen, letting you sending messages. ux and ui considerations by considering new layouts and navigation patterns, app developers can explore opportunities to build greater experence. testing the how your app reacts to optimized ui layout, app continuity, multi-active window and flex mode. community forums & tech support ask question and find answers at forums. if you need development support, submit a support request. developer forum tech support our partners meet samsung’s partners already making use of ui optimized for large screens.
Develop GameDev
docgpuwatch 1 0 & 1 5 userguide introduction gpuwatch is a tool for observing gpu activity in your application gpuwatch is made for developers who use samsung devices to get gpu related information with the least effort detailed information is overlaid onto the screen in real-time and it is very simple to enable - no pc required in the mobile world, the key to a great user experience is performance with efficiency efficient applications have low power requirements which both improves battery life and keeps the device temperature in a comfortable range developers aim to achieve not only higher performance but also better efficiency through optimization to reach these goals it is necessary to measure the current status which is the job of a profiling tool, like gpuwatch gpuwatch gives information about gpu work, resource allocation, processor status and more so that developers can get clues about bottlenecks and the best ways to fix them then they can finally get the desired performance also, they can evaluate the application in many different environments you can utilize gpuwatch in lots of different ways it’s up to you what you will do using this easily accessible and informative tool supported devices device android pgpuwatch v1 0 android qgpuwatch v1 5 galaxy s9 series △* ✓ galaxy s10 series ✓ ✓ galaxy s20 series ✓ galaxy note9 series △* ✓ galaxy note10 series ✓ ✓ galaxy note20 series ✓ galaxy fold ✓ ✓ galaxy z flip ✓ galaxy z fold2 ✓ △only devices launched in europe and korea how to use you can easily turn on the gpuwatch overlay for your application setting → developer options → gpuwatch on select the app to profile select rendering api launch the app if nothing is shown, check whether both application name and rendering api are set correctly functionality each metric is calculated from values measured over 120ms the icon ★ marks funtionality that is supported from v1 5 onwards fps metric description cur average fps during recent 120ms med★ median value among all measured fps since launch when we put fpss in line in a ascending order, the value at the center is median median fps is calculated as a value rounded to one decimal place how to calculate30 31 38 32 29 35 → 29 30 31 32 35 38 sort → the median is 31 5 average 31,32 stb★ totalfps to stablefps ratio as a percentagestablefps medianfps +- medianfps x 0 2 gpu,cpu usage metric description cpu total usage cpu working time against total time working time + idle time considering load over all cpu cores cpu★ current usage cpu time of this process including both user time and kernel time this is the default instead of 'cpu total usage' since v1 5 gpu proportion of working cycles against total cycles of gpu gpu activity frames are captured at a regular time interval if you want to capture manually, press the volume down key metric description thumbnail thumbnail of rendered image scene info frame number since launching and draw time for the frame renderpass timeline proportion of working cycles against total cycles of gpu this graph draws a box for each renderpass with a size proportional to the draw time for the renderpass in the current frame each renderpass is separated with a transparent border for short renderpasses the index is hidden and the area is shown as a lighter color without border if many renderpasses exist, graphs are animated with marquee effect about index x,y,z of graph,x means renderpass index this may not be exactly matched with fbo id y means the count of gldrawcall gldrawarrays, gldrawelements, glclear for gles, vkcmddrawxxx for vulkan z means count of vertices vertex/fragmentactivity this graph draws the vertex and fragment workload of each renderpass the y-axis shows the proportion of the vertex or fragment cycles used compared to the total cycles available the x-axis matches the “renderpass timeline” graph above, which means that the corresponding area of the activity graph with each renderpass of the timeline graph represents the workload of that renderpass configure layout you can configure layout details including colors and visibility for each widget through the "widget settings" submenu of "gpuwatch" revision history revision date description 1 0 may 2020 initial release
Develop GameDev
docgpuwatch 2 0 userguide what's new on gpuwatch 2 0? we focused on making gpuwatch more user-friendly in 2 0 we improved in 3 aspects better readability better conveniency more information now you can use a more useful gpuwatch at any galaxy device with androidl r os appearance of overlay widgets is more readable through thick graph line and text, noticeable legend area of course, you can manipulate these details of widgets also, metrics on different categories are able to be combined into one widget as custom widget, so that you can get hints of relation between two metrics gpuwatch v2 0 measures for the context of application on the top automatically you don't need to choose a specific application to measure or rendering api for it anymore gpuwatch v2 0 makes manipulation easier through supplying a shortcut on the notification panel supported devices device android pgpuwatch v1 0 android qgpuwatch v1 5 android rgpuwatch v2 0 galaxy s9 series △* ✓ ✓ galaxy s10 series ✓ ✓ ✓ galaxy s20 series ✓ ✓ galaxy note9 series △* ✓ ✓ galaxy note10 series ✓ ✓ ✓ galaxy note20 series ✓ ✓ galaxy fold ✓ ✓ ✓ galaxy z flip ✓ ✓ galaxy z fold2 ✓ ✓ other samsung mobile phones ✓ ※ only devices launched in europe and korea how to use you can easily turn on the gpuwatch overlay for profiling your application settings ⇒ developer options ⇒ gpuwatch on launch the app to measure general settings you can change the common settings of gpuwatch with the general settings menu item description update interval each metric of widgets is calculated from values measured over a time period which are set in update interval setting background color choose a background color for widgets background transparency set up the desired transparency level for widgets graph grid show / hide grid in graph widgets margins enable / disable margins between widgets notification manipulations for quick user interaction with gpuwatch and for additional configuration of widgets, gpuwatch offers several buttons in the notification panel item description off disable gpuwatch equivalent to disabling gpuwatch from developer options of settings hide / show widgets hide/show all widgets without disabling the gpuwatch lock / unlock widgets unlock for the user to move widgets around the screen by drag action and move them to the required positions lock the position of widgets select context select egl_context or vkdevice for profiling if application has several settings open gpuwatch settings widget manipulations widgets are the key component of gpuwatch gpuwatch shows measured values through widgets when you add preset widgets that have metrics you want to measure, gpuwatch starts to measure the metrics for an application at the top also you can add/move/delete widgets freely where you want you can also use a special type widget ‘custom widget’ it’s a combination of preset widget items with this, you can have insight of relation between several metrics preset widget category gpuwatch has categorized preset widgets each category is shown in a maximum 2 types of widget graph, text but some categories such as contextinfo are shown only 1 type of widget category graph widget text widget details fps surface fps drawn by surfaceflinger a service used to composite surfaces to the display -curr avg averaged fps between two measurements during update interval load load of processor 0% means idle, 100% means fully loaded -cpu cpu load of all cores -gpu gpu load contextinformation none some useful context information -api rendering api the application is using the value formed by " graphics_api #context_number" pid process id, graphics_api can be opengl,vulkan,glov_gl or glov_vk -res surface resolution -drivertype graphic driver loaded for the application the value can be system driver or game driver how to add widgets when you add widgets, measure starts you can add widgets from presets or mix preset items to one widget using customwidget how to add preset widgets push add widgets button select 'widgets from presets' button choose widgets from the 'graph' or 'text' widget type push 'ok' how to add custom widgets push add widgets button select 'custom widget' choose metrics to combine into a custom widget from 'graph' or 'text' widget type note only metrics could be selected of the same type the maximal number of the metrics in custom graph widget can't exceed 3 this is taken into account when configuring only custom graph widget not custom text widget how to change widget position by unlocking widgets using the notification panel button, you can move widgets around the screen and finally set the best possible position when you move widgets, gpuwatch lets you know whether widgets can be set on the position the red area means the space where moving widget is impossible green area is where possible after positioning all widgets to desired, you have to lock widgets not to disturb game play how to change widget settings you can change configuration for each widget including size, color, values to observe and others there are 2 ways to access this settings menu gpuwatch main menu > widget list > touch a widget double touch a widget in unlock mode working with gpuwatch after setting all the parameters and configuring the necessary widgets, just launch an application to measure gpuwatch will start working automatically
events game, mobile
blogbeing the host of the samsung developers podcast, i have had the opportunity to interview many great game developers over the years, but had yet to immerse myself in the gaming community. that was until i attended gdc 2022, the game developers conference held at moscone center in san francisco. gdc is the premiere conference related to the gaming industry. the annual conference brings together game designers, audio producers, programmers, artists, writers, and many more industry professionals from all around the world. networking at the expo as with most conferences, networking was one of the key benefits of being at an in-person conference. walking the expo floor allowed me to chat with so many amazing tech companies to learn about the latest game development tools and services. dolby. connecting with the reps at dolby, we talked about the different ways our team at samsung can help promote dolby atmos, their simulated surround sound technology available on samsung devices, to mobile game developers. wigi. the amazing people at wigi (women in games international) are doing great things. i learned how they are impacting the global games industry to advance economic equality and diversity for women and their allies. sequence. as i explored the expo, i was on the lookout for anything related to web3 and nfts. the team at sequence told me how they are helping game developers build for web3 and the world of nfts in the marketplace/metaverse, and simplifying crypto-transactions for gamers. sessions & sessions, and more sessions the expo floor was a great quick overload of everything game-tech-related, but the sessions allowed for a more comprehensive learning environment on many diverse game development topics. during the five-day conference, i attended over 25 sessions, learning in great detail from many inspirational speakers. sessions covered everything from designing and programming, to business and marketing, and so much more. below are highlights from several of my favorite sessions. free-to-play summit: the f2p game design challenge steve meretzky, abigail rindo, fawzi mesmar, shelby moledina, amanda schuckman free-to-play has dominated the gaming market and has intrigued me because it is a big part of mobile gaming. this session was very insightful as each of the different teams pitched their game ideas, explaining how they would generate revenue within a free-to-play game. the winners of this challenge quickly became audience favorites with their animal sanctuary game concept and real-world connection, giving players the opportunity to donate to their favorite animal foundation. session description: for many years, the game design challenge was one of the most popular sessions at the gdc. now, the advisory board of the free-to-play summit is proud to revive this gdc tradition, with a free-to-play focused version of the challenge. four free-to-play designers will have been given marching orders, and tasked with designing a game around that given problem. all four will present their idea to the summit audience, followed by an audience vote. innovative thinking and lively presentations are in store for all attendees! the theme of this year's challenge is designing for minnows. free-to-play games are almost always tuned to extract most of their revenue from "whales" -- those super-fans of the game who spend the big bucks, while the other 99.9% of the game's players spend little or nothing on it. we've challenged three free-to-play designers to come up with a design for a game that will extract the vast majority of its revenue not from "whales" but from "minnows" ... players who, over their entire lifetime, spend $10 or less. lost words: beyond the page dan gabriel of all the sessions that i attended, this one touched me the most. i wanted to learn more about the importance of narrative within the gameplay but received so much more from this session. dan gabriel’s approach to public speaking felt more like a storyteller than a conference speaker. lost words: beyond the page is not just a game, but a way for people to learn about themselves as they experience the challenges of life. session description: lost words: beyond the page leads the player into a rabbit hole of emotions to emerge, weary but fulfilled, on the other side of grief. attend to see how narrative and gameplay work together to create a deep, emotional bond. how metaphors pull the player deeper into the experience and how psychology shaped a story of a girl, a gran and a fantasy world. 'unpacking': the fun behind the foley jeff van dyck i love sound and everything that comes with the production of capturing sound effects (foley). seeing how this small team (husband, wife, and daughter) took on the challenge of not only creating 14,000 unique sounds, but also integrating all 14,000 sound files into the game and how that created its own set of challenges. session description: "unpacking has 14,000 foley audio files!" was a tweet that went viral in nov 2021. audio director and composer jeff van dyck (alien isolation, total war, ea sports) takes us through the unexpected complexity he and his wife angela encountered while they produced the foley for unpacking. understanding nfts: a sea-change for f2p games jordan blackman this session was exactly what i was hoping for: more insight into the new world of nfts (non-fungible tokens), blockchain, and crypto, and how they can impact f2p (free-to-play) games. as web3 games are developed, we will see more disruption to the current f2p space because these games are powered by the player community and not a single entity. session description: nfts are a consumer-driven phenomenon growing at dizzying speeds. more than merely a new way to offer iap, nft technology is set to disrupt game fundraising, community development, social media marketing, and even the very nature of the consumer/creator relationship. in this session, game designer jordan blackman will show the surprising ways nfts are already changing the game, as well as some predictions of what is to come. 'wordle': doing the opposite of what you're meant to josh wardle i am one who has definitely been swept up in the wordle craze, and getting to hear from the creator of wordle, josh wardle ... yes that is his name, was fascinating. the story of how he created the game was simple. the stories he shared about the simple connections people were making through sharing their daily wordle were absolutely wonderful. session description: wordle went from a personal gift to a global phenomenon in 3 months. this talk explores the decisions that were made throughout its development that run contrary to conventional wisdom around building successful mobile games, from wordle's origins to its seven-figure sale to the new york times. the talk also explores the human elements and considerations of creating, growing, and selling a game, both from the perspective of the developer and the game's audience. gdc vault: stream on-demand even though the conference has concluded, many of the sessions will be available to stream on-demand through the gdc vault. sponsored sessions are available for free, while technical sessions and gdc show content will require a paid subscription. if you are looking for samsung content, be sure to check out the following samsung sessions that were presented at gdc. you can view two of the sessions on youtube and all are available on the gdc vault. game performance optimization with causal models youtube · gdc vault what if your phone's avatar is in the game or metaverse? youtube · gdc vault new gpu, the ultimate reality! gdc vault unfolding your gaming potential with galaxy gamedev gdc vault awards show celebration the high point of the conference definitely was the awards show, presented by both the independent games festival and the game developers choice awards. the evening was full of recognizing not only the amazing winners, but all those nominated and truly how creative, innovative, and engaging the past year in game development has been. you can check out the full awards show below. inscryption the big winner of the evening was inscryption, taking home not only game of the year from game developers choice awards, but also the grand prize award from igf, along with awards for excellence in audio, excellence in design, and excellence in narrative. game description: from the creator of pony island and the hex comes the latest mind melting, self-destructing love letter to video games. inscryption is an inky black card-based odyssey that blends the deckbuilding roguelike, escape-room style puzzles, and psychological horror into a blood-laced smoothie. darker still are the secrets inscrybed upon the cards... unpacking the viral game with over 14,000 sounds somehow turned the painstaking task of unpacking boxes into an experience of peace and tranquility. unpacking won both the game developers choice award for best innovation and for best audio. game description: unpacking is a zen game about the familiar experience of pulling possessions out of boxes and fitting them into a new home. part block-fitting puzzle, part home decoration, you are invited to create a satisfying living space while learning clues about the life you're unpacking. over the course of eight house moves, you are given a chance to experience a sense of intimacy with a character you never see and a story you're never told. papetura the game i am most excited about is papetura, winners of igf’s excellence in visual art award. the mysterious and artistically quirky world is an absolute pleasure for the eyes, playing out in a real-life, stop-motion world. game description: papetura is an atmospheric point & click adventure game, handcrafted entirely out of paper. little creatures pape and tura will face monsters that will try to burn down their beloved paper world. closing these are just a few of the many highlights i experienced during gdc this past year. explore the gdc vault for yourself to experience gdc 2022 and impact the game developer community with whatever your expertise may be. see you at gdc 2023, march 20-24 in san francisco! official conference photos were made available via the gdc flickr account: www.flickr.com/photos/officialgdc. learn more about gaming trends and samsung’s participation at this year’s game developers conference here. be sure to also follow us on @samsung_dev to keep up-to-date on the latest developer news, and keep an eye on our blogs for other helpful resources. you can also sign up for the samsung developer program to take advantage of exclusive benefits and access helpful developer resources.
Tony Morelan
events iot, health, game, design, mobile, galaxy watch, foldable
blogthe samsung developer conference 2023 (sdc23) happened on october 5, 2023, at moscone north in san francisco and online. among the many exciting activities at the conference for developers and tech enthusiasts, code lab offered a unique opportunity to learn about the latest samsung sdks and tools. code lab is a hands-on learning experience, providing participants with a platform to explore the diverse world of samsung development. code lab activities are accessible for developers of all skill levels and interests, ensuring that everyone, from beginners to experts, can find something exciting to explore. covering a wide array of topics within the code lab, the conference catered to the diverse interests of the participants. here's a quick look at some of the sdc23 topics: 1. smartthings participants had the chance to build a matter iot app using the smartthings home api and create virtual devices that they could control using the smartthings app or their own iot apps. they also learned how to develop a smartthings find-compatible device. these topics are all about connecting and enhancing the smart home experience. 2. galaxy z participants, who are interested in foldable technology, were able to develop a widget for the flex window. this topic opens new possibilities in app design and user interaction. 3. samsung wallet participants learned to integrate the "add to samsung wallet" button into sample partner services. they also learned to implement in-app payment into a sample merchant app using the samsung pay sdk. these topics focus on enhancing the mobile wallet experience for samsung users. 4. gamedev game developers and enthusiasts had the opportunity to optimize game performance with adaptive performance in unity. they also learned to implement flex mode into unity games for foldable phones. these topics offer insights into the gaming industry's latest trends and technologies. 5. watch face studio code lab also provided an activity for participants to create a watch face design with customized styles using watch face studio. participants also learned how to convert the watch face design for galaxy z flip5's flex window display using the good lock plugin. 6. samsung health the health-focused code lab topics covered measuring skin temperature on galaxy watch and transferring heart rate data from galaxy watch to a mobile device with the samsung privileged health sdk. participants also learned how to create health research apps using the samsung health stack. these topics provide valuable insights into the health and fitness tech landscape. from creating virtual devices to building health-related apps, participants left the conference with new knowledge they could apply to their development projects. the samsung developer conference is a celebration of innovation and collaboration in the tech world. with a diverse range of topics in code lab, participants were equipped with the tools and knowledge to push the boundaries of what is possible in samsung's ecosystem. though sdc23 has ended, the innovation lives on! whether you missed the event or just want to try other activities, you can visit the code lab page anytime, anywhere. we can't wait to see you and the innovations that will emerge from this conference in the coming years. see you at sdc24!
Christopher Marquez
tutorials game
bloggame performance can vary between different android devices, because they use different chipsets, and gpus based on different mali architectures. your game may render at 60fps on a galaxy s20+, but how will it perform on the galaxy a51 5g? the a51 5g has good specs, but you may want to consider runtime changes based on the underlying hardware if your game is pushing the limits on flagship hardware. similarly, you may need to optimize your game to run at lower frame rates on the galaxy j and galaxy m models, which are often found in emerging markets. players quickly lose interest in a game if they experience drops in frame rate, or slow load times. they won’t play your game on long journeys if it drains battery or overheats the device. to reliably deploy a game globally, and ensure a great user experience, you need to performance test on a wide range of android devices. the arm mobile studio family of performance analysis tools provide games studios with a comprehensive game analysis workflow for android, giving information and advice at appropriate levels of detail, for technical artists, graphics developers, performance analysts and project leaders. monitor cpu and gpu activity arm streamline captures a comprehensive profile of your game running on an unrooted android device, and visualizes the cpu and gpu performance counter activity as you run your test scenario. you can see exactly how the cpu and gpu workloads are handled by the device, which helps you locate problem areas that might explain frame rate drops or thermal problems. however, spotting performance issues using streamline can be time-consuming, unless you know exactly what you’re looking for. your team may have varying levels of experience or expertise, so interpreting this data can be difficult. introducing performance advisor arm performance advisor is a lightweight reporting tool that transforms a streamline capture into a simple report that describes how your game performed on the device, and alerts you to problem areas that you should consider optimizing. it can be used by your whole team on a regular basis, to spot trends and diagnose problems early in the development cycle, when you’re best placed to do something about it. performance advisor reports the application frame rate, cpu load and gpu load, as well as content metrics about the workload running on the mali gpu. if it detects problematic areas, performance advisor tells you whether it’s the cpu or gpu that is struggling to process your application, and links to optimization advice to help you rectify it. a frame rate analysis chart shows how the application performed over time. the background color of the chart indicates how the game performed. when you’re hitting your target frame rate, the chart background is green. in this example, most of the chart is blue, telling us the gpu in the device is struggling to process fragment workloads. performance advisor can capture screenshots of your game, at the point that fps drops below a given threshold. this helps you to identify which content might be causing the problem. this provides valuable context when debugging and can allow some common elements to be spotted if repeated slowdowns occur. you can then investigate these frames more closely with graphics analyzer, to see exactly which graphics api calls were executing at that point. if you want to separately evaluate how different parts of the game performed, for example, loading screens, menu selection, and different levels or gameplay scenarios, you can annotate these regions in your game so that performance advisor can report data for them independently. performance budgeting because they have different performance expectations, it’s a good idea to set your own performance budgets for each device. for example, if you know the top frequency for the gpu in the device, and you have a target frame rate, you can calculate the absolute limit of gpu cost per frame. gpu cost per frame = gpu top frequency / target frame rate when you generate a report with performance advisor, you can pass in your performance budgets, which are then shown in the charts, so you can easily see if you’ve broken one. in the example below, we can see correlation between high numbers of execution engine cycles and drops in fps. performance advisor tells us that the gpu is busy with arithmetic operations, and that the shaders could be too complex. the report provides a link to an advice page on the arm developer website, that explains how to reduce arithmetic load in shaders. more charts include key performance metrics such as cpu cycles per frame and gpu bandwidth per frame, reported for read and write access. there are also charts showing the content workload , draw calls, primitives and pixels per frame, and the level of overdraw per pixel. download an example performance advisor report automated performance analysis it’s far easier to fix problems as they arise than is it to patch problems later on. performance advisor’s key application performance metrics are useful to monitor over daily runs, to see how changes to your application affect performance during development. arm mobile studio professional includes headless ci support - so you can easily deploy large-scale automated performance testing across multiple devices. using a device farm with a ci workflow, you can generate performance data and optimization advice automatically, every night, for several android devices. as you check in code, you can easily monitor how your content performs against your performance budgets over time, and raise alerts when you start to approach or break those budgets. the professional edition also enables you to build bespoke data dashboards from the data that been collected. performance advisor's machine-readable json reports can be imported into any json-compatible database and visualization platform, such as the elk stack. compare metrics between test runs to quickly determine which changes impacted performance, and which type of workload is the likely cause for a regression. query the data and compare performance against specific targets to identify optimization next steps. read more about how to integrate arm mobile studio into a ci workflow on the arm developer website. resources arm publishes various resources on their developer website, to help you optimize performance: optimization advice – quick reference to help you avoid common problems. mali best practices guide – comprehensive guide describing in detail how to ensure your content runs well on mali gpus. developer guides such as those for technical artists, covering best practises for geometry, textures, materials and shaders. download the mali gpu datasheet to see the different features and capabilities of arm mali gpus from the midgard-based mali-t720, to the latest valhall-based mali-g78. for detailed descriptions of all the performance counters you can analyze in each mali gpu refer to the mali gpu counter reference. get arm mobile studio arm mobile studio is free to use for interactive performance analysis. to use it headlessly in your ci workflow, you need an arm mobile studio professional license. download arm mobile studio
Arm Developer
tutorials game, mobile
blogwith the increasing popularity of foldable phones such as the galaxy z fold3 and galaxy z flip3, apps on these devices are adopting its foldable features. in this blog, you can get started on how to utilize these foldable features on android game apps. we focus on creating a java file containing an implementation of the android jetpack windowmanager library that can be imported into game engines like unity or unreal engine. this creates an interface allowing developers to retrieve information about the folding feature on the device. at the end of this blog, you can go deeper in learning by going to code lab. android jetpack windowmanager android jetpack, in their own words, is "a suite of libraries to help developers follow best practices, reduce boilerplate code, and write code that works consistently across android versions and devices so that developers can focus on the code they care about." windowmanager is one of these libraries, and is intended to help application developers support new device form factors and multi-window environments. the library had its 1.0.0 release in january 2022 for targeted foldable devices. according to its documentation, future versions will be extended to more display types and window features. creating the android jetpack windowmanager setup as previously mentioned, we are creating a java file that can be imported into either unity or unreal engine 4, to create an interface for retrieving information on the folding feature and pass it over to the native or engine side of your applications. set up the foldablehelper class and data storage class create a file called foldablehelper.java in visual studio or any source code editor. let's start off by giving it a package name of package com.samsung.android.gamedev.foldable; next, let's import all the necessary libraries and classes in this file: //android imports import android.app.activity; import android.graphics.rect; import android.os.handler; import android.os.looper; import android.util.log; //android jetpack windowmanager imports import androidx.annotation.nonnull; import androidx.core.util.consumer; import androidx.window.java.layout.windowinfotrackercallbackadapter; import androidx.window.layout.displayfeature; import androidx.window.layout.foldingfeature; import androidx.window.layout.windowinfotracker; import androidx.window.layout.windowlayoutinfo; import androidx.window.layout.windowmetrics; import androidx.window.layout.windowmetricscalculator; //java imports import java.util.list; import java.util.concurrent.executor; start by creating a class, foldablehelper, that is going to contain all of our helper functions. let's then create variables to store a callback object as well as windowinfotrackercallbackadapter and windowmetricscalculator. let's also create a temporary declaration of the native function to pass the data from java to the native side of application once we start working in the game engines. public class foldablehelper { private static layoutstatechangecallback layoutstatechangecallback; private static windowinfotrackercallbackadapter wit; private static windowmetricscalculator wmc; public static native void onlayoutchanged(foldablelayoutinfo resultinfo); } let's create a storage class to hold the data received from the windowmanager library. an instance of this class will also be passed to the native code to transfer the data. public static class foldablelayoutinfo { public static int undefined = -1; // hinge orientation public static int hinge_orientation_horizontal = 0; public static int hinge_orientation_vertical = 1; // state public static int state_flat = 0; public static int state_half_opened = 1; // occlusion type public static int occlusion_type_none = 0; public static int occlusion_type_full = 1; rect currentmetrics = new rect(); rect maxmetrics = new rect(); int hingeorientation = undefined; int state = undefined; int occlusiontype = undefined; boolean isseparating = false; rect bounds = new rect(); } initialize the windowinfotracker since we are working in java and the windowmanager library is written in kotlin, we have to use the windowinfotrackercallbackadapter. this is an interface provided by android to enable the use of the windowinfotracker from java. the window info tracker is how we receive information about any foldable features inside the window's bounds. next is to create windowmetricscalculator, which lets us retrieve the window metrics of an activity. window metrics consists of the windows' current and maximum bounds. we also create a new layoutstatechangecallback object. this object is passed into the window info tracker as a listener object and is called every time the layout of the device changes (for our purposes this is when the foldable state changes). public static void init(activity activity) { //create window info tracker wit = new windowinfotrackercallbackadapter(windowinfotracker.companion.getorcreate(activity)); //create window metrics calculator wmc = windowmetricscalculator.companion.getorcreate(); //create callback object layoutstatechangecallback = new layoutstatechangecallback(activity); } set up and attach the callback listener in this step, let's attach the layoutstatechangecallback to the windowinfotrackercallbackadapter as a listener. the addwindowlayoutinfolistener function takes three parameters: the activity to attach the listener to, an executor, and a consumer of windowlayoutinfo. we will set up the executor and consumer in a moment. the adding of the listener is kept separate from the initialization, since the first windowlayoutinfo is not emitted until activity.onstart has been called. as such, we'll likely not be needing to attach the listener until during or after onstart, but we can still set up the windowinfotracker and windowmetricscalculator ahead of time. public static void start(activity activity) { wit.addwindowlayoutinfolistener(activity, runonuithreadexecutor(), layoutstatechangecallback); } now, let's create the executor for the listener. this executor is straightforward and simply runs the command on the mainlooper of our activity. it is possible to set this up to run on a custom thread, however this is not going to be covered in this blog. for more information, we recommend checking the official documentation for the jetpack windowmanager. static executor runonuithreadexecutor() { return new myexecutor(); } static class myexecutor implements executor { handler handler = new handler(looper.getmainlooper()); @override public void execute(runnable command) { handler.post(command); } } we're going to create the basic layout of our layoutstatechangecallback. this consumes windowlayoutinfo and implements consumer<windowlayoutinfo>. for now, let's simply lay out the class and give it some functionality a little bit later. static class layoutstatechangecallback implements consumer<windowlayoutinfo> { private final activity activity; public layoutstatechangecallback(activity activity) { this.activity = activity; } } if the use of the listener is no longer needed, we want a way to remove it and the windowinfotrackercallbackadapter contains a function to do just that. public static void stop() { wit.removewindowlayoutinfolistener(layoutstatechangecallback); } this just tidies things up for us and ensures that the listener is cleaned up when we no longer need it. next, we're going to add some functionality to the layoutstatechangecallback class. we are going to process windowlayoutinfo into foldablelayoutinfo we created previously. using java native interface (jni), we are going to send that information over to the native side using the function onlayoutchanged. note: this doesn't actually do anything yet, but we cover how to set this up in unreal engine and in unity through code lab tutorials. static class layoutstatechangecallback implements consumer<windowlayoutinfo> { @override public void accept(windowlayoutinfo windowlayoutinfo) { foldablelayoutinfo resultinfo = updatelayout(windowlayoutinfo, activity); onlayoutchanged(resultinfo); } } let's implement the updatelayout function to process windowlayoutinfo and return a foldablelayoutinfo. firstly, create a foldablelayoutinfo that contains the processed information. follow this up by getting the window metrics, both maximum metrics and current metrics. private static foldablelayoutinfo updatelayout(windowlayoutinfo windowlayoutinfo, activity activity) { foldablelayoutinfo retlayoutinfo = new foldablelayoutinfo(); windowmetrics wm = wmc.computecurrentwindowmetrics(activity); retlayoutinfo.currentmetrics = wm.getbounds(); wm = wmc.computemaximumwindowmetrics(activity); retlayoutinfo.maxmetrics = wm.getbounds(); } get the displayfeatures present in the current window bounds using windowlayoutinfo.getdisplayfeatures. currently, the api only has one type of displayfeature: foldingfeatures, however in the future there will likely be more as screen types evolve. at this point, let's use a for loop to iterate through the resulting list until it finds a foldingfeature. once it detects a folding feature, it starts processing its data: orientation, state, seperation type, and its bounds. then, store these data in foldablelayoutinfo we've created at the start of the function call. you can learn more about these data by going to the jetpack windowmanager documentation. private static foldablelayoutinfo updatelayout(windowlayoutinfo windowlayoutinfo, activity activity) { foldablelayoutinfo retlayoutinfo = new foldablelayoutinfo(); windowmetrics wm = wmc.computecurrentwindowmetrics(activity); retlayoutinfo.currentmetrics = wm.getbounds(); wm = wmc.computemaximumwindowmetrics(activity); retlayoutinfo.maxmetrics = wm.getbounds(); list<displayfeature> displayfeatures = windowlayoutinfo.getdisplayfeatures(); if (!displayfeatures.isempty()) { for (displayfeature displayfeature : displayfeatures) { foldingfeature foldingfeature = (foldingfeature) displayfeature; if (foldingfeature != null) { if (foldingfeature.getorientation() == foldingfeature.orientation.horizontal) { retlayoutinfo.hingeorientation = foldablelayoutinfo.hinge_orientation_horizontal; } else { retlayoutinfo.hingeorientation = foldablelayoutinfo.hinge_orientation_vertical; } if (foldingfeature.getstate() == foldingfeature.state.flat) { retlayoutinfo.state = foldablelayoutinfo.state_flat; } else { retlayoutinfo.state = foldablelayoutinfo.state_half_opened; } if (foldingfeature.getocclusiontype() == foldingfeature.occlusiontype.none) { retlayoutinfo.occlusiontype = foldablelayoutinfo.occlusion_type_none; } else { retlayoutinfo.occlusiontype = foldablelayoutinfo.occlusion_type_full; } retlayoutinfo.isseparating = foldingfeature.isseparating(); retlayoutinfo.bounds = foldingfeature.getbounds(); return retlayoutinfo; } } } return retlayoutinfo; } if there's no folding feature detected, it simply returns the foldablelayoutinfo without setting its data leaving it with undefined (-1) values. conclusion the java file you have now created should be usable in new or existing unity and unreal engine projects, to provide access to the information on the folding feature. continue learning about it by going to the code lab tutorials showing how to use the file created here, to implement flex mode detection and usage in game applications. additional resources on the samsung developers site the samsung developers site has many resources for developers looking to build for and integrate with samsung devices and services. stay in touch with the latest news by creating a free account and subscribing to our monthly newsletter. visit the marketing resources page for information on promoting and distributing your apps. finally, our developer forum is an excellent way to stay up-to-date on all things related to the galaxy ecosystem.
Lochlann Henry Ramsay-Edwards
success story game, mobile
blogintroduction in recent years, the mobile game market has been growing fast. along with hardware upgrades, the implementation of mobile games is more complicated, and the loading process and the display of some scenes consume a lot of cpu and gpu resources. so samsung worked with game vendors, including the tencent game performance amelioration (mtgpa) team, to improve game user experience based on scenesdk. scenesdk focuses on performance optimization, with the game vendor's cooperation, by combining the abilities of device manufacturers to control hardware resources and the abilities of games to sync up scenario information. it can maximize the game experience. currently it includes many items such as scene guarantee and frequency reduction notification. it can get game information, send a mobile device's status to a game and supports 40+ games, including many popular games. optimization solutions scene protection scene protection divides a game into different game scenes according to certain rules (such as coarse-grained loading, lobby, single game round, ultimate kill, aiming and shooting, and much more), and then provide finer-grained performance guarantees for different game scenes. considering that hardware resources are limited, if the protection of all the scenes is the same, the actual protection effect is not ideal because the hardware is fully loaded at the beginning. the system high temperature protection is triggered quickly, the cpu and gpu are forced to reduce frequency, and the rendering performance will be even worse. therefore, the game could send game events, like loading, starting, lobby or scene loading, to scenesdk on the mobile device's side. the game information is sent in the json format {sceneid: value}. it’s flexible and can be extended with more sceneids if needed. the main value information is shown in the table below. after getting the scene info, the scenesdk service changes cpu/gpu frequency to improve game performance based on different game scenarios. during gameplay, it is necessary to classify the scene according to the importance level. the strategy of hierarchical protection is slightly different according to the underlying adjustment capabilities of each manufacturer, but the core is the highest-level scene, which is fully protected. for the protection of different levels of scenes, the effect can be shown as below. for the highest priority (critical) scene, the fps (frames per second) is more stable. for the lowest level scene, fps slowly declines without affecting the experience. at the same time, the protection of the scene switching is also effective in real time. as shown in the following figure, when the scene level is switched from low (1) to critical (3), the cpu frequency starts to increase, and the fps also gradually starts to increase. the comparison is as follows: frequency reduction notification the frequency reduction notification is to inform the game of the system cpu frequency reduction. the extent of cpu frequency reduction varies from manufacturer to manufacturer. such adjustments inevitably result in equipment that just meets the performance needs or already fails to meet the performance needs of the game so that it starts to freeze or becomes stuck. therefore, if the game can adjust the configuration items related to performance consumption instantly, by temporarily reducing or disabling some functions and using such notifications, a stuck game can be avoided to ensure the best player experience. variable refresh rate a key part of the technology used in the latest samsung flagship model is the ability to not only run conventional mobile refresh rate limits, but also to dynamically modify them based on game requirements at runtime. a common misconception is that if a device has the ability to run at 120 fps, it should always do so. however, most games support multiple options that are lower than 120 fps or don’t support 120 fps. if a game does not support 120 fps, a 120 fps refresh rate is not required and may cost more power consumption. the ideal approach is to make use of the maximum refresh rate only when it has the greatest benefit. so when the game sends the json string {“target fps”: value} to the mobile device, the device changes the refresh rate to save power by not running in the highest refresh rate. summary this table shows the benefit of using scenesdk at launch time. this table shows better performance during two rounds of testing with scenesdk on and scenesdk off. this table shows lower power consumption when dynamic refresh rate is applied. overall, scenesdk could be a good option to improve your game performance. please feel free to contact us if you need more information.
Xiangguo Qi
We use cookies to improve your experience on our website and to show you relevant advertising. Manage you settings for our cookies below.
These cookies are essential as they enable you to move around the website. This category cannot be disabled.
These cookies collect information about how you use our website. for example which pages you visit most often. All information these cookies collect is used to improve how the website works.
These cookies allow our website to remember choices you make (such as your user name, language or the region your are in) and tailor the website to provide enhanced features and content for you.
These cookies gather information about your browser habits. They remember that you've visited our website and share this information with other organizations such as advertisers.
You have successfully updated your cookie preferences.