implement flex mode on a unity game objective learn how to implement flex mode on a unity game using android jetpack windowmanager and unity's java native interface (jni) wrapper. overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps. when the phone is partially folded, it will go into flex mode. apps will reorient to fit the screen, letting you watch videos or play games without holding the phone. for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate. unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode. to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard. set up your environment you will need the following: unity hub with unity 2020.3.31f1 or later (must have android build support) visual studio or any source code editor samsung galaxy foldable device: galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available sample code here is a sample project for you to start coding in this code lab. download it and start your learning experience! flex mode on unity sample code (43.39 kb) start your project after downloading the sample project files, follow the steps below to open your project: launch the unity hub. click projects > open. locate the unzipped project folder and click open to add the project to the hub and open in the editor. notethe sample project was created in unity 2020.3.31f1. if you prefer using a different unity version, click choose another editor version when prompted and select a higher version of unity. configure android player settings to ensure that the project runs smoothly on the android platform, configure the player settings as follows: go to file > build settings. under platform, choose android and click switch platform. wait until this action finishes importing necessary assets and compiling scripts. then, click player settings to open the project settings window. go to player > other settings and scroll down to see target api level. set it to api level 31 as any less than this will result in a dependency error regarding an lstar variable. you can set the minimum api level on lower levels without any problem. next, in the resolution and presentation settings, enable resizable window. it is also recommended that render outside safe area is enabled to prevent black bars on the edges of the screen. lastly, enable the custom main manifest, custom main gradle template, and custom gradle properties template in the publishing settings. after closing the project settings window, check for the new folder structure created within your assets in the project window. the newly created android folder contains androidmanifest.xml, gradletemplate.properties, and maintemplate.gradle files. import the foldablehelper and add dependencies foldablehelper is a java file that you can use in different projects. it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments. before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device (flat for normal mode and half-opened for flex mode), window size, and orientation of the fold on the screen. download the foldablehelper.java file here: foldablehelper.java (171.80 kb) to import the foldablehelper.java file and add dependencies to the project, follow the steps below: in assets > plugins > android, right-click and select import new asset. locate and choose the foldablehelper.java file, then click import. next, open the gradletemplate.properties file to any source code editor like visual studio and add the following lines below the **additional_properties** marker. android.useandroidx = true android.enablejetifier = true useandroidx sets the project to use the appropriate androidx libraries instead of support libraries. enablejetifier automatically migrates existing third-party libraries to use androidx by rewriting their binaries. lastly, open the maintemplate.gradle file and add the dependencies for the artifacts needed for the project. **apply_plugins** dependencies { implementation filetree(dir: 'libs', include: ['*.jar']) implementation "androidx.appcompat:appcompat:1.4.1" implementation "androidx.core:core:1.7.0" implementation "androidx.core:core-ktx:1.7.0" implementation "androidx.window:window:1.0.0" implementation "androidx.window:window-java:1.0.0" implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.0" **deps**} noteyou may update the version of these dependencies when necessary, but be aware that there might be significant changes. create a new playeractivity to implement flex mode on your applications, you must make necessary changes to the activity. since it is impossible to access and change the original unityplayeractivity, you need to create a new playeractivity that inherits from the original. to do this: create a new file named foldableplayeractivity.java and import it into the android folder, same as when you imported the foldablehelper.java file. to extend the built-in playeractivity from unity, write below code in the foldableplayeractivity.java file. package com.unity3d.player; import android.os.bundle; import com.unity3d.player.unityplayeractivity; import com.samsung.android.gamedev.foldable.foldablehelper; import android.util.log; public class foldableplayeractivity extends unityplayeractivity { @override protected void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); foldablehelper.init(this); } @override protected void onstart() { super.onstart(); foldablehelper.start(this); } @override protected void onstop() { super.onstop(); foldablehelper.stop(); } } oncreate() calls the foldablehelper.init() to ensure that the windowinfotracker and metrics calculator gets created as soon as possible. onstart() calls the foldablehelper.start() since the first windowlayoutinfo doesn't get created until onstart(). onstop() calls the foldablehelper.stop() to ensure that when the application closes, the listener gets cleaned up. after creating the foldableplayeractivity, ensure that the game uses it. open the androidmanifest.xml file and change the activity name to the one you've just created. <activity android:name="foldableplayeractivity" android:theme="@style/unitythemeselector"> … </activity> store foldablelayoutinfo data to flexproxy implement a native listener that receives calls from java when the device state changes by following these steps: use the androidjavaproxy provided by unity in its jni implementation. androidjavaproxy is a class that implements a java interface, so the next thing you need to do is create an interface in the foldablehelper.java file. public interface windowinfolayoutlistener { void onchanged(foldablelayoutinfo layoutinfo); } this interface replaces the temporary native function. therefore, remove the code below: public static native void onlayoutchanged(foldablelayoutinfo resultinfo); then, go to assets > scripts, and right-click to create a new c# script inside called flexproxy.cs. inside this script, create the flexproxy class inheriting from androidjavaproxy. public class flexproxy : androidjavaproxy { } in flexproxy class, define the variables needed to store the data from foldablelayoutinfo and use enumerators for the folded state, hinge orientation, and occlusion type. for the various bounds, use unity's rectint type. also, use a boolean to store whether the data has been updated or not. public enum state { undefined, flat, half_opened }; public enum orientation { undefined, horizontal, vertical }; public enum occlusiontype { undefined, none, full }; public state state = state.undefined; public orientation orientation = orientation.undefined; public occlusiontype occlusiontype = occlusiontype.undefined; public rectint foldbounds; public rectint currentmetrics; public rectint maxmetrics; public bool hasupdated = false; next, define what java class the flexproxy is going to implement by using the interface's fully qualified name as below: public flexproxy() : base("com.samsung.android.gamedev.foldable.foldablehelper$windowinfolayoutlistener") { } com.samsung.android.gamedev.foldable is the package name of the foldablehelper.java file. foldablehelper$windowinfolayoutlistener is the class and interface name separated by a $. after linking the proxy to the java interface, create a helper method to simplify java to native conversions. private rectint converttorectint(androidjavaobject rect) { if(rect != null) { var left = rect.get<int>("left"); var top = rect.get<int>("top"); var width = rect.call<int>("width"); var height = rect.call<int>("height"); return new rectint(xmin: left, ymin: top, width: width, height: height); } else { return new rectint(-1, -1, -1, -1); } } this method takes a java rect object and converts it into a unity c# rectint. now, use this converttorectint() function for the onchanged() function: public void onchanged(androidjavaobject layoutinfo) { foldbounds = converttorectint(layoutinfo.get<androidjavaobject>("bounds")); currentmetrics = converttorectint(layoutinfo.get<androidjavaobject>("currentmetrics")); maxmetrics = converttorectint(layoutinfo.get<androidjavaobject>("maxmetrics")); orientation = (orientation)(layoutinfo.get<int>("hingeorientation") + 1); state = (state)(layoutinfo.get<int>("state") + 1); occlusiontype = (occlusiontype)(layoutinfo.get<int>("occlusiontype") + 1); hasupdated = true; } attach flexproxy to foldablehelper in this step, you need to attach the flexproxy to the java implementation. modify the foldablehelper.java and foldableplayeractivity.java files as follows: in the foldablehelper.java file, create a variable in foldablehelper class where you can store the listener. private static windowinfolayoutlistener listener = null; create a method to receive the native listener. public static void attachnativelistener(windowinfolayoutlistener nativelistener){ listener = nativelistener; } to ensure that the listener is actually used, modify the layoutstatechangecallback to: call the listener.onchanged, the androidjavaproxy version of onchanged function, instead of calling the existing onchanged function in the java file; and check if the listener exists at the time of calling. @override public void accept(windowlayoutinfo windowlayoutinfo) { if (listener != null){ foldablelayoutinfo resultinfo = updatelayout(windowlayoutinfo, activity); listener.onchanged(resultinfo); } } finally, in the foldableplayeractivity.java file, import windowinfolayoutlistener. import com.samsung.android.gamedev.foldable.foldablehelper.windowinfolayoutlistener; then, create a new method in foldableplayeractivity to pass the native listener to foldablehelper. public void attachunitylistener(windowinfolayoutlistener listener){ foldablehelper.attachnativelistener(listener); } implement native flex mode this section focuses on creating the flex mode split-screen effect on the game’s user interface (ui). create a new c# script in the scripts folder called flexmodehelper.cs. after creating the script, define the variables you need for this implementation. public class flexmodehelper : monobehaviour { private flexproxy windowmanagerlistener; [serializefield] private camera maincamera; [serializefield] private camera subcamera; [serializefield] private gameobject flat_ui_panel; [serializefield] private gameobject flex_ui_panel; private recttransform flex_ui_top; private recttransform flex_ui_bottom; screenorientation currentorientation; bool isfold = false; bool isflip = false; float landscapefov = 65; float portraitfov; flexproxy object is the callback object which receives the foldablelayoutinfo from java. maincamera and subcamera are two cameras creating the split-screen effect. however, flex mode does not require the use of two cameras. so, if you only need one viewport, you can remove the subcamera and replace it with any component you want. gameobjects, namely flat_ui_panel and flex_ui_panel, are the parent objects of two ui designs: flat mode (full screen) and flex mode (split screen). recttransforms, namely flex_ui_top and flex_ui_bottom, are the two child objects within flex_ui_panel to serve as the top screen and bottom screen. screenorientation object handles the field of view (fov) changes on the cover screen of the galaxy z fold series (except galaxy fold) and the screen of non-foldable devices. isfold and isflip are booleans to store whether the app runs on a galaxy z fold or z flip device and help create a consistent fov across both devices. landscapefov and portraitfov are two fov values to keep a consistent fov across both orientations of the device. noteidentification of the device where the app is running is not necessary. yet, it is recommended if you want to know how to customize the fov of the game concerning the device model. next, construct a start() method where you: create a flexproxy object and pass it into attachunitylistener on the activity using unity’s jni implementation; turn off the subcamera initially, assuming that the game starts on normal mode; identify which device the app is running using systeminfo.devicemodel. the model number of the galaxy z fold series starts with "sm-f9", while it's "sm-f7" for the galaxy z flip series; calculate the portraitfov from the landscapefov and the camera aspect ratio; set the initial fov depending on the device's orientation using unity's screen.orientation; and retrieve the transforms of the flex ui top and bottom panels. void start() { windowmanagerlistener = new flexproxy(); using (androidjavaclass javaclass = new androidjavaclass("com.unity3d.player.unityplayer")) { using (androidjavaobject activity = javaclass.getstatic<androidjavaobject>("currentactivity")) { activity.call("attachunitylistener", windowmanagerlistener); } } subcamera.enabled = false; string devicemodel = systeminfo.devicemodel; if (devicemodel.contains("sm-f9")) { isfold = true; } else { isflip = devicemodel.contains("sm-f7"); } portraitfov = camera.horizontaltoverticalfieldofview(landscapefov, maincamera.aspect); currentorientation = screen.orientation; if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { maincamera.fieldofview = portraitfov; } flex_ui_top = (recttransform)flex_ui_panel.transform.getchild(0); flex_ui_bottom = (recttransform)flex_ui_panel.transform.getchild(1); } after the game starts, assign a task to flexmodehelper to check if there is an update in the windowmanagerlistener. the windowmanagerlistener receives a call from java when there is a change in the folded state. if a change occurs, update the currentorientation and run the updateflexmode() method. alternatively, if the listener hasn't updated, check to see if the screen has changed orientation. since the cover screen has no folding feature, flexproxy will not update as the callback won't trigger. instead, store the screenorientation and compare if it matches the current screen orientation. otherwise, change the fov because the device has just rotated. once we have figured out if the device has rotated, update the fov of the maincamera based on whether it's in landscape or portrait mode. void update() { if (windowmanagerlistener.hasupdated) { currentorientation = screen.orientation; updateflexmode(); } else { if (screen.orientation != currentorientation) { currentorientation = screen.orientation; if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { maincamera.fieldofview = portraitfov; } } } } create the updateflexmode() method to adjust the game ui according to the folded state of the device. void updateflexmode() { } in this method, check if the folded state is half_opened. if so, enable the subcamera for it to start rendering to the bottom screen and switch the active ui panel to the one created for flex mode (flex_ui_panel). if(windowmanagerlistener.state == flexproxy.state.half_opened) { subcamera.enabled = true; flex_ui_panel.setactive(true); flat_ui_panel.setactive(false); then, check whether the orientation of the fold is horizontal. if (windowmanagerlistener.orientation == flexproxy.orientation.horizontal) { notefor this sample game, splitting the screen isn’t ideal vertically from a user experience (ux) point of view. for this code lab activity, split the screen only on the horizontal fold (top and bottom screen). if you want to split the screen vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis. so, if the device is on flex mode and horizontal fold, adjust the ui to place the maincamera at the top and subcamera at the bottom of the screen. locate the normalized location of the foldbounds. float foldratiotop = (float)windowmanagerlistener.foldbounds.ymin / windowmanagerlistener.currentmetrics.height; float foldratiobot = (float)windowmanagerlistener.foldbounds.ymax / windowmanagerlistener.currentmetrics.height; use these to set the render areas of the maincamera and subcamera above and below the foldbounds. maincamera.rect = new rect(0, foldratiotop, 1, foldratiotop); subcamera.rect = new rect(0, 0, 1, foldratiobot); next, ensure that the fovs are consistent across the two uis. reset the field of view of each camera, and use the foldratio to ensure that the size of objects in flex mode appears roughly the same as in flat mode. also, use two slightly different fovs for the galaxy fold and flip devices. if (isfold) { maincamera.fieldofview = (landscapefov * foldratiotop); subcamera.fieldofview = (landscapefov * foldratiobot); } else { if(isflip) { maincamera.fieldofview = (landscapefov); subcamera.fieldofview = (landscapefov); } } finally, update the two child objects of the flex_ui to ensure that they line up with the position of the fold. flex_ui_top.anchormin = new vector2(0, foldratiotop); flex_ui_top.anchormax = new vector2(1, 1); flex_ui_bottom.anchormin = new vector2(0, 0); flex_ui_bottom.anchormax = new vector2(1, foldratiobot); however, if the device is not on flex mode and the orientation of the fold is not horizontal, then run the restoreflatmode() method. notify the flexproxy object that its data has been used. } else { restoreflatmode(); } } else { restoreflatmode(); } windowmanagerlistener.hasupdated = false; } create the restoreflatmode() function where you: set both cameras to render to the entire screen; disable the subcamera; and disable the flex ui and enable the flat ui. void restoreflatmode() { maincamera.rect = new rect(0, 0, 1, 1); subcamera.rect = new rect(0, 0, 1, 1); subcamera.enabled = false; flex_ui_panel.setactive(false); flat_ui_panel.setactive(true); also, check to see if the folded state is undefined, which means that the app is running on the cover screen of a galaxy z fold device. if so, treat the screen as a rectangle and use either the landscapefov or the portraitfov depending on the orientation. additionally, since the app might have been initially opened on the main screen of a galaxy fold device, recalculate the portraitfov using the aspect ratio of the cover screen. if the folded state is not undefined, then the app is opened on the main screen of either a galaxy z fold or z flip. in this case, set the fov accordingly by using the isfold boolean to check. if the boolean returns true, treat the screen as a square. otherwise, treat it as a rectangle and set the fov based on the orientation. if (windowmanagerlistener.state == flexproxy.state.undefined) { if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { if(isfold) portraitfov = camera.horizontaltoverticalfieldofview(landscapefov, maincamera.aspect); maincamera.fieldofview = portraitfov; } } else { if (isfold) { maincamera.fieldofview = landscapefov; } else { if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { maincamera.fieldofview = portraitfov; } } } } set up flex scene return to the unity editor and create an empty gameobject in the scene. right-click on the sample scene > gameobject > create empty and name it flexmanager. select the flexmanager object, then drag and drop the flexmodehelper script into the inspector pane. then, select the cameras and ui panels like below: build and run the app go to file > build settings and ensure that the scenes/samplescene is selected in scenes in build. click build to build the apk. after building the apk, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement flex mode in your unity game app by yourself! if you're having trouble, you may download this file: flex mode on unity complete code (631.05 mb) to learn more, visit: www.developer.samsung.com/galaxy-z www.developer.samsung.com/galaxy-gamedev

implement flex mode on an unreal engine game objective learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface (jni). overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps. when the phone is partially folded, it will go into flex mode. apps will reorient to fit the screen, letting you watch videos or play games without holding the phone. for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate. unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode. to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard. set up your environment you will need the following: epic games launcher with unreal engine 4 or later visual studio or any source code editor samsung galaxy foldable device: galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available create and set up your project after launching unreal engine from the epic games launcher, follow the steps below to start your project: in the select or create new project window, choose games as a new project category and click next. select third person as template, then click next to proceed. noteyou can implement flex mode on any template or existing projects and use this code lab activity as a reference. in the project settings window, set the following: type of project: c++ target platform: mobile / tablet performance characteristics: scalable 3d or 2d real-time raytracing: raytracing disabled include an additional content pack: no starter content project name: tutorial_project click create project. wait for the engine to finish loading and open the unreal editor. once the project is loaded, go to edit > project settings > platforms > android. click the configure now button if the project is not yet configured for the android platform. then, proceed with the following apk packaging and build settings: a. apk packaging set target sdk version to 30. set orientation to full sensor. change the maximum supported aspect ratio to 2.8 (aspect ratio of galaxy z fold3 in decimal) to prevent black bars from appearing on the cover display. leave it if your game does not need to use the cover display. enable use display cutout region?, to prevents black bars at the edge of the main screen. otherwise, leave it unchecked. b. build disable support opengl es3.1 and enable support vulkan. notecurrently, there is a problem with opengl es and the split-screen system being investigated. the only option right now is to turn off opengl es and use vulkan instead. enable native resize event the resize event of a game when switching between displays is disabled in the engine by default. however, this behavior can be easily enabled by setting android.enablenativeresizeevent=1 in the deviceprofile. currently, the only way to create a profile for foldable devices is by creating a specific rule for each device. to save time in this code lab, enable the native resize event for all android devices instead. locate and open the tutorial_project > config folder in file explorer. inside the config folder, create a new folder named android. create a new file called androiddeviceprofiles.ini and open this file in a text editor, such as visual studio. copy below deviceprofile code to the newly created androiddeviceprofiles.ini file. [deviceprofiles] +deviceprofilenameandtypes=android,android [android deviceprofile] devicetype=android baseprofilename= +cvars=r.mobilecontentscalefactor=1.0 +cvars=slate.absoluteindices=1 +cvars=r.vulkan.delayacquirebackbuffer=2 +cvars=r.vulkan.robustbufferaccess=1 +cvars=r.vulkan.descriptorsetlayoutmode=2 ; don't enable vulkan by default. specific device profiles can set this cvar to 0 to enable vulkan. +cvars=r.android.disablevulkansupport=1 +cvars=r.android.disablevulkansm5support=1 ; pf_b8g8r8a8 +cvars=r.defaultbackbufferpixelformat=0 +cvars=android.enablenativeresizeevent=1 ; previewallowlistcvars and previewdenylistcvars are arrays of cvars that are included or excluded from being applied in mobile preview. ; if any previewallowlistcvars is set, cvars are denied by default. previewallowlistcvars=none this is a copy of the default android deviceprofile from the existing basedeviceprofiles.ini file but with the enabled nativeresizeevent console variable (cvars). notethis step is not required when you only want to implement flex mode. yet, it's recommended, to allow applications to run seamlessly from main to cover display without stretching and squashing the game, by enabling the nativeresizeevent. create a new plugin and import the foldablehelper foldablehelper is a java file that you can use in different projects. it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments. before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device (flat for normal mode and half-opened for flex mode), window size, and orientation of the fold on the screen. download the foldablehelper.java file here: foldablehelper.java (5.64 kb) to import the foldablehelper.java file to the project, follow the steps below: go to edit > plugins in the unreal editor. click the new plugin button and select blank to create a blank plugin. in the name field, type foldables_tutorial and click the create plugin button. in file explorer, locate and open tutorial_project > plugins folder. go to plugins > foldables_tutorial > source> foldables_tutorial > private and create a new folder called java. copy the foldablehelper.java file into java folder. open the tutorial_project.sln file in visual studio. in the same private folder path, add a new filter called java. right-click on the java filter and click add > existing item. locate the foldablehelper.java file, then click add to include this java file in the build. modify java activity to use foldablehelper unreal plugin language (upl) is a simple xml-based language created by epic games for manipulating xml and returning strings. using upl, you can utilize the foldablehelper.java file by modifying the java activity and related gradle files as follows: in visual studio, right-click on source > foldables_tutorial folder, then click add > new item > web > xml file (.xml). create an xml file called foldables_tutorial_upl.xml. ensure that the file location is correct before clicking add. in the newly created xml file, include the foldablehelper.java file in the build by copying the java folder to the build directory. <root xmlns:android="http://schemas.android.com/apk/res/android"> <prebuildcopies> <copydir src="$s(plugindir)/private/java" dst="$s(builddir)/src/com/samsung/android/gamedev/foldable" /> </prebuildcopies> set up the gradle dependencies in the build.gradle file by adding the following in the xml file: <buildgradleadditions> <insert> dependencies { implementation filetree(dir: 'libs', include: ['*.jar']) implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.0" implementation "androidx.core:core:1.7.0" implementation "androidx.core:core-ktx:1.7.0" implementation "androidx.appcompat:appcompat:1.4.0" implementation "androidx.window:window:1.0.0" implementation "androidx.window:window-java:1.0.0" } android{ compileoptions{ sourcecompatibility javaversion.version_1_8 targetcompatibility javaversion.version_1_8 } } </insert> </buildgradleadditions> next, modify the gameactivity: <gameactivityimportadditions> <insert> <!-- package name of foldablehelper --> import com.samsung.android.gamedev.foldable.foldablehelper; </insert> </gameactivityimportadditions> <gameactivityoncreateadditions> <insert> foldablehelper.init(this); </insert> </gameactivityoncreateadditions> <gameactivityonstartadditions> <insert> foldablehelper.start(this); </insert> </gameactivityonstartadditions> <gameactivityonstopadditions> <insert> foldablehelper.stop(); </insert> </gameactivityonstopadditions> </root> gameactivityimportadditions adds the com.samsung.android.gamedev.foldable.foldablehelper into the gameactivity with the existing imports. gameactivityoncreateadditions adds the code to the oncreate() method inside the gameactivity. gameactivityonstartadditions adds the code to the onstart() method inside the gameactivity. gameactivityonstopadditions adds the code to the onstop() method inside the gameactivity. save the xml file. then, ensure that the engine uses the upl file by modifying the foldables_tutorial.build.cs script, located in the same folder as the foldables_tutorial_upl.xml file. after the dynamicallyloadedmodulenames.addrange call, add the following: if (target.platform == unrealtargetplatform.android) { additionalpropertiesforreceipt.add("androidplugin", moduledirectory + "\\foldables_tutorial_upl.xml"); } this means that the game engine will use the upl file if the platform is android. otherwise, the foldablehelper won’t work. implement a storage struct the next thing to implement is a struct, the native version of java’s foldablelayoutinfo class. to store the data retrieved from the java code using a struct, do the following: in content browser of unreal editor, right-click on c++ classes > add/import content. then, click new c++ class. select none for the parent class and click next. name the new class as foldablelayoutinfo. assign it to the foldables_tutorial plugin. then, click create class. delete the created foldablelayoutinfo.cpp file and only keep its header file. in the header file called foldablelayoutinfo.h, set up a struct to store all needed data from the windowmanager. #pragma once #include "core.h" enum efoldstate { undefined_state, flat, half_opened }; enum efoldorientation { undefined_orientation, horizontal, vertical }; enum efoldocclusiontype { undefined_occlusion, none, full }; struct ffoldablelayoutinfo { efoldstate state; efoldorientation orientation; efoldocclusiontype occlusiontype; fvector4 foldbounds; fvector4 currentmetrics; fvector4 maxmetrics; bool isseparating; ffoldablelayoutinfo() : state(efoldstate::undefined_state), orientation(efoldorientation::undefined_orientation), occlusiontype(efoldocclusiontype::undefined_occlusion), foldbounds(-1, -1, -1, -1), currentmetrics(-1, -1, -1, -1), maxmetrics(-1, -1, -1, -1), isseparating(false) { } }; implement jni code to implement jni, create a new c++ class with no parent and name it foldables_helper. assign the class to the same plugin, then modify the c++ header and source files as follows: in the created header file (foldables_helper.h), include foldablelayoutinfo.h. #include "foldablelayoutinfo.h" then, declare a multicast_delegate to serve as a listener for passing the data from the java implementation to the rest of the engine. declare_multicast_delegate_oneparam(fonlayoutchangeddelegate, ffoldablelayoutinfo); lastly, set up the methods and member variables. class foldables_tutorial_api ffoldables_helper { public: static void init(); static bool haslistener; static fonlayoutchangeddelegate onlayoutchanged; }; moving to the source file (foldables_helper.cpp), set up the definitions for the methods and member variables created in the header file. bool ffoldables_helper::haslistener = false; fonlayoutchangeddelegate ffoldables_helper::onlayoutchanged; void ffoldables_helper::init() { haslistener = true; } now, in the same source file, create the native version of the onlayoutchanged() function created in the foldablehelper.java file. since the java onlayoutchanged() function only works on android, surround the function with an #if directive to ensure that it compiles only on android. #if platform_android #endif within this directive, copy the code below to use the jni definition of the java onlayoutchanged() function. extern "c" jniexport void jnicall java_com_samsung_android_gamedev_foldable_foldablehelper_onlayoutchanged(jnienv * env, jclass clazz, jobject jfoldablelayoutinfo) { create the ffoldablelayoutinfo to store the data retrieved from java. ffoldablelayoutinfo result; retrieve the field ids of the foldablelayoutinfo and rect objects created in the java file. //java foldablelayoutinfo field ids jclass jfoldablelayoutinfocls = env->getobjectclass(jfoldablelayoutinfo); jfieldid currentmetricsid = env->getfieldid(jfoldablelayoutinfocls, "currentmetrics", "landroid/graphics/rect;"); jfieldid maxmetricsid = env->getfieldid(jfoldablelayoutinfocls, "maxmetrics", "landroid/graphics/rect;"); jfieldid hingeorientationid = env->getfieldid(jfoldablelayoutinfocls, "hingeorientation", "i"); jfieldid stateid = env->getfieldid(jfoldablelayoutinfocls, "state", "i"); jfieldid occlusiontypeid = env->getfieldid(jfoldablelayoutinfocls, "occlusiontype", "i"); jfieldid isseparatingid = env->getfieldid(jfoldablelayoutinfocls, "isseparating", "z"); jfieldid boundsid = env->getfieldid(jfoldablelayoutinfocls, "bounds", "landroid/graphics/rect;"); jobject currentmetricsrect = env->getobjectfield(jfoldablelayoutinfo, currentmetricsid); //java rect object field ids jclass rectcls = env->getobjectclass(currentmetricsrect); jfieldid leftid = env->getfieldid(rectcls, "left", "i"); jfieldid topid = env->getfieldid(rectcls, "top", "i"); jfieldid rightid = env->getfieldid(rectcls, "right", "i"); jfieldid bottomid = env->getfieldid(rectcls, "bottom", "i"); retrieve the current windowmetrics and store it in the ffoldablelayoutinfo as an fintvector4. // currentmetrics int left = env->getintfield(currentmetricsrect, leftid); int top = env->getintfield(currentmetricsrect, topid); int right = env->getintfield(currentmetricsrect, rightid); int bottom = env->getintfield(currentmetricsrect, bottomid); // store currentmetrics rect to fvector4 result.currentmetrics = fintvector4{ left, top, right, bottom }; do the same for the other variables in the java foldablelayoutinfo. // maxmetrics jobject maxmetricsrect = env->getobjectfield(jfoldablelayoutinfo, maxmetricsid); left = env->getintfield(maxmetricsrect, leftid); top = env->getintfield(maxmetricsrect, topid); right = env->getintfield(maxmetricsrect, rightid); bottom = env->getintfield(maxmetricsrect, bottomid); //store maxmetrics rect to fvector4 result.maxmetrics = fintvector4{ left, top, right, bottom }; int hingeorientation = env->getintfield(jfoldablelayoutinfo, hingeorientationid); int state = env->getintfield(jfoldablelayoutinfo, stateid); int occlusiontype = env->getintfield(jfoldablelayoutinfo, occlusiontypeid); bool isseparating = env->getbooleanfield(jfoldablelayoutinfo, isseparatingid); // store the values to an object for unreal result.orientation = tenumasbyte<efoldorientation>(hingeorientation + 1); result.state = tenumasbyte<efoldstate>(state + 1); result.occlusiontype = tenumasbyte<efoldocclusiontype>(occlusiontype + 1); result.isseparating = isseparating; // boundsrect jobject boundsrect = env->getobjectfield(jfoldablelayoutinfo, boundsid); left = env->getintfield(boundsrect, leftid); top = env->getintfield(boundsrect, topid); right = env->getintfield(boundsrect, rightid); bottom = env->getintfield(boundsrect, bottomid); // store maxmetrics rect to fvector4 result.foldbounds = fintvector4{ left, top, right, bottom }; broadcast the result via the onlayoutchanged delegate for use in the engine. if (ffoldables_helper::haslistener) { ue_log(logtemp, warning, text("broadcast")); ffoldables_helper::onlayoutchanged.broadcast(result); } } create a player controller and two ui states this section focuses on adding a player controller and creating two user interface (ui) states for flat and flex modes. these objects are needed for the flex mode logic implementation. following are the steps to add a player controller and create two ui states : add a new player controller blueprint. in content browser, go to content > thirdpersoncpp and right-click on blueprints > add/import content > blueprint class. pick player controller as its parent class. rename it as flexplayercontroller. notethe flexplayercontroller added is generic and can be replaced by your custom player controller in an actual project. add a new c++ class with actor component as its parent class. name it as foldables_manager and assign it to the foldables_tutorial plugin. click the create class button. open the flexplayercontroller blueprint by double-clicking it. click open full blueprint editor. attach the actor component to the flexplayercontroller. in the left pane, click add component, then find and select the foldables_manager. next, create a pair of userwidget classes for the ui layouts needed: flat mode ui for the full screen or normal mode; and flex mode ui for split-screen. in add c++ class window, select the show all classes checkbox. find and pick userwidget as the parent class. then, click next. name the new user widget as flatui and attach it to the plugin. click next. repeat the process but name the new user widget as flexui. you might get an error when trying to compile stating that the userwidget is an undeclared symbol. to fix this, open the foldables_tutorial.build.cs file, and in the publicdependencymodulenames.addrange call, add "inputcore" and "umg" to the list. create a pair of blueprints made from subclasses of these two user widgets. right-click on content and create a new folder called foldui. inside the newly created folder, right-click to add a new blueprint class. in all classes, choose flatui and click the select button. rename the blueprint as bp_flatui. in the same folder, repeat the process but choose the flexui class and rename the blueprint as bp_flexui. double-click on bp_flatui and bp_flexui, then design your two uis like below to visualize switching between flat and flex mode: flat ui flex ui notethis code lab activity does not cover the steps on how to create or design ui. if you want to learn about how to create your own game design in unreal engine 4, refer to unreal motion graphics ui designer guide. implement the flex mode logic after creating the flexplayercontroller and the two ui states (bp_flatui and bp_flexui), you can now implement flex mode logic in the foldables_manager. open the foldables_manager.h and include the necessary c++ header files: #pragma once #include "coreminimal.h" #include "components/actorcomponent.h" #include "engine.h" #include "flatui.h" #include "flexui.h" #include "foldables_helper.h" #include "foldables_manager.generated.h" remove the line below to save a little bit of performance as this component doesn't need to tick. public: // called every frame virtual void tickcomponent(float deltatime, eleveltick ticktype, factorcomponenttickfunction* thistickfunction) override; set up the functions needed in foldables_manager: the constructor, a function to create the ui widgets the implementation of onlayoutchanged delegate public: // sets default values for this component's properties ufoldables_manager(); void createwidgets(); protected: // called when the game starts virtual void beginplay() override; protected: void onlayoutchanged(ffoldablelayoutinfo foldablelayoutinfo); then, set up the variables needed: references to the flat and flex ui classes references to the flat and flex ui objects mark the pointers as uproperty to ensure that garbage collection does not delete the objects they point to. tsubclassof<uuserwidget> flatuiclass; tsubclassof<uuserwidget> flexuiclass; uproperty() class uflatui* flatui; uproperty() class uflexui* flexui; finally, define a new private function restoreflatmode(), to disable flex mode and return to normal mode. private: void restoreflatmode(); moving over to foldables_manager.cpp, implement the constructor. using the constructorhelpers, find the ui classes and set the variables to store these classes. also, set the bcanevertick to false to prevent the component from ticking and remove the code block of tickcomponent() function. // sets default values for this component's properties ufoldables_manager::ufoldables_manager() { primarycomponenttick.bcanevertick = false; static constructorhelpers::fclassfinder<uflatui> flatuibpclass(text("/game/foldui/bp_flatui")); static constructorhelpers::fclassfinder<uflexui> flexuibpclass(text("/game/foldui/bp_flexui")); if (flatuibpclass.succeeded()) { flatuiclass = flatuibpclass.class; } if (flexuibpclass.succeeded()) { flexuiclass = flexuibpclass.class; } } next, set up the beginplay() function to link the delegate to the onlayoutchanged() function, to initialize the foldables_helper, and to create the widgets ready for use in the first frame. // called when the game starts void ufoldables_manager::beginplay() { super::beginplay(); ffoldables_helper::onlayoutchanged.adduobject(this, &ufoldables_manager::onlayoutchanged); ffoldables_helper::init(); createwidgets(); } set up the createwidgets() function to create the widgets using the ui classes acquired in the constructor. add the flatui widget to the viewport, assuming the app opens in normal mode until it receives the foldablelayoutinfo. void ufoldables_manager::createwidgets() { flatui = createwidget<uflatui>((aplayercontroller*)getowner(), flatuiclass, fname(text("flatui"))); flexui = createwidget<uflexui>((aplayercontroller*)getowner(), flexuiclass, fname(text("flexui"))); flatui->addtoviewport(); } afterward, create the onlayoutchanged() function, which will be called via a delegate. inside this function, check whether the device’s folded state is half_opened. if so, check whether the orientation of the fold is horizontal. void ufoldables_manager::onlayoutchanged(ffoldablelayoutinfo foldablelayoutinfo) { //if state is now flex if (foldablelayoutinfo.state == efoldstate::half_opened) { if (foldablelayoutinfo.orientation == efoldorientation::horizontal) { notefor this third person template, splitting the screen vertically isn’t ideal from a user experience (ux) point of view. for this code lab activity, split the screen only on the horizontal fold (top and bottom screen). if you want it vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis. you must also ensure that you have a flex ui object for the vertical layout. if the device is both on flex mode and horizontal fold, change the viewport to only render on the top screen using the normalized position of the folding feature. then in an asynctask on the game thread, disable the flatui and enable the flexui. however, if the device is on normal mode, then return to flat ui using restoreflatmode() function. //horizontal split float foldratio = (float)foldablelayoutinfo.foldbounds.y / (foldablelayoutinfo.currentmetrics.w - foldablelayoutinfo.currentmetrics.y); gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizex = 1.0f; gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizey = foldratio; asynctask(enamedthreads::gamethread, [=]() { if (flatui->isinviewport()) { flatui->removefromparent(); } if (!flexui->isinviewport()) { flexui->addtoviewport(0); } }); } else { restoreflatmode(); } } else { restoreflatmode(); } } reverse the flex mode implementation logic to create the restoreflatmode() function by setting the viewport to fill the screen, then disable the flexui and enable the flatui. void ufoldables_manager::restoreflatmode() { gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizex = 1.0f; gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizey = 1.0f; asynctask(enamedthreads::gamethread, [=]() { if (!flatui->isinviewport()) { flatui->addtoviewport(0); } if (flexui->isinviewport()) { flexui->removefromparent(); } }); } set up a game mode and attach the flexplayercontroller the game mode defines the game rules, scoring, and any game-specific behavior. set up the game mode in unreal editor by creating a blueprint class in the content > thirdpersoncpp > blueprints folder. pick game mode base as the parent class and rename it as flexgamemode. double-click on flexgamemode. in the drop-down menu next to player controller class, choose the flexplayercontroller. lastly, go to edit > project settings > project > maps & modes and select flexgamemode as the default gamemode. build and run the app go to edit > package project > android to build the apk. ensure that the android development environment for unreal is already set up to your computer. noteif the build failed due to corrupted build tools, consider downgrading the version to 30 or lower using the sdk manager. or, simply rename d8.bat to the name of the missing file (dx.bat) in (sdk path) > build-tools > (version number) directory and, in lib folder of the same directory, rename d8.jar to dx.jar. after packaging your android project, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement flex mode in your unreal engine game app by yourself! if you're having trouble, you may download this file: flex mode on unreal complete code (20.16 mb) to learn more, visit: www.developer.samsung.com/galaxy-z www.developer.samsung.com/galaxy-gamedev

code lab code lab is an education platform that engages anyone to understand, through a series of topics, the entirety of the sdks and tools powered by samsung. all tags all tags sdc22 bixby smartthings control a smart bulb control a smart light bulb and change its color using bixby home studio. 30 mins start sdc22 samsung pay integrate in-app payment with merchant apps learn how to integrate in-app payment with your merchant apps using samsung pay sdk. 45 mins start sdc22 samsung wallet add boarding passes, tickets, and coupons learn how to integrate the add to wallet functionality with partner services, enabling users to add boarding passes, tickets, and coupons to samsung wallet. 30 mins start sdc22 watch face studio apply conditional lines on watch faces learn how to create a watch face that responds to time and notification using conditional lines in watch face studio. 20 mins start sdc22 health measure blood oxygen level on galaxy watch create a health app for galaxy watches powered by wear os, utilizing the new samsung privileged health sdk to trigger and obtain blood oxygen level (spo2) measurement results. 30 mins start sdc22 health measure blood oxygen level and heart rate on galaxy watch create a health app for galaxy watches powered by wear os, utilizing the new samsung privileged health sdk to trigger and obtain results of simultaneous blood oxygen level (spo2) and heart rate measurements. 40 mins start sdc22 galaxy z implement multi-window picture-in-picture on a video player learn how to implement multi-window picture-in-picture (pip) feature on a video player app for a seamless viewing experience on foldable devices. 20 mins start sdc22 samsung blockchain transfer erc20 token with blockchain app develop a decentralized application (dapp) to transfer erc20 tokens between ethereum accounts using samsung blockchain platform sdk. 45 mins start sdc22 galaxy ar emoji gamedev use ar emoji on games and 3d apps learn how to use ar emoji as character of games or 3d applications using galaxy ar emoji sdk for unity. 60 mins start sdc22 samsung internet galaxy z develop a camera web app on foldables learn how to develop a camera web application that detects partially folded posture and adjusts its layout accordingly to improve user's experience on samsung foldable devices. 30 mins start sdc22 gamedev galaxy z implement flex mode on a unity game learn how to implement flex mode on a unity game using android jetpack windowmanager and unity's java native interface (jni) wrapper. 90 mins start sdc22 gamedev galaxy z implement flex mode on an unreal engine game learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface (jni). 120 mins start sdc22 smartthings integrate iot devices into the smartthings ecosystem learn how to create your own iot device using smartthings sdk for direct connected devices. 45 mins start health create a daily step counter on galaxy watch create a native app for galaxy watch, operating on wear os powered by samsung, using health platform to read your daily steps. 40 mins start health track deadlift exercise on galaxy watch create a native app for galaxy watch, operating on wear os powered by samsung, using health services to track deadlift exercise. this app measures repetition count, calories burned, and time spent during the exercise. 40 mins start watch face studio create a watch face using tag expressions learn how to create a watch face that responds based on the date, time, step count, heart rate, and battery level using tag expressions in watch face studio. 60 mins start galaxy z implement app continuity and optimize large screen ui of a gallery app learn how to apply app continuity and large screen optimization on a gallery app for seamless experience on foldable devices. 40 mins start galaxy z configure an app to enable copy and paste in multi-window learn how to implement copy and paste on a note-taking app when in multi-window mode. 30 mins start galaxy z configure an app to enable drag and drop in multi-window learn how to implement drag and drop on a note-taking app when in multi-window mode. 30 mins start gamedev optimize unity game performance learn how to optimize the performance of a demo game using the adaptive performance package from within unity. 30 mins start galaxy s pen remote implement keyevent.callback by mapping air actions learn how to implement keyevent.callback on your s pen remote by mapping the air actions on a car racing game. 30 mins start galaxy s pen remote handle s pen's raw data learn how to implement s pen remote sdk and handle raw data of s pen. 30 mins start samsung blockchain develop a secure blockchain app learn how to create your own decentralized applications (dapp) using samsung blockchain keystore sdk. 40 mins start samsung blockchain develop a blockchain shopping app create a shopping application with samsung blockchain platform sdk and pay using ethereum network. 40 mins start samsung dex open a new desktop world on pc or mac optimize your application and its input capabilities on samsung dex. 60 mins start need technical support? if you have any queries or problems while developing a sample app in code lab, login using your samsung account and submit a support request. on the support request screen, select i am a developer and i need assistance from developer support engineers, choose sample app for code lab, and provide appropriate details. go to support request

samsung developer hashtags when you publish your post in social media, include one or two of the applicable samsung hashtags listed in the tables below. hashtags for games published in galaxy store samsung games android hardware #samsunggalaxy#samsungmobile#samsung#samsunggamelauncher #gamedev#gaming#gamer#mobilegaming #mobilegamer#indiegame#indiegamedev#indiegaming#gamerlife #androidgaming#androidgamer #galaxyzflip4#galaxyzflip4bespokeedition#galaxys23#galaxybuds2#galaxybuds2pro hashtags for galaxy themes samsung design themes hardware #samsunggalaxy#samsungmobile#samsung #design#designer#android#androidui#themes#ui #androidtheme#samsungtheme#samsungthemes#galaxytheme#galaxythemes#themestore#samsungthemestore #galaxyzflip4#galaxyzflip4bespokeedition#galaxys23 hashtags for galaxy watch samsung wearable watch face hardware #samsunggalaxy#samsunggear#samsungwatch#samsung #smartwatch#wearable#clockface#dialface#wearabletech#watch#galaxyapps#watchaddict#watchnerd#watchgeek #watchfacestudio#galaxywatchfortizen#watchface#watchfaces#watchfaceoftheday#galaxywatchstudiofortizen#watchfacedesign#galaxywatchdevelopers #galaxywatch5#galaxywatch

adaptive performance v1.0adaptive performance v1.0 v2.0adaptive performance v2.0 introduction adaptive performance provides that way to manage both the thermals and performance of a game on a mobile device. samsung has partnered with unity, to provide this solution. why is managing both the thermals and performance important on a device? the answer is that today’s mobile devices have more resources than in the past, but they are still limited. as resources increased, developers wanted more natural and authentic graphics. so the game needs more resources, which naturally leads to an increase in temperature. but in this situation, the problem is that there is no information about those resources. and that means that developers couldn’t recognize the reason for performance drops. to avoid this situation, developers not only optimize the entire game, but also use settings to control various graphics qualities to target the appropriate performance on each device. but if a developer could know sufficient hardware information before the device faces a performance drop, then they can easily and effectively manage the performance of the device. samsung provides hints that can easily read the temperature and state of the device through the gamesdk, then unity processes that information and provides a way to manage thermal and performance through adaptive performance. this graph shows how adaptive performance helps sustain a high frame rate in the mega city unity demo running on the samsung galaxy s10. the blue line shows a much more stable frame rate with adaptive performance while the red line shows the behaviour before adaptive performance was added. as you can see from this result, adaptive performance makes much more stable frame rate, and it can improve the user experience. how to use adaptive performance is a package for unity engine. to use it, follow the instructions below. unity editor → window → package manager → choose package : all → install adaptive performance & adaptive performance samsung android actually that is all that is required to use adaptive performance, you don't need any setting for this. but if you need more detailed information, look at the links below. adaptive performance release note adaptive performance preview! install and user guide (korean) a deep look inside games : gamesdk and adaptive performance (english) unity blog (korean) unity blog (english) supported devices with the first launch of adaptive performance, we supported devices after galaxy s10 running android pie. after launch we have added support for all old and new samsung galaxy models with android 10. also we provide below support devices list. series models galaxy s galaxy s10e / s10 / s10+galaxy s10 lite galaxy s20 / s20+ / s20 ultra galaxy s20 lite galaxy note galaxy note9 galaxy note10 / note10+ galaxy note10 litegalaxy note20 / note20 ultra galaxy z galaxy z fold2 galaxy a galaxy a10sgalaxy a11 galaxy a21s galaxy a31 galaxy a41 galaxy a50s galaxy a51 galaxy a51 5g galaxy a71 galaxy a71 5g galaxy a80 galaxy a8 2018 galaxy a9 2018 galaxy a9 2018 galaxy a90 5g galaxy xcover galaxy xcover 4s galaxy xcover pro galaxy m galaxy m11 galaxy m30s galaxy m31s galaxy tab galaxy tab s6galaxy tab s7 galaxy tab s7+ support status for each model could be different by the selling area and the os version of the device.

gpuwatch introduction gpuwatch is a tool for observing gpu activity in your application. gpuwatch is made for developers who use samsung devices to get gpu related information with the least effort. detailed information is overlaid onto the screen in real-time. and it is very simple to enable -no pc required. in the mobile world, the key to a great user experience is performance with efficiency. efficient applications have low power requirements which both improves battery life and keeps the device temperature in a comfortable range. developers aim to achieve not only higher performance but also better efficiency through optimization. to reach these goals it is necessary to measure the current status which is the job of a profiling tool, like gpu watch. gpuwatch gives information about gpu work, resource allocation, processor status and more so that developers can get clues about bottlenecks and the best way to fix them. then they can finally get the desired performance. also they can evaluate the application in many different environments. you can utilize gpuwatch in lots of different ways. it’s up to you what you will do using this prompt and convenient tool. how to use you can easily turn on the gpuwatch overlay for profiling your application: settings ⇒ developer options ⇒ gpuwatch on launch the app to measure for further information about functionalities, refer to the userguide page of each version. supported devices device android pgpuwatch v1.0 android qgpuwatch v1.5 android rgpuwatch v2.0 } galaxy s9 series △* ✓ ✓ galaxy s10 series ✓ ✓ ✓ galaxy s20 series ✓ ✓ galaxy note9 series △* ✓ ✓ galaxy note10 series ✓ ✓ ✓ galaxy note20 series ✓ ✓ galaxy fold ✓ ✓ ✓ galaxy z flip ✓ ✓ galaxy z fold2 ✓ ✓ other samsung mobile phones ✓ ※ only devices launched in europe and korea

foldables and large screens new opportunities in the mobile experience design guidelines design guidelines boost your apps’ value withfoldable & large screen optimization your galaxy becomes simpler, more convenient and better optimized with one ui. one ui provides meaningful innovations and improves your everyday life, ensuring that you adapt in the ever-changing world. also, one ui continues to evolve to bring your joyful experiences to life. rich ui optimized for large screens! foldable devices can provide richer experiences than phones. learn how to optimize your app's ui for large screen devices in our newly updated one ui guide. read more all tags all tags sdc22 galaxy z implement multi-window picture-in-picture on a video player learn how to implement multi-window picture-in-picture (pip) feature on a video player app for a seamless viewing experience on foldable devices. 20 mins start sdc22 gamedev galaxy z implement flex mode on an unreal engine game learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface (jni). 120 mins start sdc22 gamedev galaxy z implement flex mode on a unity game learn how to implement flex mode on a unity game using android jetpack windowmanager and unity's java native interface (jni) wrapper. 90 mins start sdc22 samsung internet galaxy z develop a camera web app on foldables learn how to develop a camera web application that detects partially folded posture and adjusts its layout accordingly to improve user's experience on samsung foldable devices. 30 mins start galaxy z implement app continuity and optimize large screen ui of a gallery app learn how to apply app continuity and large screen optimization on a gallery app for seamless experience on foldable devices. 40 mins start galaxy z configure an app to enable drag and drop in multi-window learn how to implement drag & drop on a note-taking app when in multi-window mode. 30 mins start galaxy z implement flex mode on a video player learn how to implement flex mode on a video player app using android jetpack window manager. 30 mins start if you don’t have any galaxy z devices try remote test lab! remote control test lab provides a foldable device for testing app even when you don’t onw one. check out the video tutorial on the remote test lab to max out your knowledge! video thumbanil blogs go to blog search this wide range of blog articles for tips and valuable know-how in developing apps related to foldable & large screen optimization. galaxy z tech docs get detailed info on the galaxy z (foldable) in the following tech documents. overview the new form factor is not the only thing notable in foldable phones. it opens new opportunities in the mobile experience. app continuity app continuity refers to the capability of an app to seamlessly restore the user state when it receives a configuration change. multi-window the ability to have multiple windows open simultaneously benefits from larger screen real estate. flex mode when your phon is partially folded, it will go into flex mode. apps will reorient to fit the screen, letting you sending messages. ux and ui considerations by considering new layouts and navigation patterns, app developers can explore opportunities to build greater experence. testing the how your app reacts to optimized ui layout, app continuity, multi-active window and flex mode. community forums & tech support ask question and find answers at forums. if you need development support, submit a support request. developer forum tech support our partners meet samsung’s partners already making use of ui optimized for large screens.

being the host of the samsung developers podcast, i have had the opportunity to interview many great game developers over the years, but had yet to immerse myself in the gaming community. that was until i attended gdc 2022, the game developers conference held at moscone center in san francisco. gdc is the premiere conference related to the gaming industry. the annual conference brings together game designers, audio producers, programmers, artists, writers, and many more industry professionals from all around the world. networking at the expo as with most conferences, networking was one of the key benefits of being at an in-person conference. walking the expo floor allowed me to chat with so many amazing tech companies to learn about the latest game development tools and services. dolby. connecting with the reps at dolby, we talked about the different ways our team at samsung can help promote dolby atmos, their simulated surround sound technology available on samsung devices, to mobile game developers. wigi. the amazing people at wigi (women in games international) are doing great things. i learned how they are impacting the global games industry to advance economic equality and diversity for women and their allies. sequence. as i explored the expo, i was on the lookout for anything related to web3 and nfts. the team at sequence told me how they are helping game developers build for web3 and the world of nfts in the marketplace/metaverse, and simplifying crypto-transactions for gamers. sessions & sessions, and more sessions the expo floor was a great quick overload of everything game-tech-related, but the sessions allowed for a more comprehensive learning environment on many diverse game development topics. during the five-day conference, i attended over 25 sessions, learning in great detail from many inspirational speakers. sessions covered everything from designing and programming, to business and marketing, and so much more. below are highlights from several of my favorite sessions. free-to-play summit: the f2p game design challenge steve meretzky, abigail rindo, fawzi mesmar, shelby moledina, amanda schuckman free-to-play has dominated the gaming market and has intrigued me because it is a big part of mobile gaming. this session was very insightful as each of the different teams pitched their game ideas, explaining how they would generate revenue within a free-to-play game. the winners of this challenge quickly became audience favorites with their animal sanctuary game concept and real-world connection, giving players the opportunity to donate to their favorite animal foundation. session description: for many years, the game design challenge was one of the most popular sessions at the gdc. now, the advisory board of the free-to-play summit is proud to revive this gdc tradition, with a free-to-play focused version of the challenge. four free-to-play designers will have been given marching orders, and tasked with designing a game around that given problem. all four will present their idea to the summit audience, followed by an audience vote. innovative thinking and lively presentations are in store for all attendees! the theme of this year's challenge is designing for minnows. free-to-play games are almost always tuned to extract most of their revenue from "whales" -- those super-fans of the game who spend the big bucks, while the other 99.9% of the game's players spend little or nothing on it. we've challenged three free-to-play designers to come up with a design for a game that will extract the vast majority of its revenue not from "whales" but from "minnows" ... players who, over their entire lifetime, spend $10 or less. lost words: beyond the page dan gabriel of all the sessions that i attended, this one touched me the most. i wanted to learn more about the importance of narrative within the gameplay but received so much more from this session. dan gabriel’s approach to public speaking felt more like a storyteller than a conference speaker. lost words: beyond the page is not just a game, but a way for people to learn about themselves as they experience the challenges of life. session description: lost words: beyond the page leads the player into a rabbit hole of emotions to emerge, weary but fulfilled, on the other side of grief. attend to see how narrative and gameplay work together to create a deep, emotional bond. how metaphors pull the player deeper into the experience and how psychology shaped a story of a girl, a gran and a fantasy world. 'unpacking': the fun behind the foley jeff van dyck i love sound and everything that comes with the production of capturing sound effects (foley). seeing how this small team (husband, wife, and daughter) took on the challenge of not only creating 14,000 unique sounds, but also integrating all 14,000 sound files into the game and how that created its own set of challenges. session description: "unpacking has 14,000 foley audio files!" was a tweet that went viral in nov 2021. audio director and composer jeff van dyck (alien isolation, total war, ea sports) takes us through the unexpected complexity he and his wife angela encountered while they produced the foley for unpacking. understanding nfts: a sea-change for f2p games jordan blackman this session was exactly what i was hoping for: more insight into the new world of nfts (non-fungible tokens), blockchain, and crypto, and how they can impact f2p (free-to-play) games. as web3 games are developed, we will see more disruption to the current f2p space because these games are powered by the player community and not a single entity. session description: nfts are a consumer-driven phenomenon growing at dizzying speeds. more than merely a new way to offer iap, nft technology is set to disrupt game fundraising, community development, social media marketing, and even the very nature of the consumer/creator relationship. in this session, game designer jordan blackman will show the surprising ways nfts are already changing the game, as well as some predictions of what is to come. 'wordle': doing the opposite of what you're meant to josh wardle i am one who has definitely been swept up in the wordle craze, and getting to hear from the creator of wordle, josh wardle ... yes that is his name, was fascinating. the story of how he created the game was simple. the stories he shared about the simple connections people were making through sharing their daily wordle were absolutely wonderful. session description: wordle went from a personal gift to a global phenomenon in 3 months. this talk explores the decisions that were made throughout its development that run contrary to conventional wisdom around building successful mobile games, from wordle's origins to its seven-figure sale to the new york times. the talk also explores the human elements and considerations of creating, growing, and selling a game, both from the perspective of the developer and the game's audience. gdc vault: stream on-demand even though the conference has concluded, many of the sessions will be available to stream on-demand through the gdc vault. sponsored sessions are available for free, while technical sessions and gdc show content will require a paid subscription. if you are looking for samsung content, be sure to check out the following samsung sessions that were presented at gdc. you can view two of the sessions on youtube and all are available on the gdc vault. game performance optimization with causal models youtube · gdc vault what if your phone's avatar is in the game or metaverse? youtube · gdc vault new gpu, the ultimate reality! gdc vault unfolding your gaming potential with galaxy gamedev gdc vault awards show celebration the high point of the conference definitely was the awards show, presented by both the independent games festival and the game developers choice awards. the evening was full of recognizing not only the amazing winners, but all those nominated and truly how creative, innovative, and engaging the past year in game development has been. you can check out the full awards show below. inscryption the big winner of the evening was inscryption, taking home not only game of the year from game developers choice awards, but also the grand prize award from igf, along with awards for excellence in audio, excellence in design, and excellence in narrative. game description: from the creator of pony island and the hex comes the latest mind melting, self-destructing love letter to video games. inscryption is an inky black card-based odyssey that blends the deckbuilding roguelike, escape-room style puzzles, and psychological horror into a blood-laced smoothie. darker still are the secrets inscrybed upon the cards... unpacking the viral game with over 14,000 sounds somehow turned the painstaking task of unpacking boxes into an experience of peace and tranquility. unpacking won both the game developers choice award for best innovation and for best audio. game description: unpacking is a zen game about the familiar experience of pulling possessions out of boxes and fitting them into a new home. part block-fitting puzzle, part home decoration, you are invited to create a satisfying living space while learning clues about the life you're unpacking. over the course of eight house moves, you are given a chance to experience a sense of intimacy with a character you never see and a story you're never told. papetura the game i am most excited about is papetura, winners of igf’s excellence in visual art award. the mysterious and artistically quirky world is an absolute pleasure for the eyes, playing out in a real-life, stop-motion world. game description: papetura is an atmospheric point & click adventure game, handcrafted entirely out of paper. little creatures pape and tura will face monsters that will try to burn down their beloved paper world. closing these are just a few of the many highlights i experienced during gdc this past year. explore the gdc vault for yourself to experience gdc 2022 and impact the game developer community with whatever your expertise may be. see you at gdc 2023, march 20-24 in san francisco! official conference photos were made available via the gdc flickr account: www.flickr.com/photos/officialgdc. learn more about gaming trends and samsung’s participation at this year’s game developers conference here. be sure to also follow us on @samsung_dev to keep up-to-date on the latest developer news, and keep an eye on our blogs for other helpful resources. you can also sign up for the samsung developer program to take advantage of exclusive benefits and access helpful developer resources.

gpuwatch 2.0 userguide what's new on gpuwatch 2.0? we focused on making gpuwatch more user-friendly in 2.0. we improved in 3 aspects. better readability better conveniency more information now you can use a more useful gpuwatch at any galaxy device with androidl r os. appearance of overlay widgets is more readable through thick graph line and text, noticeable legend area. of course, you can manipulate these details of widgets. also, metrics on different categories are able to be combined into one widget as custom widget, so that you can get hints of relation between two metrics. gpuwatch v2.0 measures for the context of application on the top automatically. you don't need to choose a specific application to measure or rendering api for it anymore. gpuwatch v2.0 makes manipulation easier through supplying a shortcut on the notification panel. supported devices device android pgpuwatch v1.0 android qgpuwatch v1.5 android rgpuwatch v2.0 galaxy s9 series △* ✓ ✓ galaxy s10 series ✓ ✓ ✓ galaxy s20 series ✓ ✓ galaxy note9 series △* ✓ ✓ galaxy note10 series ✓ ✓ ✓ galaxy note20 series ✓ ✓ galaxy fold ✓ ✓ ✓ galaxy z flip ✓ ✓ galaxy z fold2 ✓ ✓ other samsung mobile phones ✓ ※ only devices launched in europe and korea how to use you can easily turn on the gpuwatch overlay for profiling your application: settings ⇒ developer options ⇒ gpuwatch on launch the app to measure general settings you can change the common settings of gpuwatch with the general settings menu. item description update interval each metric of widgets is calculated from values measured over a time period which are set in update interval setting. background color choose a background color for widgets. background transparency set up the desired transparency level for widgets. graph grid show / hide grid in graph widgets. margins enable / disable margins between widgets. notification manipulations for quick user interaction with gpuwatch and for additional configuration of widgets, gpuwatch offers several buttons in the notification panel. item description off disable gpuwatch. equivalent to disabling gpuwatch from developer options of settings. hide / show widgets hide/show all widgets without disabling the gpuwatch. lock / unlock widgets unlock for the user to move widgets around the screen by drag action and move them to the required positions.lock the position of widgets. select context select egl_context or vkdevice for profiling if application has several. settings open gpuwatch settings. widget manipulations widgets are the key component of gpuwatch. gpuwatch shows measured values through widgets. when you add preset widgets that have metrics you want to measure, gpuwatch starts to measure the metrics for an application at the top. also you can add/move/delete widgets freely where you want. you can also use a special type widget : ‘custom widget’. it’s a combination of preset widget items. with this, you can have insight of relation between several metrics. preset widget category gpuwatch has categorized preset widgets. each category is shown in a maximum 2 types of widget: graph, text. but some categories such as contextinfo are shown only 1 type of widget. category graph widget text widget details fps(surface) fps drawn by surfaceflinger. (a service used to composite surfaces to the display)-curr avg : averaged fps between two measurements during update interval. load load of processor. 0% means idle, 100% means fully loaded.-cpu: cpu load of all cores.-gpu: gpu load. contextinformation none some useful context information.-api:rendering api the application is using. the value formed by " graphics_api #context_number"(pid:process id, graphics_api:can be opengl,vulkan,glov_gl or glov_vk)-res: surface resolution.-drivertype: graphic driver loaded for the application. the value can be system driver or game driver. how to add widgets when you add widgets, measure starts. you can add widgets from presets or mix preset items to one widget using customwidget. how to add preset widgets push add widgets button. select 'widgets from presets' button. choose widgets from the 'graph' or 'text' widget type. push 'ok'. how to add custom widgets push add widgets button. select 'custom widget'. choose metrics to combine into a custom widget from 'graph' or 'text' widget type.note: only metrics could be selected of the same type. the maximal number of the metrics in custom graph widget can't exceed 3. this is taken into account when configuring only custom graph widget not custom text widget. how to change widget position by unlocking widgets using the notification panel button, you can move widgets around the screen and finally set the best possible position. when you move widgets, gpuwatch lets you know whether widgets can be set on the position. the red area means the space where moving widget is impossible. green area is where possible. after positioning all widgets to desired, you have to lock widgets not to disturb game play. how to change widget settings you can change configuration for each widget including size, color, values to observe and others. there are 2 ways to access this settings menu. gpuwatch main menu > widget list > touch a widget double touch a widget in unlock mode. working with gpuwatch after setting all the parameters and configuring the necessary widgets, just launch an application to measure.gpuwatch will start working automatically.

gpuwatch 1.0 & 1.5 userguide introduction gpuwatch is a tool for observing gpu activity in your application. gpuwatch is made for developers who use samsung devices to get gpu related information with the least effort. detailed information is overlaid onto the screen in real-time. and it is very simple to enable - no pc required. in the mobile world, the key to a great user experience is performance with efficiency. efficient applications have low power requirements which both improves battery life and keeps the device temperature in a comfortable range. developers aim to achieve not only higher performance but also better efficiency through optimization. to reach these goals it is necessary to measure the current status which is the job of a profiling tool, like gpuwatch. gpuwatch gives information about gpu work, resource allocation, processor status and more so that developers can get clues about bottlenecks and the best ways to fix them. then they can finally get the desired performance. also, they can evaluate the application in many different environments. you can utilize gpuwatch in lots of different ways. it’s up to you what you will do using this easily accessible and informative tool. supported devices device android pgpuwatch v1.0 android qgpuwatch v1.5 galaxy s9 series △* ✓ galaxy s10 series ✓ ✓ galaxy s20 series ✓ galaxy note9 series △* ✓ galaxy note10 series ✓ ✓ galaxy note20 series ✓ galaxy fold ✓ ✓ galaxy z flip ✓ galaxy z fold2 ✓ △only devices launched in europe and korea. how to use you can easily turn on the gpuwatch overlay for your application: setting → developer options → gpuwatch on select the app to profile select rendering api launch the app if nothing is shown, check whether both application name and rendering api are set correctly. functionality each metric is calculated from values measured over 120ms. the icon ★ marks funtionality that is supported from v1.5 onwards. fps metric description cur average fps during recent 120ms med★ median value among all measured fps since launch when we put fpss in line in a ascending order, the value at the center is median. median fps is calculated as a value rounded to one decimal place. how to calculate30 31 38 32 29 35 → 29 30 31 32 35 38 sort → the median is 31.5(average(31,32)) stb★ totalfps to stablefps ratio as a percentagestablefps : medianfps +- medianfps x 0.2 gpu,cpu usage metric description cpu(total usage) cpu working time against total time (working time + idle time) considering load over all cpu cores. cpu★(current usage) cpu time of this process including both user time and kernel time.(this is the default instead of 'cpu total usage' since v1.5) gpu proportion of working cycles against total cycles of gpu. gpu activity frames are captured at a regular time interval. if you want to capture manually, press the volume down key. metric description thumbnail thumbnail of rendered image. scene info frame number since launching and draw time for the frame. renderpass timeline proportion of working cycles against total cycles of gpu. this graph draws a box for each renderpass with a size proportional to the draw time for the renderpass in the current frame. each renderpass is separated with a transparent border. for short renderpasses the index is hidden and the area is shown as a lighter color without border. if many renderpasses exist, graphs are animated with marquee effect.about index (x,y,z) of graph,x means renderpass index (this may not be exactly matched with fbo id.) y means the count of gldrawcall (gldrawarrays, gldrawelements, glclear for gles, vkcmddrawxxx for vulkan) z means count of vertices vertex/fragmentactivity this graph draws the vertex and fragment workload of each renderpass. the y-axis shows the proportion of the vertex or fragment cycles used compared to the total cycles available.the x-axis matches the “renderpass timeline” graph above, which means that the corresponding area of the activity graph with each renderpass of the timeline graph represents the workload of that renderpass. configure layout you can configure layout details including colors and visibility for each widget through the "widget settings" submenu of "gpuwatch". revision history revision date description 1.0 may 2020 initial release