Filter
-
Content Type
-
Category
Mobile/Wearable
Visual Display
Digital Appliance
Platform
Recommendations
Filter
Learn Code Lab
codelabimplement flex mode into a unity game objective learn how to implement flex mode into a unity game using android jetpack windowmanager and unity's java native interface jni wrapper overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps when the phone is partially folded, it will go into flex mode apps will reorient to fit the screen, letting you watch videos or play games without holding the phone for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard set up your environment you will need the following unity hub with unity 2022 3 5f1 or later must have android build support visual studio or any source code editor galaxy z fold2 or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample project for you to start coding in this code lab download it and start your learning experience! flex mode on unity sample code 1 17 gb start your project after downloading the sample project files, follow the steps below to open your project launch the unity hub click projects > open locate the unzipped project folder and click open to add the project to the hub and open in the editor notethe sample project was created in unity 2022 3 5f1 if you prefer using a different unity version, click choose another editor version when prompted and select a higher version of unity configure android player settings to ensure that the project runs smoothly on the android platform, configure the player settings as follows go to file > build settings under platform, choose android and click switch platform wait until this action finishes importing necessary assets and compiling scripts then, click player settings to open the project settings window go to player > other settings and scroll down to see target api level set it to api level 33 as any less than this will result in a dependency error regarding an lstar variable you can set the minimum api level on lower levels without any problem next, in the resolution and presentation settings, enable resizable window it is also recommended that render outside safe area is enabled to prevent black bars on the edges of the screen lastly, enable the custom main manifest, custom main gradle template, and custom gradle properties template in the publishing settings after closing the project settings window, check for the new folder structure created within your assets in the project window the newly created android folder contains androidmanifest xml, gradletemplate properties, and maintemplate gradle files import the foldablehelper and add dependencies foldablehelper is a java file that you can use in different projects it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device flat for normal mode and half-opened for flex mode , window size, and orientation of the fold on the screen download the foldablehelper java file here foldablehelper java 6 22 kb to import the foldablehelper java file and add dependencies to the project, follow the steps below in assets > plugins > android, right-click and select import new asset locate and choose the foldablehelper java file, then click import next, open the gradletemplate properties file to any source code editor like visual studio and add the following lines below the **additional_properties** marker android useandroidx = true android enablejetifier = true useandroidx sets the project to use the appropriate androidx libraries instead of support libraries enablejetifier automatically migrates existing third-party libraries to use androidx by rewriting their binaries lastly, open the maintemplate gradle file and add the dependencies for the artifacts needed for the project **apply_plugins** dependencies { implementation filetree dir 'libs', include ['* jar'] implementation "androidx appcompat appcompat 1 6 1" implementation "androidx core core 1 10 1" implementation "androidx core core-ktx 1 10 1" implementation "androidx window window 1 0 0" implementation "androidx window window-java 1 0 0" implementation "org jetbrains kotlin kotlin-stdlib-jdk8 1 9 0" **deps**} create a new playeractivity to implement flex mode on your applications, you must make necessary changes to the activity since it is impossible to access and change the original unityplayeractivity, you need to create a new playeractivity that inherits from the original to do this create a new file named foldableplayeractivity java and import it into the android folder, same as when you imported the foldablehelper java file to extend the built-in playeractivity from unity, write below code in the foldableplayeractivity java file package com unity3d player; import android os bundle; import com unity3d player unityplayeractivity; import com samsung android gamedev foldable foldablehelper; import com samsung android gamedev foldable foldablehelper windowinfolayoutlistener; import android util log; public class foldableplayeractivity extends unityplayeractivity { @override protected void oncreate bundle savedinstancestate { super oncreate savedinstancestate ; foldablehelper init this ; } @override protected void onstart { super onstart ; foldablehelper start this ; } @override protected void onstop { super onstop ; foldablehelper stop ; } @override protected void onrestart { super onrestart ; foldablehelper init this ; } public void attachunitylistener windowinfolayoutlistener listener { foldablehelper attachnativelistener listener, this ; } } oncreate calls the foldablehelper init to ensure that the windowinfotracker and metrics calculator gets created as soon as possible onstart calls the foldablehelper start since the first windowlayoutinfo doesn't get created until onstart onstop calls the foldablehelper stop to ensure that when the application closes, the listener gets cleaned up onrestart calls foldablehelper init when returning to the app after switching away windowinfotracker must be re-initialized; otherwise, flex mode will no longer update after creating the foldableplayeractivity, ensure that the game uses it open the androidmanifest xml file and change the activity name to the one you've just created <activity android name="com unity3d player foldableplayeractivity" android theme="@style/unitythemeselector"> … </activity> store foldablelayoutinfo data to flexproxy implement a native listener that receives calls from java when the device state changes by following these steps use the androidjavaproxy provided by unity in its jni implementation androidjavaproxy is a class that implements a java interface, so the next thing you need to do is create an interface in the foldablehelper java file public interface windowinfolayoutlistener { void onchanged foldablelayoutinfo layoutinfo ; } this interface replaces the temporary native function therefore, remove the code below from the foldablehelper java file public static native void onlayoutchanged foldablelayoutinfo resultinfo ; then, go to the assets > flex_scripts folder and right-click to create a new c# script called flexproxy cs inside this script, replace the automatically generated class with the flexproxy class inheriting from androidjavaproxy public class flexproxy androidjavaproxy { } in flexproxy class, define the variables needed to store the data from foldablelayoutinfo and use enumerators for the folded state, hinge orientation, and occlusion type for the various bounds, use unity's rectint type also, use a boolean to store whether the data has been updated or not public enum state { undefined, flat, half_opened }; public enum orientation { undefined, horizontal, vertical }; public enum occlusiontype { undefined, none, full }; public state state = state undefined; public orientation orientation = orientation undefined; public occlusiontype occlusiontype = occlusiontype undefined; public rectint foldbounds; public rectint currentmetrics; public rectint maxmetrics; public bool needsupdate = false; next, define what java class the flexproxy is going to implement by using the interface's fully qualified name as below public flexproxy base "com samsung android gamedev foldable foldablehelper$windowinfolayoutlistener" { } com samsung android gamedev foldable is the package name of the foldablehelper java file foldablehelper$windowinfolayoutlistener is the class and interface name separated by a $ after linking the proxy to the java interface, create a helper method to simplify java to native conversions private rectint converttorectint androidjavaobject rect { if rect != null { var left = rect get<int> "left" ; var top = rect get<int> "top" ; var width = rect call<int> "width" ; var height = rect call<int> "height" ; return new rectint xmin left, ymin top, width width, height height ; } else { return new rectint -1, -1, -1, -1 ; } } this method takes a java rect object and converts it into a unity c# rectint now, use this converttorectint function for the onchanged function to retrieve the information from the java object and store it in the flex proxy class public void onchanged androidjavaobject layoutinfo { foldbounds = converttorectint layoutinfo get<androidjavaobject> "bounds" ; currentmetrics = converttorectint layoutinfo get<androidjavaobject> "currentmetrics" ; maxmetrics = converttorectint layoutinfo get<androidjavaobject> "maxmetrics" ; orientation = orientation layoutinfo get<int> "hingeorientation" + 1 ; state = state layoutinfo get<int> "state" + 1 ; occlusiontype = occlusiontype layoutinfo get<int> "occlusiontype" + 1 ; needsupdate = true; } implement native flex mode this section focuses on creating the flex mode split-screen effect on the game’s ui create a new c# script in the flex_scripts folder called flexmodemanager cs after creating the script, define the variables you need for this implementation public class flexmodemanager monobehaviour { private flexproxy windowmanagerlistener; [serializefield] private camera maincamera; [serializefield] private camera skyboxcamera; [serializefield] private canvas controlscanvas; [serializefield] private canvas healthcanvas; [serializefield] private gameobject flexbg; [serializefield] private gameobject uiblur; windowmanagerlistener is the callback object which receives the foldablelayoutinfo from the foldablehelper java implementation maincamera and skyboxcamera are two cameras to modify in this project for creating a seamless flex mode implementation controlscanvas and healthcanvas are the two ui elements to manipulate for implementing the flex mode flexbg is a background image to disable in normal mode and enable in flex mode to fill the bottom screen uiblur is a background blur element used as part of the tutorial text it doesn't function properly with flex mode, so disabling it when in flex mode makes the ui looks better next, in the start method, create a new instance of the flexproxy class and attach it to the unity application's activity via the attachunitylistener function void start { windowmanagerlistener = new flexproxy ; using androidjavaclass javaclass = new androidjavaclass "com unity3d player unityplayer" { using androidjavaobject activity = javaclass getstatic<androidjavaobject> "currentactivity" { activity call "attachunitylistener", windowmanagerlistener ; } } } in the update method, check if the windowmanagerlistener has received any new data on the folded state of the device if the system needs an update, then call updateflexmode void update { if windowmanagerlistener needsupdate { updateflexmode ; } } create the updateflexmode method to enable or disable flex mode notethis project is set up for landscape mode only the implementation discussed in this code lab only covers setting up flex mode with a horizontal fold in landscape however, if you were able to follow, it should be simple to set up something similar for different fold orientations on devices such as the galaxy z flip series of devices private void updateflexmode { } check the folded state of the device via the windowmanagerlistener if the device is half_opened, implement flex mode private void updateflexmode { if windowmanagerlistener state == flexproxy state half_opened { } } to split the ui screen horizontally, set the anchor points of the controlscanvas and the healthcanvas so they are locked at the bottom screen or below the fold also, set the viewports of the maincamera and skyboxcamera to be above the fold - which is the top screen next, set the anchors for the flexbg object and enable it to fill the space behind the ui on the bottom screen deactivate the uiblur element if it exists the ui blur element is only present at level 1 of the demo game a check is necessary to ensure the flex mode manager works on the second level private void updateflexmode { if windowmanagerlistener state == flexproxy state half_opened { float lowerscreenanchormax = float windowmanagerlistener foldbounds ymin / windowmanagerlistener currentmetrics height; recttransform controlscanvastransform = controlscanvas getcomponent<recttransform> ; recttransform healthcanvastransform = healthcanvas getcomponent<recttransform> ; recttransform flexbgtransform = flexbg getcomponent<recttransform> ; controlscanvastransform anchormin = new vector2 0, 0 ; controlscanvastransform anchormax = new vector2 1, lowerscreenanchormax ; healthcanvastransform anchormin = new vector2 0, lowerscreenanchormax ; healthcanvastransform anchormax = new vector2 0, lowerscreenanchormax ; flexbgtransform anchormin = new vector2 0, 0 ; flexbgtransform anchormax = new vector2 1, lowerscreenanchormax ; float upperscreenrectheight = float windowmanagerlistener foldbounds ymax / windowmanagerlistener currentmetrics height; maincamera rect = new rect 0, upperscreenrectheight, 1, upperscreenrectheight ; skyboxcamera rect = new rect 0, upperscreenrectheight, 1, upperscreenrectheight ; flexbg setactive true ; if uiblur != null uiblur setactive false ; } } return the ui to full screen when the device is no longer in flex mode by disabling flexbg; enabling uiblur when it exists; and setting all the anchor points and viewports back to their original values and finally, inform the windowmanagerlistener that it doesn't need an update else { recttransform controlscanvastransform = controlscanvas getcomponent<recttransform> ; recttransform healthcanvastransform = healthcanvas getcomponent<recttransform> ; controlscanvastransform anchormin = new vector2 0, 0 ; controlscanvastransform anchormax = new vector2 1, 1 ; healthcanvastransform anchormin = new vector2 0, 1 ; healthcanvastransform anchormax = new vector2 0, 1 ; maincamera rect = new rect 0, 0, 1, 1 ; skyboxcamera rect = new rect 0, 0, 1, 1 ; flexbg setactive false ; if uiblur != null uiblur setactive true ; } windowmanagerlistener needsupdate = false; set up the scenes for flex mode go back to the unity editor in assets > 3dgamekit > scenes > gameplay, double-click on the level 1 scene to open it right-click in the hierarchy window and select create empty name the new gameobject as flexmanager or a similar name to reflect its purpose select the flexmanager object and click the add component button in the inspector window type in flexmodemanager, and select the script when it shows up select the relevant objects for each camera, canvas and game object as below do the same for level 2 but leave the ui blur empty build and run the app go to file > build settings click build at the bottom of the window to build the apk after building the apk, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement flex mode in your unity game app by yourself! to learn more, visit www developer samsung com/galaxy-z www developer samsung com/galaxy-gamedev
Learn Code Lab
codelaboptimize game performance with adaptive performance in unity objective learn how to optimize the performance of a demo game on samsung galaxy devices using the adaptive performance in unity the adaptive performance package provides you with tools to properly adjust you’re game and improve its overall game performance overview galaxy gamesdk delivers an interface between game application and device which helps developers optimize their games integrating unity adaptive performance with galaxy gamesdk allows unity developers to use this feature on unity editor within unity package manager and also customize their game contents by c# scripting game performance and quality settings can be adjusted in real time by identifying device performance status and heat trends moreover, using a set of simple ui controls, you can scale the quality of the game to match the device platform the latest version of adaptive performance, now uses new android apis that gather information from samsung hardware to provide more detailed insights into the device's thermal components and cpu optimization there are two providers for adaptive performance the samsung provider 5 0 utilizes apis from the gamesdk, which is supported from samsung galaxy s10 or note10 devices until newer models on the other hand, the android provider 1 2 uses the android dynamic performance framework adpf apis and is supported on devices running android 12 or higher this code lab focuses on using the samsung provider of adaptive performance thermal throttling mobile devices lack active cooling systems, causing temperatures to rise this triggers a warning to the unity subsystems, which lowers hardware demand to control heat and avoid performance degradation the ideal goal is to make performance stable with low temperature adaptive performance, primarily for performance and quality balance management, can help achieve this thermal warning the warning system implemented in gamesdk can trigger both internal unity systems and developer-defined behavior, such as disabling custom scripts and shaders quality settings it also provides you with the option to adjust the quality of the game to maintain stable performance at different warning levels you can scale the quality of your game in real time to meet the desired performance this includes changes in level of detail, animation, physics, and visual effects scalers with adaptive performance enabled, the game's fps becomes more stable and consistently higher over time this is because the game adapts its contents based on thermal warnings provided by samsung's thermal callback system the adaptive performance in unity provides developers with scalers that affect various aspects of the game frame rate resolution batching level of detail lod lookup texture lut multisample anti-aliasing msaa shadow cascade shadow distance shadow map resolution shadow quality sorting transparency view distance physics decals layer culling these general scalers can be used to scale the content based on your settings in the editor however, you may also create custom scalers that integrate with your own systems for instance, your own scalers can disable cpu-expensive scripts from running when thermal warnings are reached the latest version of adaptive performance, v5 0, includes new additions to the scalers decal scaler changes the draw distance of decals it controls how far a decal can be before not being rendered layer culling scaler adjusts the distance that expensive layers, such as transparency or water, when they start being culled it’s a useful scaler for scenes with more expensive shader calculations in every frame, unity calculates how all the physics in a level interacts with everything else, such as when the ball is dropping to the floor physics scaler adjusts the frequency at which this calculation is performed a lower frequency means fewer cpu calculations per second set up your environment you will need the following unity editor version 2022 3 visual studio or any source code editor supported samsung galaxy device remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample project for you to start coding in this code lab download it and start your learning experience! adaptive performance sample code 903 96 mb demo game the sample project contains a demo game named boat attack it is an open-source demo game provided by unity it has the following features triangles 800,000 vertices 771,000 shadow casters 133 start your project after downloading the sample project, follow the steps to open your project launch the unity hub click projects > open after locating the unzipped project folder, you can open it in unity editor it initially downloads the needed resources for your project open benchmark_island-flythrough scene found under assets notethe project was tested in unity 2022 3 it is recommended to use this version in this code lab set up adaptive performance components in window > package manager > adaptive performance, you can check if the adaptive performance is already included in the project go to edit > project settings > adaptive performance enable the initialize adaptive performance on startup and select samsung android provider enable the scalers by going to adaptive performance > samsung android > indexer settings check the scaler settings and just use the default values add adaptive performance script to your project you are going to use a script that contains examples of the adaptive performance api code it gives you access to the frame time information, thermal information, and cpu/gpu bottlenecks this script adjusts the performance based on the device's thermal state, allowing longer game time by reducing in-game demands download the adaptive performance script adaptiveperformanceconroller cs 6 15 kb simply drag and drop the script into assets folder create a new object and attach the script to the object change the frame rate based on thermal warnings get to know about thermal warnings throttling warnings allow you to know when your game is about to be forcibly throttled and have sections of lag and lowered frame rate adaptive performance provides these thermal alerts that allow you to take control and adjust settings before the performance downgrades check the registered event handler to trigger thermal status ithermalstatus thermalstatus thermalevent then, check thermalmetrics information in the handler name description value nowarning no warning is the normal warning level during standard thermal state 0 throttlingimminent if throttling is imminent, the application should perform adjustments to avoid thermal throttling 1 throttling if the application is in the throttling state, it should make adjustments to go back to normal temperature levels 2 temperaturelevel 0 0 ~ 1 0 current normalized temperature level in the range of [0, 1] a value of 0 means standard operation temperature and the device is not in a throttling state a value of 1 means that the maximum temperature of the device is reached and the device is going into or is already in throttling state temperaturetrend -1 0 ~ 1 0 current normalized temperature trend in the range of [-1, 1] a value of 1 describes a rapid increase in temperature a value of 0 describes a constant temperature a value of -1 describes a rapid decrease in temperature it takes at least 10s until the temperature trend may reflect any changes adjust the frame rate in the adaptiveperformancecontroller cs, the following code is responsible for changing the frame rate depending on the current thermal alert void onthermalevent thermalmetrics ev { switch ev warninglevel { case warninglevel nowarning application targetframerate = 30; break; case warninglevel throttlingimminent application targetframerate = 25; break; case warninglevel throttling application targetframerate = 15; break; } } change the quality settings the performance bottleneck api informs you via an enum value if there is a bottleneck so you can adjust the workload iadaptiveperformance performancestatus performancemetrics performancebottleneck namespace unityengine adaptiveperformance { public enum performancebottleneck { unknown = 0, cpu = 1, gpu = 2, targetframerate = 3 } } in the adaptiveperformancecontroller cs script, the code below is responsible for controlling the lod of the models depending on the performance bottleneck a model with multiple lod is a prime example of a quality setting that can heavily affect the games performance lowering it lessens the load on the gpu, frees up resources, and eventually prevents thermal throttling change the quality setting by adjusting lod in real time by using the performancebottleneck api switch ap performancestatus performancemetrics performancebottleneck // iadaptiveperformance performancestatus performancemetrics performancebottleneck { case performancebottleneck gpu debug logformat "[adp] performancebottleneck gpu " ; lowerlod ; break; case performancebottleneck cpu debug logformat "[adp] performancebottleneck cpu " ; break; case performancebottleneck targetframerate debug logformat "[adp] performancebottleneck targetframerate " ; break; case performancebottleneck unknown debug logformat "[adp] performancebottleneck unknown" ; break; } create a custom scaler to create custom scalers, you need to create a new class that inherits from adaptiveperformancescaler the adaptive performance package includes this class that can be added to the adaptive performance framework, which adjusts quality settings based on unity's thermal settings download this custom scaler to control the texture quality of the demo game texturescaler cs 1 17 kb simply drag and drop the downloaded file into the project test using device simulator in unity the device simulator in unity allows you to test out adaptive performance functionality without a physical mobile device go to window > general > device simulator go to edit > project settings > adaptive performance enable the initialize adaptive performance on startup and select device simulator provider in the device simulator, you can try to send thermal warnings and create artificial bottlenecks to test different behaviors of the demo game the visual scripts display the scalers available and show if it is enabled enabling a specific scaler means that the adaptive performance w you can override the scalers to test their impact on the demo game's performance and scene some scalers may not affect the scene but may improve the performance in the long run build and launch the apk go to file > build settings connect your galaxy device enable development build and select scripts only build option make sure that the autoconnect profiler is enabled to check the profiler in unity and measure the performance in build to device, click the patch button to install or update the apk in your device test using unity profiler unity has also introduced a new module in its profiler that allows you to monitor the scalers changes and extract frame time information to your editor the profiler allows you to get real-time information from the adaptive performance plugin navigate to windows > analysis > profiler set the play mode to your connected galaxy device once connected, you can analyze the frame time stats and check the state of the scalers during runtime scalers in this profiler are reacting to the thermal trend and are being raised or lowered in response to the thermal warning observe the scalers when enabled or disabled texture scaler the texture scaler is the custom script to lower the texture quality based on thermal levels when enabled and maxed out, the texture fidelity has been lowered this lowers the gpu load and memory usage lod scaler you may notice a subtle change in the appearance of the rocks, trees, and umbrella models this is due to the adaptation of the lod bias that determines which version of the models to use as the thermal levels rise, lower poly models are selected to reduce the number of triangles rendered this reduces the gpu load and minimizes thermal build-up shadow map resolution shadow map resolution creates a blurring effect on the shadows which incidentally lowers the performance load required to render them basically, lower shadow detail means fewer gpu calculations, which lead to less heat build-up check the game performance using gpuwatch gpuwatch is a profiling tool for observing gpu activity in your app the following are the common information shown by the gpuwatch fps counters current average cpu and gpu load cpu load gpu load it is very helpful in profiling your game during the post-development stage so you can further optimize to enable gpuwatch on your galaxy device you can easily reposition the gpuwatch widgets on the screen by enabling unlock widgets under the notification menu of gpuwatch you’re done! congratulations! you have successfully achieved the goal of this code lab now, you can improve and optimize your android game on samsung galaxy devices using adaptive performance in unity by learning about scalers, thermal warnings, and bottleneck apis the performance of your mobile games can be pushed further by utilizing these tools if you face any trouble, you may download this file adaptive performance complete project 1 06 gb to learn more, visit developer samsung com/galaxy-gamedev
Learn Code Lab
codelabimplement flex mode on an unreal engine game objective learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface jni overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps when the phone is partially folded, it will go into flex mode apps will reorient to fit the screen, letting you watch videos or play games without holding the phone for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard set up your environment you will need the following epic games launcher with unreal engine 4 or later visual studio or any source code editor samsung galaxy foldable device galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available create and set up your project after launching unreal engine from the epic games launcher, follow the steps below to start your project in the select or create new project window, choose games as a new project category and click next select third person as template, then click next to proceed noteyou can implement flex mode on any template or existing projects and use this code lab activity as a reference in the project settings window, set the following type of project c++ target platform mobile / tablet performance characteristics scalable 3d or 2d real-time raytracing raytracing disabled include an additional content pack no starter content project name tutorial_project click create project wait for the engine to finish loading and open the unreal editor once the project is loaded, go to edit > project settings > platforms > android click the configure now button if the project is not yet configured for the android platform then, proceed with the following apk packaging and build settings a apk packaging set target sdk version to 30 set orientation to full sensor change the maximum supported aspect ratio to 2 8 aspect ratio of galaxy z fold3 in decimal to prevent black bars from appearing on the cover display leave it if your game does not need to use the cover display enable use display cutout region?, to prevents black bars at the edge of the main screen otherwise, leave it unchecked b build disable support opengl es3 1 and enable support vulkan notecurrently, there is a problem with opengl es and the split-screen system being investigated the only option right now is to turn off opengl es and use vulkan instead enable native resize event the resize event of a game when switching between displays is disabled in the engine by default however, this behavior can be easily enabled by setting android enablenativeresizeevent=1 in the deviceprofile currently, the only way to create a profile for foldable devices is by creating a specific rule for each device to save time in this code lab, enable the native resize event for all android devices instead locate and open the tutorial_project > config folder in file explorer inside the config folder, create a new folder named android create a new file called androiddeviceprofiles ini and open this file in a text editor, such as visual studio copy below deviceprofile code to the newly created androiddeviceprofiles ini file [deviceprofiles] +deviceprofilenameandtypes=android,android [android deviceprofile] devicetype=android baseprofilename= +cvars=r mobilecontentscalefactor=1 0 +cvars=slate absoluteindices=1 +cvars=r vulkan delayacquirebackbuffer=2 +cvars=r vulkan robustbufferaccess=1 +cvars=r vulkan descriptorsetlayoutmode=2 ; don't enable vulkan by default specific device profiles can set this cvar to 0 to enable vulkan +cvars=r android disablevulkansupport=1 +cvars=r android disablevulkansm5support=1 ; pf_b8g8r8a8 +cvars=r defaultbackbufferpixelformat=0 +cvars=android enablenativeresizeevent=1 ; previewallowlistcvars and previewdenylistcvars are arrays of cvars that are included or excluded from being applied in mobile preview ; if any previewallowlistcvars is set, cvars are denied by default previewallowlistcvars=none this is a copy of the default android deviceprofile from the existing basedeviceprofiles ini file but with the enabled nativeresizeevent console variable cvars notethis step is not required when you only want to implement flex mode yet, it's recommended, to allow applications to run seamlessly from main to cover display without stretching and squashing the game, by enabling the nativeresizeevent create a new plugin and import the foldablehelper foldablehelper is a java file that you can use in different projects it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device flat for normal mode and half-opened for flex mode , window size, and orientation of the fold on the screen download the foldablehelper java file here foldablehelper java 5 64 kb to import the foldablehelper java file to the project, follow the steps below go to edit > plugins in the unreal editor click the new plugin button and select blank to create a blank plugin in the name field, type foldables_tutorial and click the create plugin button in file explorer, locate and open tutorial_project > plugins folder go to plugins > foldables_tutorial > source> foldables_tutorial > private and create a new folder called java copy the foldablehelper java file into java folder open the tutorial_project sln file in visual studio in the same private folder path, add a new filter called java right-click on the java filter and click add > existing item locate the foldablehelper java file, then click add to include this java file in the build modify java activity to use foldablehelper unreal plugin language upl is a simple xml-based language created by epic games for manipulating xml and returning strings using upl, you can utilize the foldablehelper java file by modifying the java activity and related gradle files as follows in visual studio, right-click on source > foldables_tutorial folder, then click add > new item > web > xml file xml create an xml file called foldables_tutorial_upl xml ensure that the file location is correct before clicking add in the newly created xml file, include the foldablehelper java file in the build by copying the java folder to the build directory <root xmlns android="http //schemas android com/apk/res/android"> <prebuildcopies> <copydir src="$s plugindir /private/java" dst="$s builddir /src/com/samsung/android/gamedev/foldable" /> </prebuildcopies> set up the gradle dependencies in the build gradle file by adding the following in the xml file <buildgradleadditions> <insert> dependencies { implementation filetree dir 'libs', include ['* jar'] implementation "org jetbrains kotlin kotlin-stdlib-jdk8 1 6 0" implementation "androidx core core 1 7 0" implementation "androidx core core-ktx 1 7 0" implementation "androidx appcompat appcompat 1 4 0" implementation "androidx window window 1 0 0" implementation "androidx window window-java 1 0 0" } android{ compileoptions{ sourcecompatibility javaversion version_1_8 targetcompatibility javaversion version_1_8 } } </insert> </buildgradleadditions> next, modify the gameactivity <gameactivityimportadditions> <insert> <!-- package name of foldablehelper --> import com samsung android gamedev foldable foldablehelper; </insert> </gameactivityimportadditions> <gameactivityoncreateadditions> <insert> foldablehelper init this ; </insert> </gameactivityoncreateadditions> <gameactivityonstartadditions> <insert> foldablehelper start this ; </insert> </gameactivityonstartadditions> <gameactivityonstopadditions> <insert> foldablehelper stop ; </insert> </gameactivityonstopadditions> </root> gameactivityimportadditions adds the com samsung android gamedev foldable foldablehelper into the gameactivity with the existing imports gameactivityoncreateadditions adds the code to the oncreate method inside the gameactivity gameactivityonstartadditions adds the code to the onstart method inside the gameactivity gameactivityonstopadditions adds the code to the onstop method inside the gameactivity save the xml file then, ensure that the engine uses the upl file by modifying the foldables_tutorial build cs script, located in the same folder as the foldables_tutorial_upl xml file after the dynamicallyloadedmodulenames addrange call, add the following if target platform == unrealtargetplatform android { additionalpropertiesforreceipt add "androidplugin", moduledirectory + "\\foldables_tutorial_upl xml" ; } this means that the game engine will use the upl file if the platform is android otherwise, the foldablehelper won’t work implement a storage struct the next thing to implement is a struct, the native version of java’s foldablelayoutinfo class to store the data retrieved from the java code using a struct, do the following in content browser of unreal editor, right-click on c++ classes > add/import content then, click new c++ class select none for the parent class and click next name the new class as foldablelayoutinfo assign it to the foldables_tutorial plugin then, click create class delete the created foldablelayoutinfo cpp file and only keep its header file in the header file called foldablelayoutinfo h, set up a struct to store all needed data from the windowmanager #pragma once #include "core h" enum efoldstate { undefined_state, flat, half_opened }; enum efoldorientation { undefined_orientation, horizontal, vertical }; enum efoldocclusiontype { undefined_occlusion, none, full }; struct ffoldablelayoutinfo { efoldstate state; efoldorientation orientation; efoldocclusiontype occlusiontype; fvector4 foldbounds; fvector4 currentmetrics; fvector4 maxmetrics; bool isseparating; ffoldablelayoutinfo state efoldstate undefined_state , orientation efoldorientation undefined_orientation , occlusiontype efoldocclusiontype undefined_occlusion , foldbounds -1, -1, -1, -1 , currentmetrics -1, -1, -1, -1 , maxmetrics -1, -1, -1, -1 , isseparating false { } }; implement jni code to implement jni, create a new c++ class with no parent and name it foldables_helper assign the class to the same plugin, then modify the c++ header and source files as follows in the created header file foldables_helper h , include foldablelayoutinfo h #include "foldablelayoutinfo h" then, declare a multicast_delegate to serve as a listener for passing the data from the java implementation to the rest of the engine declare_multicast_delegate_oneparam fonlayoutchangeddelegate, ffoldablelayoutinfo ; lastly, set up the methods and member variables class foldables_tutorial_api ffoldables_helper { public static void init ; static bool haslistener; static fonlayoutchangeddelegate onlayoutchanged; }; moving to the source file foldables_helper cpp , set up the definitions for the methods and member variables created in the header file bool ffoldables_helper haslistener = false; fonlayoutchangeddelegate ffoldables_helper onlayoutchanged; void ffoldables_helper init { haslistener = true; } now, in the same source file, create the native version of the onlayoutchanged function created in the foldablehelper java file since the java onlayoutchanged function only works on android, surround the function with an #if directive to ensure that it compiles only on android #if platform_android #endif within this directive, copy the code below to use the jni definition of the java onlayoutchanged function extern "c" jniexport void jnicall java_com_samsung_android_gamedev_foldable_foldablehelper_onlayoutchanged jnienv * env, jclass clazz, jobject jfoldablelayoutinfo { create the ffoldablelayoutinfo to store the data retrieved from java ffoldablelayoutinfo result; retrieve the field ids of the foldablelayoutinfo and rect objects created in the java file //java foldablelayoutinfo field ids jclass jfoldablelayoutinfocls = env->getobjectclass jfoldablelayoutinfo ; jfieldid currentmetricsid = env->getfieldid jfoldablelayoutinfocls, "currentmetrics", "landroid/graphics/rect;" ; jfieldid maxmetricsid = env->getfieldid jfoldablelayoutinfocls, "maxmetrics", "landroid/graphics/rect;" ; jfieldid hingeorientationid = env->getfieldid jfoldablelayoutinfocls, "hingeorientation", "i" ; jfieldid stateid = env->getfieldid jfoldablelayoutinfocls, "state", "i" ; jfieldid occlusiontypeid = env->getfieldid jfoldablelayoutinfocls, "occlusiontype", "i" ; jfieldid isseparatingid = env->getfieldid jfoldablelayoutinfocls, "isseparating", "z" ; jfieldid boundsid = env->getfieldid jfoldablelayoutinfocls, "bounds", "landroid/graphics/rect;" ; jobject currentmetricsrect = env->getobjectfield jfoldablelayoutinfo, currentmetricsid ; //java rect object field ids jclass rectcls = env->getobjectclass currentmetricsrect ; jfieldid leftid = env->getfieldid rectcls, "left", "i" ; jfieldid topid = env->getfieldid rectcls, "top", "i" ; jfieldid rightid = env->getfieldid rectcls, "right", "i" ; jfieldid bottomid = env->getfieldid rectcls, "bottom", "i" ; retrieve the current windowmetrics and store it in the ffoldablelayoutinfo as an fintvector4 // currentmetrics int left = env->getintfield currentmetricsrect, leftid ; int top = env->getintfield currentmetricsrect, topid ; int right = env->getintfield currentmetricsrect, rightid ; int bottom = env->getintfield currentmetricsrect, bottomid ; // store currentmetrics rect to fvector4 result currentmetrics = fintvector4{ left, top, right, bottom }; do the same for the other variables in the java foldablelayoutinfo // maxmetrics jobject maxmetricsrect = env->getobjectfield jfoldablelayoutinfo, maxmetricsid ; left = env->getintfield maxmetricsrect, leftid ; top = env->getintfield maxmetricsrect, topid ; right = env->getintfield maxmetricsrect, rightid ; bottom = env->getintfield maxmetricsrect, bottomid ; //store maxmetrics rect to fvector4 result maxmetrics = fintvector4{ left, top, right, bottom }; int hingeorientation = env->getintfield jfoldablelayoutinfo, hingeorientationid ; int state = env->getintfield jfoldablelayoutinfo, stateid ; int occlusiontype = env->getintfield jfoldablelayoutinfo, occlusiontypeid ; bool isseparating = env->getbooleanfield jfoldablelayoutinfo, isseparatingid ; // store the values to an object for unreal result orientation = tenumasbyte<efoldorientation> hingeorientation + 1 ; result state = tenumasbyte<efoldstate> state + 1 ; result occlusiontype = tenumasbyte<efoldocclusiontype> occlusiontype + 1 ; result isseparating = isseparating; // boundsrect jobject boundsrect = env->getobjectfield jfoldablelayoutinfo, boundsid ; left = env->getintfield boundsrect, leftid ; top = env->getintfield boundsrect, topid ; right = env->getintfield boundsrect, rightid ; bottom = env->getintfield boundsrect, bottomid ; // store maxmetrics rect to fvector4 result foldbounds = fintvector4{ left, top, right, bottom }; broadcast the result via the onlayoutchanged delegate for use in the engine if ffoldables_helper haslistener { ue_log logtemp, warning, text "broadcast" ; ffoldables_helper onlayoutchanged broadcast result ; } } create a player controller and two ui states this section focuses on adding a player controller and creating two user interface ui states for flat and flex modes these objects are needed for the flex mode logic implementation following are the steps to add a player controller and create two ui states add a new player controller blueprint in content browser, go to content > thirdpersoncpp and right-click on blueprints > add/import content > blueprint class pick player controller as its parent class rename it as flexplayercontroller notethe flexplayercontroller added is generic and can be replaced by your custom player controller in an actual project add a new c++ class with actor component as its parent class name it as foldables_manager and assign it to the foldables_tutorial plugin click the create class button open the flexplayercontroller blueprint by double-clicking it click open full blueprint editor attach the actor component to the flexplayercontroller in the left pane, click add component, then find and select the foldables_manager next, create a pair of userwidget classes for the ui layouts needed flat mode ui for the full screen or normal mode; and flex mode ui for split-screen in add c++ class window, select the show all classes checkbox find and pick userwidget as the parent class then, click next name the new user widget as flatui and attach it to the plugin click next repeat the process but name the new user widget as flexui you might get an error when trying to compile stating that the userwidget is an undeclared symbol to fix this, open the foldables_tutorial build cs file, and in the publicdependencymodulenames addrange call, add "inputcore" and "umg" to the list create a pair of blueprints made from subclasses of these two user widgets right-click on content and create a new folder called foldui inside the newly created folder, right-click to add a new blueprint class in all classes, choose flatui and click the select button rename the blueprint as bp_flatui in the same folder, repeat the process but choose the flexui class and rename the blueprint as bp_flexui double-click on bp_flatui and bp_flexui, then design your two uis like below to visualize switching between flat and flex mode flat ui flex ui notethis code lab activity does not cover the steps on how to create or design ui if you want to learn about how to create your own game design in unreal engine 4, refer to unreal motion graphics ui designer guide implement the flex mode logic after creating the flexplayercontroller and the two ui states bp_flatui and bp_flexui , you can now implement flex mode logic in the foldables_manager open the foldables_manager h and include the necessary c++ header files #pragma once #include "coreminimal h" #include "components/actorcomponent h" #include "engine h" #include "flatui h" #include "flexui h" #include "foldables_helper h" #include "foldables_manager generated h" remove the line below to save a little bit of performance as this component doesn't need to tick public // called every frame virtual void tickcomponent float deltatime, eleveltick ticktype, factorcomponenttickfunction* thistickfunction override; set up the functions needed in foldables_manager the constructor, a function to create the ui widgets the implementation of onlayoutchanged delegate public // sets default values for this component's properties ufoldables_manager ; void createwidgets ; protected // called when the game starts virtual void beginplay override; protected void onlayoutchanged ffoldablelayoutinfo foldablelayoutinfo ; then, set up the variables needed references to the flat and flex ui classes references to the flat and flex ui objects mark the pointers as uproperty to ensure that garbage collection does not delete the objects they point to tsubclassof<uuserwidget> flatuiclass; tsubclassof<uuserwidget> flexuiclass; uproperty class uflatui* flatui; uproperty class uflexui* flexui; finally, define a new private function restoreflatmode , to disable flex mode and return to normal mode private void restoreflatmode ; moving over to foldables_manager cpp, implement the constructor using the constructorhelpers, find the ui classes and set the variables to store these classes also, set the bcanevertick to false to prevent the component from ticking and remove the code block of tickcomponent function // sets default values for this component's properties ufoldables_manager ufoldables_manager { primarycomponenttick bcanevertick = false; static constructorhelpers fclassfinder<uflatui> flatuibpclass text "/game/foldui/bp_flatui" ; static constructorhelpers fclassfinder<uflexui> flexuibpclass text "/game/foldui/bp_flexui" ; if flatuibpclass succeeded { flatuiclass = flatuibpclass class; } if flexuibpclass succeeded { flexuiclass = flexuibpclass class; } } next, set up the beginplay function to link the delegate to the onlayoutchanged function, to initialize the foldables_helper, and to create the widgets ready for use in the first frame // called when the game starts void ufoldables_manager beginplay { super beginplay ; ffoldables_helper onlayoutchanged adduobject this, &ufoldables_manager onlayoutchanged ; ffoldables_helper init ; createwidgets ; } set up the createwidgets function to create the widgets using the ui classes acquired in the constructor add the flatui widget to the viewport, assuming the app opens in normal mode until it receives the foldablelayoutinfo void ufoldables_manager createwidgets { flatui = createwidget<uflatui> aplayercontroller* getowner , flatuiclass, fname text "flatui" ; flexui = createwidget<uflexui> aplayercontroller* getowner , flexuiclass, fname text "flexui" ; flatui->addtoviewport ; } afterward, create the onlayoutchanged function, which will be called via a delegate inside this function, check whether the device’s folded state is half_opened if so, check whether the orientation of the fold is horizontal void ufoldables_manager onlayoutchanged ffoldablelayoutinfo foldablelayoutinfo { //if state is now flex if foldablelayoutinfo state == efoldstate half_opened { if foldablelayoutinfo orientation == efoldorientation horizontal { notefor this third person template, splitting the screen vertically isn’t ideal from a user experience ux point of view for this code lab activity, split the screen only on the horizontal fold top and bottom screen if you want it vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis you must also ensure that you have a flex ui object for the vertical layout if the device is both on flex mode and horizontal fold, change the viewport to only render on the top screen using the normalized position of the folding feature then in an asynctask on the game thread, disable the flatui and enable the flexui however, if the device is on normal mode, then return to flat ui using restoreflatmode function //horizontal split float foldratio = float foldablelayoutinfo foldbounds y / foldablelayoutinfo currentmetrics w - foldablelayoutinfo currentmetrics y ; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizex = 1 0f; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizey = foldratio; asynctask enamedthreads gamethread, [=] { if flatui->isinviewport { flatui->removefromparent ; } if !flexui->isinviewport { flexui->addtoviewport 0 ; } } ; } else { restoreflatmode ; } } else { restoreflatmode ; } } reverse the flex mode implementation logic to create the restoreflatmode function by setting the viewport to fill the screen, then disable the flexui and enable the flatui void ufoldables_manager restoreflatmode { gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizex = 1 0f; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizey = 1 0f; asynctask enamedthreads gamethread, [=] { if !flatui->isinviewport { flatui->addtoviewport 0 ; } if flexui->isinviewport { flexui->removefromparent ; } } ; } set up a game mode and attach the flexplayercontroller the game mode defines the game rules, scoring, and any game-specific behavior set up the game mode in unreal editor by creating a blueprint class in the content > thirdpersoncpp > blueprints folder pick game mode base as the parent class and rename it as flexgamemode double-click on flexgamemode in the drop-down menu next to player controller class, choose the flexplayercontroller lastly, go to edit > project settings > project > maps & modes and select flexgamemode as the default gamemode build and run the app go to edit > package project > android to build the apk ensure that the android development environment for unreal is already set up to your computer noteif the build failed due to corrupted build tools, consider downgrading the version to 30 or lower using the sdk manager or, simply rename d8 bat to the name of the missing file dx bat in sdk path > build-tools > version number directory and, in lib folder of the same directory, rename d8 jar to dx jar after packaging your android project, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement flex mode in your unreal engine game app by yourself! if you're having trouble, you may download this file flex mode on unreal complete code 20 16 mb to learn more, visit www developer samsung com/galaxy-z www developer samsung com/galaxy-gamedev
Preferences Submitted
You have successfully updated your cookie preferences.