Filter
-
Content Type
-
Category
Mobile/Wearable
Visual Display
Digital Appliance
Platform
Recommendations
Filter
Learn Code Lab
codelabimplement flex mode on a video player objective learn how to implement flex mode on a video player app using android jetpack windowmanager overview the galaxy z fold4, or its previous versions, is a foldable phone with a vertical axis that opens like a book at the same time, the galaxy z flip4, or its predecessors, folds horizontally, dividing its screen into upper and lower sections when the phone is partially folded, it is called flex mode unfolding the phone will return the apps in full-screen mode in flex mode, the app's ui separates its controls on the bottom from the rest of the content on the top to fit the partially folded screen when the phone is unfolded, window components reoccupy their original positions to fit the full-screen mode, like usual smartphones it gives an outstanding experience on camera, video calls, multimedia, and other apps to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard set up your environment you will need the following java se development kit jdk 8 or later android studio latest version recommended samsung galaxy foldable device galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! flex mode sample code 15 60 mb start your project after downloading the sample code containing the project files, open your android studio and click open to open an existing project locate the downloaded android project videoplayer-main from the directory and click ok add the maven repository and windowmanager library windowmanager, a new jetpack library introduced by google, helps application developers support new device form factors to add a dependency on windowmanager, you must first add the maven central repository to the project go to gradle scripts > build gradle project video_player and enter the following in the allprojects block mavencentral then, go to gradle scripts > build gradle module video_player app and add the windowmanager library to the dependencies block implementation "androidx window window 1 0 0" implementation 'androidx lifecycle lifecycle-runtime-ktx 2 6 1' set up a flow to collect data from windowlayoutinfo in the mainactivity kt, add a lifecycle-aware coroutine to obtain information from windowlayoutinfo the coroutine executes each time the lifecycle starts lifecyclescope launch dispatchers main { lifecycle repeatonlifecycle lifecycle state started { windowinfotracker getorcreate this@mainactivity windowlayoutinfo this@mainactivity collect { newlayoutinfo -> updatestateandfeatureviews newlayoutinfo } } } noteupdatestateandfeatureviews is a user-defined function and is not included in the windowmanager library in this code lab, it needs to be called to check the device posture determine the device posture and update the ui next, in the updatestateandfeatureviews function, check the current posture of the device and update the ui accordingly for flex mode when the device is currently folded, the displayfeatures is null if newlayoutinfo displayfeatures isempty when the device is currently in flex mode or partially folded, the device posture is half_opened using the device posture and orientation, implement your flex mode ui if it state == foldingfeature state half_opened && it orientation == foldingfeature orientation vertical { updateuiforfold }else if it state == foldingfeature state half_opened && it orientation == foldingfeature orientation horizontal { updateuiforflip } notethe device orientation of a galaxy fold device is vertical, whereas a galaxy flip's device orientation is horizontal when the device is unfolded, the device posture is flat else if it state == foldingfeature state flat run the app after building the apk, you can run the optimized video player app on a foldable galaxy device and see how it adapts when the device is on flex mode however, if you don't have a physical device, you can test it on a remote test lab device watch the video below and know how to test your app via remote test lab tipfor a better testing experience using remote test lab, choose a device from a location near you you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement flex mode in your app by yourself! if you're having trouble, you may download this file flex mode complete code 9 07 mb to learn more about developing apps for galaxy foldable devices, visit www developer samsung com/galaxy-z
Learn Code Lab
codelabimplement multi-window picture-in-picture on a video player objective learn how to implement multi-window picture-in-picture pip feature on a video player app for a seamless viewing experience on foldable devices overview android allows activities of apps to launch in a small window called picture-in-picture pip since android 8 0 it is a type of multi-window mode primarily used for video playback pip allows the user to continue watching in a small window pinned to a corner of the screen while browsing content on the main screen or navigating between apps set up your environment you will need the following java se development kit jdk 8 or later android studio latest version recommended samsung galaxy fold, z fold2, z fold3, or z fold4 remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! pip video player sample code 13 30 mb start your project after downloading the sample project files, follow the steps below to open your project in android studio, click open locate the downloaded android project videoplayer_challenge from the directory and click ok configure android manifest to support pip notetake a look and check the files of your project before starting to code you can implement picture-in-picture in your app by configuring the manifest file and calling the enterpictureinpicturemode method moreover, you need to consider that the multi-window and app continuity features work correctly in pip mode to prevent the activity from restarting, handle the layout configuration change for mainactivity in the androidmanifest xml file this will ensure that it will not restart when there are layout changes from pip mode to full screen and vice versa android configchanges="screensize|smallestscreensize|screenlayout|orientation" picture-in-picture is not supported by default in android apps set android supportspictureinpicture to true in the manifest file android supportspictureinpicture="true" switch your video player to pip mode you want to trigger pip mode in your video player app when the user taps the home button to implement pip mode, go to java > com xyz codelab and make the following changes to the mainactivity kt file override fun onuserleavehint { enterpipmode } noteenterpipmode is a user-defined method you can view its full implementation in the provided sample code call enterpictureinpicturemode method in enterpipmode , you need to call enterpictureinpicturemode pictureinpictureparams method with params set in the previous lines val pictureinpictureparams = pictureinpictureparams builder setaspectratio rational build enterpictureinpicturemode pictureinpictureparams run the app after building the apk, you can now run the video player app, enhanced with a pip mode test it by playing a video while browsing the main screen or navigating between apps in multi-window mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement multi-window picture-in-picture on a video player app for foldable devices by yourself! if you're having trouble, you may download this file pip video player complete code 13 19 mb to learn more about developing apps for galaxy foldable devices, visit developer samsung com/galaxy-z
Learn Code Lab
codelabconfigure an app to enable drag and drop in multi-window objective learn how to implement drag and drop on a note-taking app when in multi-window mode overview in galaxy fold and it's latest versions, the advantage of its larger display is to split its screen and simultaneously use up to three apps in multi-window mode, you can split the screen with one window being the main focus, and the other two windows off to the side all three windows are active, not just the largest one you can multitask in either landscape or portrait mode, giving you even more flexibility when using multi-window, drag & drop is one of the useful features when multitasking drag and drop allows you to easily move data from one app to another to provide users with better multitasking experience on samsung's foldable devices, developers need to optimize their apps to work on multi-window mode set up your environment you will need the following java se development kit jdk 8 or later android studio latest version recommended samsung galaxy fold, z fold2, z fold3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! multi-window drag & drop sample code 19 27 mb start your project open android studio and click open an existing project locate the downloaded android project simplenotes_dragdrop from the directory and click ok make the app resizable to ensure that the app works in multi-window mode, you need to add an attribute in the manifest’s <activity> element if you set android resizeableactivity to true, the activity can be launched in multi-window or in pop-up view, and adapt different screen size android resizeableactivity= "true" noteif this attribute is set to true, the activity can be launched in split-screen and freeform modes otherwise, it will disable multi-window display remember to handle the changes required to fit your ux in small windows when in multi-window mode for this code lab, you do not need to worry about the ux as it is already handled register a drag event listener in newnote kt, register a drag event listener object by calling setondraglistener for both title and description view of the app title setondraglistener ondraglistener desc setondraglistener ondraglistenerdescription store the action type to a variable declare a variable to store the action type for the incoming event val action int = event getaction get and drop the text data inside the dragevent action_drop, get the item from clipdata and check the mime type if the mime type is set to text/plain, get the text value from the item object and allow the operation of drop otherwise, simply show a toast message val item item = event getclipdata getitemat 0 var mtype = event clipdescription getmimetype 0 if mtype == "text/plain" || mtype== "text/html" { // gets the text data from the item dragdata = item text tostring }else{ toast maketext applicationcontext,"operation not allowed"+mtype,toast length_long show return@ondraglistener true } tipin this code lab, to simplify the demonstration of implementing drag and drop, mime type or media type is set to plain text only run the app after building the apk, you can run the optimized note-taking app and start dragging and dropping texts between apps in multi-window mode if you don’t have any physical device, you can also test it on a remote test lab device notewatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement drag and drop in your app for your foldable device by yourself! if you're having trouble, you may download this file multi-window drag & drop complete code 19 33 mb to learn more about developing apps for galaxy foldable devices, visit www developer samsung com/galaxy-z
Learn Code Lab
codelabconfigure an app to enable copy and paste in multi-window objective learn how to implement copy and paste on a note-taking app when in multi-window mode overview in galaxy fold and it's latest versions, the advantage of its larger display is to split its screen and simultaneously use up to three apps in multi-window mode, you can split the screen with one window being the main focus, and the other two windows off to the side all three windows are active, not just the largest one you can multitask in either landscape or portrait mode, giving you even more flexibility when using multi-window, copy & paste is one of the useful features when multitasking copy and paste is a common feature used by many, to duplicate an object and place it in a desired location to provide users with better multitasking experience on samsung's foldable devices, developers need to optimize their apps to work on multi-window mode set up your environment you will need the following java se development kit jdk 8 or later android studio latest version recommended samsung galaxy fold, z fold2, z fold3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! multi-window copy & paste sample code 19 48 mb start your project open android studio and click open an existing project locate the downloaded android project simplenotes_copypaste from the directory and click ok make the app resizable to ensure that the app works in multi-window mode, you need to add an attribute in the manifest’s <activity> element if you set android resizeableactivity to true, the activity can be launched in multi-window, pop-up view and adapt different screen size android resizeableactivity= "true" noteif this attribute is set to true, the activity can be launched in split-screen and free-form modes otherwise, it will disable multi-window display remember to handle the changes required to fit your ux in small windows when in multi-window mode for this code lab, you do not need to worry about the ux as it is already handled get a reference to clipboardmanager class clipboardmanager provides methods to get and set the current primary clipboard data expressed as a clipdata object, which defines the protocol for data exchange between applications in newnote kt, get a reference to clipboardmanager class by invoking getsystemservice clipboard_service val clipboard = getsystemservice context clipboard_service as clipboardmanager complete the copy operation now, create a new clipdata of simple text using newplaintext of clipboardmanager and pass it on setprimaryclip to complete the copy operation val clip clipdata = clipdata newplaintext "simple text", textdata clipboard setprimaryclip clip get the text data from the clipboard return the clipdata by getting the primaryclip data from the clipboard from the primaryclip, acquire the item using getitemat 0 val itemtext = clipboard primaryclip? getitemat 0 ? text run the app after building the apk, you can run the optimized app and test it by copying and pasting texts between apps when in multi-window mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement multi-window features such as copy and paste in your app by yourself! if you're having trouble, you may download this file multi-window copy & paste complete code 19 53 mb to learn more about developing apps for galaxy foldable devices, visit www developer samsung com/galaxy-z
Learn Code Lab
codelabimplement app continuity and optimize large screen ui of a gallery app objective learn how to apply app continuity and large screen optimization on a gallery app for seamless experience on foldable devices overview samsung foldable devices have two different physical screens the cover display and the larger main display app continuity lets you experience a seamless transition between these displays when using an app that supports app continuity, you can unfold the device and continue where you left off with this feature, apps can run without interruption when it receives a configuration change event, such as when a device is folded or unfolded it is essential to continue the operation by keeping the same state and location of the app after the transition large screen optimization plays another significant role in terms of ux designing for foldable devices the screens of foldable devices have a wide range of aspect ratios, careful ux design is needed to suit two different size displays set up your environment you will need the following java se development kit jdk 8 or later android studio latest version recommended samsung galaxy fold, z fold2, z fold3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! app continuity sample code 2 80 mb start your project after downloading the sample code containing the project file for this code lab, open your android studio and click open an existing project locate the downloaded android project from the directory and click ok handle the configuration change to have a seamless experience when using a gallery app, the photo that is displayed after folding or unfolding the device should be the same as the previous you need to store the position or id of the photo, as android destroys and recreate the activity when folding and unfolding the phone use bundle with its key/value pair to store and restore the values in addition, you need to consider the screen rotation in order to keep the app running, as android restarts the activity of the app after the configuration changes let's first start to prevent the activity from restarting, by handling the configuration change for mainactivity in androidmanifest xml file android configchanges="keyboardhidden|orientation|screensize" save the instance state by storing the data before the activity is destroyed when the screen changes from main to cover display or vice versa, android provides us an override method called onsaveinstancestate it accepts a bundle as a parameter use putint method to store data into key/value pair override fun onsaveinstancestate outstate bundle { outstate putint "currentposition", currentimageposition log i "tag", "onsave "+currentimageposition super onsaveinstancestate outstate } retrieve the stored values when the app is first developed, the savedinstancestate will be null inside the oncreate method if it is not null, you can retrieve the stored values from the bundle using getint and the key name you set up in the previous step while storing if savedinstancestate != null { selectedimageview!! setimageresource images[savedinstancestate getint "currentposition" ] customgalleryadapter!! updateposition images[savedinstancestate getint "currentposition" ] currentimageposition = savedinstancestate getint "currentposition" log i "tag", "onsaved " + savedinstancestate getint "currentposition" } create a layout file for the main display the next thing to do, is to optimize the app's large screen ui here, you will add a navigator for the photos when using the app in the main display, since it has more space android offers an alternate resources option based on various criteria like display size, density, and other more you can use two different layout files for each display using alternate resources of android create a new directory in res in the form of [resource]-[qualifier] for cover display and main display the appropriate version of the resource is shown automatically by the system based on the smallest width qualifier of sw the cover display will be between sw240 and sw480, while the main display will be sw600 create a new directory named layout-sw600dp under the res directory and then create an xml file named activity_main res/ layout/ activity_main xml layout-sw600dp/ activity_main xml add the layout code add the gallery layout code shown below, in layout-sw600dp/activity_main xml file <?xml version="1 0" encoding="utf-8"?> <androidx constraintlayout widget constraintlayout xmlns android="http //schemas android com/apk/res/android" xmlns app="http //schemas android com/apk/res-auto" xmlns tools="http //schemas android com/tools" android id="@+id/linearlayout" android layout_width="match_parent" android layout_height="match_parent" tools context="com xyz gallery mainactivity"> <imageview android id="@+id/imageview" android layout_width="wrap_content" android layout_height="match_parent" android gravity="center_vertical" app layout_constraintend_toendof="parent" app layout_constraintstart_tostartof="parent" app layout_constrainttop_totopof="parent" /> <gallery android id="@+id/allgallery" android layout_width="fill_parent" android layout_height="108dp" android layout_marginbottom="4dp" android gravity="fill" android orientation="horizontal" android padding="10dp" android spacing="5dp" app layout_constraintbottom_tobottomof="parent" app layout_constraintend_toendof="parent" app layout_constrainthorizontal_bias="0 0" app layout_constraintstart_tostartof="parent" /> </androidx constraintlayout widget constraintlayout> run the app after building the apk, you can run the gallery app and see how it change when you fold and unfold the device if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab noteapp continuity when folding the device, can be turned on for selected apps in settings if the app supports app continuity to continue using the gallery app on the cover display when you close your phone, go to settings> display > continue apps on cover screen > toggle on gallery app foldable devices let you decide whether you want to see less or more content at the same time in the main display to see the photo navigator added in the app, go to settings> display > screen layout and zoom > see more content at the same time you're done! congratulations! you have successfully achieved the goal of this code lab now, you can optimize your app to support continuity and large screen layout by yourself! if you're having trouble, you may download this file app continuity complete code 24 44 mb to learn more about developing apps for galaxy foldable devices, visit www developer samsung com/galaxy-z
Learn Code Lab
codelabdevelop a widget for flex window objective learn how to create a calendar widget on the flex window of galaxy z flip5 overview the flex window of galaxy z flip5 provides a wider screen than the previous model, galaxy z flip4 it provides more functionalities, such as having customizable clocks, rich wallpapers, and informative notification list the widget that you are going to develop for this code lab can provide a simple interface by executing simple tasks directly visible on the flex window it can guide users to open their z flip5 to perform complex actions on the main screen set up your environment you will need the following samsung galaxy z flip5 android studio latest version recommended java se development kit jdk 11 or later remote test lab if physical device is not available sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! flex window widget sample code 404 29 kb start your project after downloading the sample code containing the project files, open your android studio and click open to open an existing project locate the downloaded android project capsule_samplewidget from the directory and click ok make the appwidgetprovider noteto know the basics about creating a widget in android, you may refer to this android developer guide the android appwidget provider defines the properties of the widget while com samsung android appwidget provider implements the methods of the appwidgetprovider class to make the appwidgetprovider, you need to declare the location of the xml file in androidmanifest xml to do this go to app > manifests and open androidmanifest xml look for the <receiver> tag with the simplecalendarwidgetprovider identifier add the following meta-data just above the </receiver> tag <meta-data android name="android appwidget provider" android resource="@xml/meta_info_calendar_widget" /> <meta-data android name="com samsung android appwidget provider" android resource="@xml/samsung_meta_info_sample_widget" /> set the size of the widget go to app > res > xml and open meta_info_calendar_widget xml add the following attributes to set the size of the flex window and set it as a lock screen android minwidth="352dp" android minheight="339dp" android resizemode="horizontal|vertical" android widgetcategory="keyguard" add the samsung appwidgetproviderinfo meta-data go to app > res > xml and open samsung_meta_info_calendar_widget xml add the following attribute to configure the display of the flex window <samsung-appwidget-provider display="sub_screen"> </samsung-appwidget-provider> start the activity from the widget the launchdisplayid indicates whether the widget is on the cover screen or on the main screen if you want to launch an activity from the widget, simply follow these steps to specify the screen display id go to app > java > com samsung android samplewidget > provider open simplecalendarwidgetprovider kt and assign 0 to the launchdisplayid if you want to display the widget on the main screen, and assign 1 if you want it to run on the flex window here, let's start a calendar activity on the main screen launchdisplayid = main_screen_id allow modification of the widget settings create a widget configuration activity to allow modification of the widget settings the appwidgetproviderinfo object describes the meta-data for the widget, such as its layout and update frequency go to app > res > xml and open meta_info_calendar_widget xml declare the configuration activity in the <appwidget-provider> tag by placing the android configure attribute android configure="com samsung android samplewidget activity editsettingactivity" create ongoing notifications first, enable the code to send notification in simplecalendarwidgetprovider kt in simplecalendarwidgetprovider kt, call the makenotification function from the notificationutil class to provide a notification for the sample widget val notificationutil = notificationutil context val notification = notificationutil makenotification pendingintent getactivity context, 0, intent context, editsettingactivity class java , pendingintent flag_cancel_current or pendingintent flag_immutable , it size notificationutil sendnotification notificationutil notification_id, notification set the notification's properties in the same function in notificationutil kt setsmallicon r drawable ic_cover_screen setcolor color parsecolor "#338577" setcontenttitle “sample widget” setongoing true after creating the notification, it should go to the main window of the flex window you can now check and see that there's an ongoing notification notethis feature only works with one ui 6 0 enable the calendar widget now, you can build the project in android studio and test it on your device to enable the widget in your galaxy z flip5, you have to go to settings > cover screen and tap widgets enable the widget that you've created in this code lab you can now see the widget on the flex window of your galaxy z flip5 check other features of the widget you can also test other features of the widget included in the sample project background of the widget in this code lab, you can check the sample project on how to change the background of the widget to be opaque or transparent vertical scrolling also, vertical scrolling is already implemented in the sample project you can check its implementation in the code this allows you to preview more notifications as possible in the flex window you're done! congratulations! you have successfully achieved the goal of this code lab now, you can develop a widget for the flex window by yourself! if you face any trouble, you may download this file flex window widget complete project 404 27 kb to learn more about developing apps for galaxy foldable devices, visit www developer samsung com/galaxy-z
Learn Code Lab
codelabimplement flex mode into a unity game objective learn how to implement flex mode into a unity game using android jetpack windowmanager and unity's java native interface jni wrapper overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps when the phone is partially folded, it will go into flex mode apps will reorient to fit the screen, letting you watch videos or play games without holding the phone for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard set up your environment you will need the following unity hub with unity 2022 3 5f1 or later must have android build support visual studio or any source code editor galaxy z fold2 or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample project for you to start coding in this code lab download it and start your learning experience! flex mode on unity sample code 1 17 gb start your project after downloading the sample project files, follow the steps below to open your project launch the unity hub click projects > open locate the unzipped project folder and click open to add the project to the hub and open in the editor notethe sample project was created in unity 2022 3 5f1 if you prefer using a different unity version, click choose another editor version when prompted and select a higher version of unity configure android player settings to ensure that the project runs smoothly on the android platform, configure the player settings as follows go to file > build settings under platform, choose android and click switch platform wait until this action finishes importing necessary assets and compiling scripts then, click player settings to open the project settings window go to player > other settings and scroll down to see target api level set it to api level 33 as any less than this will result in a dependency error regarding an lstar variable you can set the minimum api level on lower levels without any problem next, in the resolution and presentation settings, enable resizable window it is also recommended that render outside safe area is enabled to prevent black bars on the edges of the screen lastly, enable the custom main manifest, custom main gradle template, and custom gradle properties template in the publishing settings after closing the project settings window, check for the new folder structure created within your assets in the project window the newly created android folder contains androidmanifest xml, gradletemplate properties, and maintemplate gradle files import the foldablehelper and add dependencies foldablehelper is a java file that you can use in different projects it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device flat for normal mode and half-opened for flex mode , window size, and orientation of the fold on the screen download the foldablehelper java file here foldablehelper java 6 22 kb to import the foldablehelper java file and add dependencies to the project, follow the steps below in assets > plugins > android, right-click and select import new asset locate and choose the foldablehelper java file, then click import next, open the gradletemplate properties file to any source code editor like visual studio and add the following lines below the **additional_properties** marker android useandroidx = true android enablejetifier = true useandroidx sets the project to use the appropriate androidx libraries instead of support libraries enablejetifier automatically migrates existing third-party libraries to use androidx by rewriting their binaries lastly, open the maintemplate gradle file and add the dependencies for the artifacts needed for the project **apply_plugins** dependencies { implementation filetree dir 'libs', include ['* jar'] implementation "androidx appcompat appcompat 1 6 1" implementation "androidx core core 1 10 1" implementation "androidx core core-ktx 1 10 1" implementation "androidx window window 1 0 0" implementation "androidx window window-java 1 0 0" implementation "org jetbrains kotlin kotlin-stdlib-jdk8 1 9 0" **deps**} create a new playeractivity to implement flex mode on your applications, you must make necessary changes to the activity since it is impossible to access and change the original unityplayeractivity, you need to create a new playeractivity that inherits from the original to do this create a new file named foldableplayeractivity java and import it into the android folder, same as when you imported the foldablehelper java file to extend the built-in playeractivity from unity, write below code in the foldableplayeractivity java file package com unity3d player; import android os bundle; import com unity3d player unityplayeractivity; import com samsung android gamedev foldable foldablehelper; import com samsung android gamedev foldable foldablehelper windowinfolayoutlistener; import android util log; public class foldableplayeractivity extends unityplayeractivity { @override protected void oncreate bundle savedinstancestate { super oncreate savedinstancestate ; foldablehelper init this ; } @override protected void onstart { super onstart ; foldablehelper start this ; } @override protected void onstop { super onstop ; foldablehelper stop ; } @override protected void onrestart { super onrestart ; foldablehelper init this ; } public void attachunitylistener windowinfolayoutlistener listener { foldablehelper attachnativelistener listener, this ; } } oncreate calls the foldablehelper init to ensure that the windowinfotracker and metrics calculator gets created as soon as possible onstart calls the foldablehelper start since the first windowlayoutinfo doesn't get created until onstart onstop calls the foldablehelper stop to ensure that when the application closes, the listener gets cleaned up onrestart calls foldablehelper init when returning to the app after switching away windowinfotracker must be re-initialized; otherwise, flex mode will no longer update after creating the foldableplayeractivity, ensure that the game uses it open the androidmanifest xml file and change the activity name to the one you've just created <activity android name="com unity3d player foldableplayeractivity" android theme="@style/unitythemeselector"> … </activity> store foldablelayoutinfo data to flexproxy implement a native listener that receives calls from java when the device state changes by following these steps use the androidjavaproxy provided by unity in its jni implementation androidjavaproxy is a class that implements a java interface, so the next thing you need to do is create an interface in the foldablehelper java file public interface windowinfolayoutlistener { void onchanged foldablelayoutinfo layoutinfo ; } this interface replaces the temporary native function therefore, remove the code below from the foldablehelper java file public static native void onlayoutchanged foldablelayoutinfo resultinfo ; then, go to the assets > flex_scripts folder and right-click to create a new c# script called flexproxy cs inside this script, replace the automatically generated class with the flexproxy class inheriting from androidjavaproxy public class flexproxy androidjavaproxy { } in flexproxy class, define the variables needed to store the data from foldablelayoutinfo and use enumerators for the folded state, hinge orientation, and occlusion type for the various bounds, use unity's rectint type also, use a boolean to store whether the data has been updated or not public enum state { undefined, flat, half_opened }; public enum orientation { undefined, horizontal, vertical }; public enum occlusiontype { undefined, none, full }; public state state = state undefined; public orientation orientation = orientation undefined; public occlusiontype occlusiontype = occlusiontype undefined; public rectint foldbounds; public rectint currentmetrics; public rectint maxmetrics; public bool needsupdate = false; next, define what java class the flexproxy is going to implement by using the interface's fully qualified name as below public flexproxy base "com samsung android gamedev foldable foldablehelper$windowinfolayoutlistener" { } com samsung android gamedev foldable is the package name of the foldablehelper java file foldablehelper$windowinfolayoutlistener is the class and interface name separated by a $ after linking the proxy to the java interface, create a helper method to simplify java to native conversions private rectint converttorectint androidjavaobject rect { if rect != null { var left = rect get<int> "left" ; var top = rect get<int> "top" ; var width = rect call<int> "width" ; var height = rect call<int> "height" ; return new rectint xmin left, ymin top, width width, height height ; } else { return new rectint -1, -1, -1, -1 ; } } this method takes a java rect object and converts it into a unity c# rectint now, use this converttorectint function for the onchanged function to retrieve the information from the java object and store it in the flex proxy class public void onchanged androidjavaobject layoutinfo { foldbounds = converttorectint layoutinfo get<androidjavaobject> "bounds" ; currentmetrics = converttorectint layoutinfo get<androidjavaobject> "currentmetrics" ; maxmetrics = converttorectint layoutinfo get<androidjavaobject> "maxmetrics" ; orientation = orientation layoutinfo get<int> "hingeorientation" + 1 ; state = state layoutinfo get<int> "state" + 1 ; occlusiontype = occlusiontype layoutinfo get<int> "occlusiontype" + 1 ; needsupdate = true; } implement native flex mode this section focuses on creating the flex mode split-screen effect on the game’s ui create a new c# script in the flex_scripts folder called flexmodemanager cs after creating the script, define the variables you need for this implementation public class flexmodemanager monobehaviour { private flexproxy windowmanagerlistener; [serializefield] private camera maincamera; [serializefield] private camera skyboxcamera; [serializefield] private canvas controlscanvas; [serializefield] private canvas healthcanvas; [serializefield] private gameobject flexbg; [serializefield] private gameobject uiblur; windowmanagerlistener is the callback object which receives the foldablelayoutinfo from the foldablehelper java implementation maincamera and skyboxcamera are two cameras to modify in this project for creating a seamless flex mode implementation controlscanvas and healthcanvas are the two ui elements to manipulate for implementing the flex mode flexbg is a background image to disable in normal mode and enable in flex mode to fill the bottom screen uiblur is a background blur element used as part of the tutorial text it doesn't function properly with flex mode, so disabling it when in flex mode makes the ui looks better next, in the start method, create a new instance of the flexproxy class and attach it to the unity application's activity via the attachunitylistener function void start { windowmanagerlistener = new flexproxy ; using androidjavaclass javaclass = new androidjavaclass "com unity3d player unityplayer" { using androidjavaobject activity = javaclass getstatic<androidjavaobject> "currentactivity" { activity call "attachunitylistener", windowmanagerlistener ; } } } in the update method, check if the windowmanagerlistener has received any new data on the folded state of the device if the system needs an update, then call updateflexmode void update { if windowmanagerlistener needsupdate { updateflexmode ; } } create the updateflexmode method to enable or disable flex mode notethis project is set up for landscape mode only the implementation discussed in this code lab only covers setting up flex mode with a horizontal fold in landscape however, if you were able to follow, it should be simple to set up something similar for different fold orientations on devices such as the galaxy z flip series of devices private void updateflexmode { } check the folded state of the device via the windowmanagerlistener if the device is half_opened, implement flex mode private void updateflexmode { if windowmanagerlistener state == flexproxy state half_opened { } } to split the ui screen horizontally, set the anchor points of the controlscanvas and the healthcanvas so they are locked at the bottom screen or below the fold also, set the viewports of the maincamera and skyboxcamera to be above the fold - which is the top screen next, set the anchors for the flexbg object and enable it to fill the space behind the ui on the bottom screen deactivate the uiblur element if it exists the ui blur element is only present at level 1 of the demo game a check is necessary to ensure the flex mode manager works on the second level private void updateflexmode { if windowmanagerlistener state == flexproxy state half_opened { float lowerscreenanchormax = float windowmanagerlistener foldbounds ymin / windowmanagerlistener currentmetrics height; recttransform controlscanvastransform = controlscanvas getcomponent<recttransform> ; recttransform healthcanvastransform = healthcanvas getcomponent<recttransform> ; recttransform flexbgtransform = flexbg getcomponent<recttransform> ; controlscanvastransform anchormin = new vector2 0, 0 ; controlscanvastransform anchormax = new vector2 1, lowerscreenanchormax ; healthcanvastransform anchormin = new vector2 0, lowerscreenanchormax ; healthcanvastransform anchormax = new vector2 0, lowerscreenanchormax ; flexbgtransform anchormin = new vector2 0, 0 ; flexbgtransform anchormax = new vector2 1, lowerscreenanchormax ; float upperscreenrectheight = float windowmanagerlistener foldbounds ymax / windowmanagerlistener currentmetrics height; maincamera rect = new rect 0, upperscreenrectheight, 1, upperscreenrectheight ; skyboxcamera rect = new rect 0, upperscreenrectheight, 1, upperscreenrectheight ; flexbg setactive true ; if uiblur != null uiblur setactive false ; } } return the ui to full screen when the device is no longer in flex mode by disabling flexbg; enabling uiblur when it exists; and setting all the anchor points and viewports back to their original values and finally, inform the windowmanagerlistener that it doesn't need an update else { recttransform controlscanvastransform = controlscanvas getcomponent<recttransform> ; recttransform healthcanvastransform = healthcanvas getcomponent<recttransform> ; controlscanvastransform anchormin = new vector2 0, 0 ; controlscanvastransform anchormax = new vector2 1, 1 ; healthcanvastransform anchormin = new vector2 0, 1 ; healthcanvastransform anchormax = new vector2 0, 1 ; maincamera rect = new rect 0, 0, 1, 1 ; skyboxcamera rect = new rect 0, 0, 1, 1 ; flexbg setactive false ; if uiblur != null uiblur setactive true ; } windowmanagerlistener needsupdate = false; set up the scenes for flex mode go back to the unity editor in assets > 3dgamekit > scenes > gameplay, double-click on the level 1 scene to open it right-click in the hierarchy window and select create empty name the new gameobject as flexmanager or a similar name to reflect its purpose select the flexmanager object and click the add component button in the inspector window type in flexmodemanager, and select the script when it shows up select the relevant objects for each camera, canvas and game object as below do the same for level 2 but leave the ui blur empty build and run the app go to file > build settings click build at the bottom of the window to build the apk after building the apk, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement flex mode in your unity game app by yourself! to learn more, visit www developer samsung com/galaxy-z www developer samsung com/galaxy-gamedev
Learn Code Lab
codelabimplement flex mode on an unreal engine game objective learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface jni overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps when the phone is partially folded, it will go into flex mode apps will reorient to fit the screen, letting you watch videos or play games without holding the phone for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard set up your environment you will need the following epic games launcher with unreal engine 4 or later visual studio or any source code editor samsung galaxy foldable device galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available create and set up your project after launching unreal engine from the epic games launcher, follow the steps below to start your project in the select or create new project window, choose games as a new project category and click next select third person as template, then click next to proceed noteyou can implement flex mode on any template or existing projects and use this code lab activity as a reference in the project settings window, set the following type of project c++ target platform mobile / tablet performance characteristics scalable 3d or 2d real-time raytracing raytracing disabled include an additional content pack no starter content project name tutorial_project click create project wait for the engine to finish loading and open the unreal editor once the project is loaded, go to edit > project settings > platforms > android click the configure now button if the project is not yet configured for the android platform then, proceed with the following apk packaging and build settings a apk packaging set target sdk version to 30 set orientation to full sensor change the maximum supported aspect ratio to 2 8 aspect ratio of galaxy z fold3 in decimal to prevent black bars from appearing on the cover display leave it if your game does not need to use the cover display enable use display cutout region?, to prevents black bars at the edge of the main screen otherwise, leave it unchecked b build disable support opengl es3 1 and enable support vulkan notecurrently, there is a problem with opengl es and the split-screen system being investigated the only option right now is to turn off opengl es and use vulkan instead enable native resize event the resize event of a game when switching between displays is disabled in the engine by default however, this behavior can be easily enabled by setting android enablenativeresizeevent=1 in the deviceprofile currently, the only way to create a profile for foldable devices is by creating a specific rule for each device to save time in this code lab, enable the native resize event for all android devices instead locate and open the tutorial_project > config folder in file explorer inside the config folder, create a new folder named android create a new file called androiddeviceprofiles ini and open this file in a text editor, such as visual studio copy below deviceprofile code to the newly created androiddeviceprofiles ini file [deviceprofiles] +deviceprofilenameandtypes=android,android [android deviceprofile] devicetype=android baseprofilename= +cvars=r mobilecontentscalefactor=1 0 +cvars=slate absoluteindices=1 +cvars=r vulkan delayacquirebackbuffer=2 +cvars=r vulkan robustbufferaccess=1 +cvars=r vulkan descriptorsetlayoutmode=2 ; don't enable vulkan by default specific device profiles can set this cvar to 0 to enable vulkan +cvars=r android disablevulkansupport=1 +cvars=r android disablevulkansm5support=1 ; pf_b8g8r8a8 +cvars=r defaultbackbufferpixelformat=0 +cvars=android enablenativeresizeevent=1 ; previewallowlistcvars and previewdenylistcvars are arrays of cvars that are included or excluded from being applied in mobile preview ; if any previewallowlistcvars is set, cvars are denied by default previewallowlistcvars=none this is a copy of the default android deviceprofile from the existing basedeviceprofiles ini file but with the enabled nativeresizeevent console variable cvars notethis step is not required when you only want to implement flex mode yet, it's recommended, to allow applications to run seamlessly from main to cover display without stretching and squashing the game, by enabling the nativeresizeevent create a new plugin and import the foldablehelper foldablehelper is a java file that you can use in different projects it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device flat for normal mode and half-opened for flex mode , window size, and orientation of the fold on the screen download the foldablehelper java file here foldablehelper java 5 64 kb to import the foldablehelper java file to the project, follow the steps below go to edit > plugins in the unreal editor click the new plugin button and select blank to create a blank plugin in the name field, type foldables_tutorial and click the create plugin button in file explorer, locate and open tutorial_project > plugins folder go to plugins > foldables_tutorial > source> foldables_tutorial > private and create a new folder called java copy the foldablehelper java file into java folder open the tutorial_project sln file in visual studio in the same private folder path, add a new filter called java right-click on the java filter and click add > existing item locate the foldablehelper java file, then click add to include this java file in the build modify java activity to use foldablehelper unreal plugin language upl is a simple xml-based language created by epic games for manipulating xml and returning strings using upl, you can utilize the foldablehelper java file by modifying the java activity and related gradle files as follows in visual studio, right-click on source > foldables_tutorial folder, then click add > new item > web > xml file xml create an xml file called foldables_tutorial_upl xml ensure that the file location is correct before clicking add in the newly created xml file, include the foldablehelper java file in the build by copying the java folder to the build directory <root xmlns android="http //schemas android com/apk/res/android"> <prebuildcopies> <copydir src="$s plugindir /private/java" dst="$s builddir /src/com/samsung/android/gamedev/foldable" /> </prebuildcopies> set up the gradle dependencies in the build gradle file by adding the following in the xml file <buildgradleadditions> <insert> dependencies { implementation filetree dir 'libs', include ['* jar'] implementation "org jetbrains kotlin kotlin-stdlib-jdk8 1 6 0" implementation "androidx core core 1 7 0" implementation "androidx core core-ktx 1 7 0" implementation "androidx appcompat appcompat 1 4 0" implementation "androidx window window 1 0 0" implementation "androidx window window-java 1 0 0" } android{ compileoptions{ sourcecompatibility javaversion version_1_8 targetcompatibility javaversion version_1_8 } } </insert> </buildgradleadditions> next, modify the gameactivity <gameactivityimportadditions> <insert> <!-- package name of foldablehelper --> import com samsung android gamedev foldable foldablehelper; </insert> </gameactivityimportadditions> <gameactivityoncreateadditions> <insert> foldablehelper init this ; </insert> </gameactivityoncreateadditions> <gameactivityonstartadditions> <insert> foldablehelper start this ; </insert> </gameactivityonstartadditions> <gameactivityonstopadditions> <insert> foldablehelper stop ; </insert> </gameactivityonstopadditions> </root> gameactivityimportadditions adds the com samsung android gamedev foldable foldablehelper into the gameactivity with the existing imports gameactivityoncreateadditions adds the code to the oncreate method inside the gameactivity gameactivityonstartadditions adds the code to the onstart method inside the gameactivity gameactivityonstopadditions adds the code to the onstop method inside the gameactivity save the xml file then, ensure that the engine uses the upl file by modifying the foldables_tutorial build cs script, located in the same folder as the foldables_tutorial_upl xml file after the dynamicallyloadedmodulenames addrange call, add the following if target platform == unrealtargetplatform android { additionalpropertiesforreceipt add "androidplugin", moduledirectory + "\\foldables_tutorial_upl xml" ; } this means that the game engine will use the upl file if the platform is android otherwise, the foldablehelper won’t work implement a storage struct the next thing to implement is a struct, the native version of java’s foldablelayoutinfo class to store the data retrieved from the java code using a struct, do the following in content browser of unreal editor, right-click on c++ classes > add/import content then, click new c++ class select none for the parent class and click next name the new class as foldablelayoutinfo assign it to the foldables_tutorial plugin then, click create class delete the created foldablelayoutinfo cpp file and only keep its header file in the header file called foldablelayoutinfo h, set up a struct to store all needed data from the windowmanager #pragma once #include "core h" enum efoldstate { undefined_state, flat, half_opened }; enum efoldorientation { undefined_orientation, horizontal, vertical }; enum efoldocclusiontype { undefined_occlusion, none, full }; struct ffoldablelayoutinfo { efoldstate state; efoldorientation orientation; efoldocclusiontype occlusiontype; fvector4 foldbounds; fvector4 currentmetrics; fvector4 maxmetrics; bool isseparating; ffoldablelayoutinfo state efoldstate undefined_state , orientation efoldorientation undefined_orientation , occlusiontype efoldocclusiontype undefined_occlusion , foldbounds -1, -1, -1, -1 , currentmetrics -1, -1, -1, -1 , maxmetrics -1, -1, -1, -1 , isseparating false { } }; implement jni code to implement jni, create a new c++ class with no parent and name it foldables_helper assign the class to the same plugin, then modify the c++ header and source files as follows in the created header file foldables_helper h , include foldablelayoutinfo h #include "foldablelayoutinfo h" then, declare a multicast_delegate to serve as a listener for passing the data from the java implementation to the rest of the engine declare_multicast_delegate_oneparam fonlayoutchangeddelegate, ffoldablelayoutinfo ; lastly, set up the methods and member variables class foldables_tutorial_api ffoldables_helper { public static void init ; static bool haslistener; static fonlayoutchangeddelegate onlayoutchanged; }; moving to the source file foldables_helper cpp , set up the definitions for the methods and member variables created in the header file bool ffoldables_helper haslistener = false; fonlayoutchangeddelegate ffoldables_helper onlayoutchanged; void ffoldables_helper init { haslistener = true; } now, in the same source file, create the native version of the onlayoutchanged function created in the foldablehelper java file since the java onlayoutchanged function only works on android, surround the function with an #if directive to ensure that it compiles only on android #if platform_android #endif within this directive, copy the code below to use the jni definition of the java onlayoutchanged function extern "c" jniexport void jnicall java_com_samsung_android_gamedev_foldable_foldablehelper_onlayoutchanged jnienv * env, jclass clazz, jobject jfoldablelayoutinfo { create the ffoldablelayoutinfo to store the data retrieved from java ffoldablelayoutinfo result; retrieve the field ids of the foldablelayoutinfo and rect objects created in the java file //java foldablelayoutinfo field ids jclass jfoldablelayoutinfocls = env->getobjectclass jfoldablelayoutinfo ; jfieldid currentmetricsid = env->getfieldid jfoldablelayoutinfocls, "currentmetrics", "landroid/graphics/rect;" ; jfieldid maxmetricsid = env->getfieldid jfoldablelayoutinfocls, "maxmetrics", "landroid/graphics/rect;" ; jfieldid hingeorientationid = env->getfieldid jfoldablelayoutinfocls, "hingeorientation", "i" ; jfieldid stateid = env->getfieldid jfoldablelayoutinfocls, "state", "i" ; jfieldid occlusiontypeid = env->getfieldid jfoldablelayoutinfocls, "occlusiontype", "i" ; jfieldid isseparatingid = env->getfieldid jfoldablelayoutinfocls, "isseparating", "z" ; jfieldid boundsid = env->getfieldid jfoldablelayoutinfocls, "bounds", "landroid/graphics/rect;" ; jobject currentmetricsrect = env->getobjectfield jfoldablelayoutinfo, currentmetricsid ; //java rect object field ids jclass rectcls = env->getobjectclass currentmetricsrect ; jfieldid leftid = env->getfieldid rectcls, "left", "i" ; jfieldid topid = env->getfieldid rectcls, "top", "i" ; jfieldid rightid = env->getfieldid rectcls, "right", "i" ; jfieldid bottomid = env->getfieldid rectcls, "bottom", "i" ; retrieve the current windowmetrics and store it in the ffoldablelayoutinfo as an fintvector4 // currentmetrics int left = env->getintfield currentmetricsrect, leftid ; int top = env->getintfield currentmetricsrect, topid ; int right = env->getintfield currentmetricsrect, rightid ; int bottom = env->getintfield currentmetricsrect, bottomid ; // store currentmetrics rect to fvector4 result currentmetrics = fintvector4{ left, top, right, bottom }; do the same for the other variables in the java foldablelayoutinfo // maxmetrics jobject maxmetricsrect = env->getobjectfield jfoldablelayoutinfo, maxmetricsid ; left = env->getintfield maxmetricsrect, leftid ; top = env->getintfield maxmetricsrect, topid ; right = env->getintfield maxmetricsrect, rightid ; bottom = env->getintfield maxmetricsrect, bottomid ; //store maxmetrics rect to fvector4 result maxmetrics = fintvector4{ left, top, right, bottom }; int hingeorientation = env->getintfield jfoldablelayoutinfo, hingeorientationid ; int state = env->getintfield jfoldablelayoutinfo, stateid ; int occlusiontype = env->getintfield jfoldablelayoutinfo, occlusiontypeid ; bool isseparating = env->getbooleanfield jfoldablelayoutinfo, isseparatingid ; // store the values to an object for unreal result orientation = tenumasbyte<efoldorientation> hingeorientation + 1 ; result state = tenumasbyte<efoldstate> state + 1 ; result occlusiontype = tenumasbyte<efoldocclusiontype> occlusiontype + 1 ; result isseparating = isseparating; // boundsrect jobject boundsrect = env->getobjectfield jfoldablelayoutinfo, boundsid ; left = env->getintfield boundsrect, leftid ; top = env->getintfield boundsrect, topid ; right = env->getintfield boundsrect, rightid ; bottom = env->getintfield boundsrect, bottomid ; // store maxmetrics rect to fvector4 result foldbounds = fintvector4{ left, top, right, bottom }; broadcast the result via the onlayoutchanged delegate for use in the engine if ffoldables_helper haslistener { ue_log logtemp, warning, text "broadcast" ; ffoldables_helper onlayoutchanged broadcast result ; } } create a player controller and two ui states this section focuses on adding a player controller and creating two user interface ui states for flat and flex modes these objects are needed for the flex mode logic implementation following are the steps to add a player controller and create two ui states add a new player controller blueprint in content browser, go to content > thirdpersoncpp and right-click on blueprints > add/import content > blueprint class pick player controller as its parent class rename it as flexplayercontroller notethe flexplayercontroller added is generic and can be replaced by your custom player controller in an actual project add a new c++ class with actor component as its parent class name it as foldables_manager and assign it to the foldables_tutorial plugin click the create class button open the flexplayercontroller blueprint by double-clicking it click open full blueprint editor attach the actor component to the flexplayercontroller in the left pane, click add component, then find and select the foldables_manager next, create a pair of userwidget classes for the ui layouts needed flat mode ui for the full screen or normal mode; and flex mode ui for split-screen in add c++ class window, select the show all classes checkbox find and pick userwidget as the parent class then, click next name the new user widget as flatui and attach it to the plugin click next repeat the process but name the new user widget as flexui you might get an error when trying to compile stating that the userwidget is an undeclared symbol to fix this, open the foldables_tutorial build cs file, and in the publicdependencymodulenames addrange call, add "inputcore" and "umg" to the list create a pair of blueprints made from subclasses of these two user widgets right-click on content and create a new folder called foldui inside the newly created folder, right-click to add a new blueprint class in all classes, choose flatui and click the select button rename the blueprint as bp_flatui in the same folder, repeat the process but choose the flexui class and rename the blueprint as bp_flexui double-click on bp_flatui and bp_flexui, then design your two uis like below to visualize switching between flat and flex mode flat ui flex ui notethis code lab activity does not cover the steps on how to create or design ui if you want to learn about how to create your own game design in unreal engine 4, refer to unreal motion graphics ui designer guide implement the flex mode logic after creating the flexplayercontroller and the two ui states bp_flatui and bp_flexui , you can now implement flex mode logic in the foldables_manager open the foldables_manager h and include the necessary c++ header files #pragma once #include "coreminimal h" #include "components/actorcomponent h" #include "engine h" #include "flatui h" #include "flexui h" #include "foldables_helper h" #include "foldables_manager generated h" remove the line below to save a little bit of performance as this component doesn't need to tick public // called every frame virtual void tickcomponent float deltatime, eleveltick ticktype, factorcomponenttickfunction* thistickfunction override; set up the functions needed in foldables_manager the constructor, a function to create the ui widgets the implementation of onlayoutchanged delegate public // sets default values for this component's properties ufoldables_manager ; void createwidgets ; protected // called when the game starts virtual void beginplay override; protected void onlayoutchanged ffoldablelayoutinfo foldablelayoutinfo ; then, set up the variables needed references to the flat and flex ui classes references to the flat and flex ui objects mark the pointers as uproperty to ensure that garbage collection does not delete the objects they point to tsubclassof<uuserwidget> flatuiclass; tsubclassof<uuserwidget> flexuiclass; uproperty class uflatui* flatui; uproperty class uflexui* flexui; finally, define a new private function restoreflatmode , to disable flex mode and return to normal mode private void restoreflatmode ; moving over to foldables_manager cpp, implement the constructor using the constructorhelpers, find the ui classes and set the variables to store these classes also, set the bcanevertick to false to prevent the component from ticking and remove the code block of tickcomponent function // sets default values for this component's properties ufoldables_manager ufoldables_manager { primarycomponenttick bcanevertick = false; static constructorhelpers fclassfinder<uflatui> flatuibpclass text "/game/foldui/bp_flatui" ; static constructorhelpers fclassfinder<uflexui> flexuibpclass text "/game/foldui/bp_flexui" ; if flatuibpclass succeeded { flatuiclass = flatuibpclass class; } if flexuibpclass succeeded { flexuiclass = flexuibpclass class; } } next, set up the beginplay function to link the delegate to the onlayoutchanged function, to initialize the foldables_helper, and to create the widgets ready for use in the first frame // called when the game starts void ufoldables_manager beginplay { super beginplay ; ffoldables_helper onlayoutchanged adduobject this, &ufoldables_manager onlayoutchanged ; ffoldables_helper init ; createwidgets ; } set up the createwidgets function to create the widgets using the ui classes acquired in the constructor add the flatui widget to the viewport, assuming the app opens in normal mode until it receives the foldablelayoutinfo void ufoldables_manager createwidgets { flatui = createwidget<uflatui> aplayercontroller* getowner , flatuiclass, fname text "flatui" ; flexui = createwidget<uflexui> aplayercontroller* getowner , flexuiclass, fname text "flexui" ; flatui->addtoviewport ; } afterward, create the onlayoutchanged function, which will be called via a delegate inside this function, check whether the device’s folded state is half_opened if so, check whether the orientation of the fold is horizontal void ufoldables_manager onlayoutchanged ffoldablelayoutinfo foldablelayoutinfo { //if state is now flex if foldablelayoutinfo state == efoldstate half_opened { if foldablelayoutinfo orientation == efoldorientation horizontal { notefor this third person template, splitting the screen vertically isn’t ideal from a user experience ux point of view for this code lab activity, split the screen only on the horizontal fold top and bottom screen if you want it vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis you must also ensure that you have a flex ui object for the vertical layout if the device is both on flex mode and horizontal fold, change the viewport to only render on the top screen using the normalized position of the folding feature then in an asynctask on the game thread, disable the flatui and enable the flexui however, if the device is on normal mode, then return to flat ui using restoreflatmode function //horizontal split float foldratio = float foldablelayoutinfo foldbounds y / foldablelayoutinfo currentmetrics w - foldablelayoutinfo currentmetrics y ; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizex = 1 0f; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizey = foldratio; asynctask enamedthreads gamethread, [=] { if flatui->isinviewport { flatui->removefromparent ; } if !flexui->isinviewport { flexui->addtoviewport 0 ; } } ; } else { restoreflatmode ; } } else { restoreflatmode ; } } reverse the flex mode implementation logic to create the restoreflatmode function by setting the viewport to fill the screen, then disable the flexui and enable the flatui void ufoldables_manager restoreflatmode { gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizex = 1 0f; gengine->gameviewport->splitscreeninfo[0] playerdata[0] sizey = 1 0f; asynctask enamedthreads gamethread, [=] { if !flatui->isinviewport { flatui->addtoviewport 0 ; } if flexui->isinviewport { flexui->removefromparent ; } } ; } set up a game mode and attach the flexplayercontroller the game mode defines the game rules, scoring, and any game-specific behavior set up the game mode in unreal editor by creating a blueprint class in the content > thirdpersoncpp > blueprints folder pick game mode base as the parent class and rename it as flexgamemode double-click on flexgamemode in the drop-down menu next to player controller class, choose the flexplayercontroller lastly, go to edit > project settings > project > maps & modes and select flexgamemode as the default gamemode build and run the app go to edit > package project > android to build the apk ensure that the android development environment for unreal is already set up to your computer noteif the build failed due to corrupted build tools, consider downgrading the version to 30 or lower using the sdk manager or, simply rename d8 bat to the name of the missing file dx bat in sdk path > build-tools > version number directory and, in lib folder of the same directory, rename d8 jar to dx jar after packaging your android project, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode if you don’t have any physical device, you can also test it on a remote test lab device tipwatch this tutorial video and know how to easily test your app via remote test lab you're done! congratulations! you have successfully achieved the goal of this code lab now, you can implement flex mode in your unreal engine game app by yourself! if you're having trouble, you may download this file flex mode on unreal complete code 20 16 mb to learn more, visit www developer samsung com/galaxy-z www developer samsung com/galaxy-gamedev
Learn Code Lab
codelaboptimize game performance with adaptive performance in unity objective learn how to optimize the performance of a demo game on samsung galaxy devices using the adaptive performance in unity the adaptive performance package provides you with tools to properly adjust you’re game and improve its overall game performance overview galaxy gamesdk delivers an interface between game application and device which helps developers optimize their games integrating unity adaptive performance with galaxy gamesdk allows unity developers to use this feature on unity editor within unity package manager and also customize their game contents by c# scripting game performance and quality settings can be adjusted in real time by identifying device performance status and heat trends moreover, using a set of simple ui controls, you can scale the quality of the game to match the device platform the latest version of adaptive performance, now uses new android apis that gather information from samsung hardware to provide more detailed insights into the device's thermal components and cpu optimization there are two providers for adaptive performance the samsung provider 5 0 utilizes apis from the gamesdk, which is supported from samsung galaxy s10 or note10 devices until newer models on the other hand, the android provider 1 2 uses the android dynamic performance framework adpf apis and is supported on devices running android 12 or higher this code lab focuses on using the samsung provider of adaptive performance thermal throttling mobile devices lack active cooling systems, causing temperatures to rise this triggers a warning to the unity subsystems, which lowers hardware demand to control heat and avoid performance degradation the ideal goal is to make performance stable with low temperature adaptive performance, primarily for performance and quality balance management, can help achieve this thermal warning the warning system implemented in gamesdk can trigger both internal unity systems and developer-defined behavior, such as disabling custom scripts and shaders quality settings it also provides you with the option to adjust the quality of the game to maintain stable performance at different warning levels you can scale the quality of your game in real time to meet the desired performance this includes changes in level of detail, animation, physics, and visual effects scalers with adaptive performance enabled, the game's fps becomes more stable and consistently higher over time this is because the game adapts its contents based on thermal warnings provided by samsung's thermal callback system the adaptive performance in unity provides developers with scalers that affect various aspects of the game frame rate resolution batching level of detail lod lookup texture lut multisample anti-aliasing msaa shadow cascade shadow distance shadow map resolution shadow quality sorting transparency view distance physics decals layer culling these general scalers can be used to scale the content based on your settings in the editor however, you may also create custom scalers that integrate with your own systems for instance, your own scalers can disable cpu-expensive scripts from running when thermal warnings are reached the latest version of adaptive performance, v5 0, includes new additions to the scalers decal scaler changes the draw distance of decals it controls how far a decal can be before not being rendered layer culling scaler adjusts the distance that expensive layers, such as transparency or water, when they start being culled it’s a useful scaler for scenes with more expensive shader calculations in every frame, unity calculates how all the physics in a level interacts with everything else, such as when the ball is dropping to the floor physics scaler adjusts the frequency at which this calculation is performed a lower frequency means fewer cpu calculations per second set up your environment you will need the following unity editor version 2022 3 visual studio or any source code editor supported samsung galaxy device remote test lab if physical device is not available requirements samsung account java runtime environment jre 7 or later with java web start internet environment where port 2600 is available sample code here is a sample project for you to start coding in this code lab download it and start your learning experience! adaptive performance sample code 903 96 mb demo game the sample project contains a demo game named boat attack it is an open-source demo game provided by unity it has the following features triangles 800,000 vertices 771,000 shadow casters 133 start your project after downloading the sample project, follow the steps to open your project launch the unity hub click projects > open after locating the unzipped project folder, you can open it in unity editor it initially downloads the needed resources for your project open benchmark_island-flythrough scene found under assets notethe project was tested in unity 2022 3 it is recommended to use this version in this code lab set up adaptive performance components in window > package manager > adaptive performance, you can check if the adaptive performance is already included in the project go to edit > project settings > adaptive performance enable the initialize adaptive performance on startup and select samsung android provider enable the scalers by going to adaptive performance > samsung android > indexer settings check the scaler settings and just use the default values add adaptive performance script to your project you are going to use a script that contains examples of the adaptive performance api code it gives you access to the frame time information, thermal information, and cpu/gpu bottlenecks this script adjusts the performance based on the device's thermal state, allowing longer game time by reducing in-game demands download the adaptive performance script adaptiveperformanceconroller cs 6 15 kb simply drag and drop the script into assets folder create a new object and attach the script to the object change the frame rate based on thermal warnings get to know about thermal warnings throttling warnings allow you to know when your game is about to be forcibly throttled and have sections of lag and lowered frame rate adaptive performance provides these thermal alerts that allow you to take control and adjust settings before the performance downgrades check the registered event handler to trigger thermal status ithermalstatus thermalstatus thermalevent then, check thermalmetrics information in the handler name description value nowarning no warning is the normal warning level during standard thermal state 0 throttlingimminent if throttling is imminent, the application should perform adjustments to avoid thermal throttling 1 throttling if the application is in the throttling state, it should make adjustments to go back to normal temperature levels 2 temperaturelevel 0 0 ~ 1 0 current normalized temperature level in the range of [0, 1] a value of 0 means standard operation temperature and the device is not in a throttling state a value of 1 means that the maximum temperature of the device is reached and the device is going into or is already in throttling state temperaturetrend -1 0 ~ 1 0 current normalized temperature trend in the range of [-1, 1] a value of 1 describes a rapid increase in temperature a value of 0 describes a constant temperature a value of -1 describes a rapid decrease in temperature it takes at least 10s until the temperature trend may reflect any changes adjust the frame rate in the adaptiveperformancecontroller cs, the following code is responsible for changing the frame rate depending on the current thermal alert void onthermalevent thermalmetrics ev { switch ev warninglevel { case warninglevel nowarning application targetframerate = 30; break; case warninglevel throttlingimminent application targetframerate = 25; break; case warninglevel throttling application targetframerate = 15; break; } } change the quality settings the performance bottleneck api informs you via an enum value if there is a bottleneck so you can adjust the workload iadaptiveperformance performancestatus performancemetrics performancebottleneck namespace unityengine adaptiveperformance { public enum performancebottleneck { unknown = 0, cpu = 1, gpu = 2, targetframerate = 3 } } in the adaptiveperformancecontroller cs script, the code below is responsible for controlling the lod of the models depending on the performance bottleneck a model with multiple lod is a prime example of a quality setting that can heavily affect the games performance lowering it lessens the load on the gpu, frees up resources, and eventually prevents thermal throttling change the quality setting by adjusting lod in real time by using the performancebottleneck api switch ap performancestatus performancemetrics performancebottleneck // iadaptiveperformance performancestatus performancemetrics performancebottleneck { case performancebottleneck gpu debug logformat "[adp] performancebottleneck gpu " ; lowerlod ; break; case performancebottleneck cpu debug logformat "[adp] performancebottleneck cpu " ; break; case performancebottleneck targetframerate debug logformat "[adp] performancebottleneck targetframerate " ; break; case performancebottleneck unknown debug logformat "[adp] performancebottleneck unknown" ; break; } create a custom scaler to create custom scalers, you need to create a new class that inherits from adaptiveperformancescaler the adaptive performance package includes this class that can be added to the adaptive performance framework, which adjusts quality settings based on unity's thermal settings download this custom scaler to control the texture quality of the demo game texturescaler cs 1 17 kb simply drag and drop the downloaded file into the project test using device simulator in unity the device simulator in unity allows you to test out adaptive performance functionality without a physical mobile device go to window > general > device simulator go to edit > project settings > adaptive performance enable the initialize adaptive performance on startup and select device simulator provider in the device simulator, you can try to send thermal warnings and create artificial bottlenecks to test different behaviors of the demo game the visual scripts display the scalers available and show if it is enabled enabling a specific scaler means that the adaptive performance w you can override the scalers to test their impact on the demo game's performance and scene some scalers may not affect the scene but may improve the performance in the long run build and launch the apk go to file > build settings connect your galaxy device enable development build and select scripts only build option make sure that the autoconnect profiler is enabled to check the profiler in unity and measure the performance in build to device, click the patch button to install or update the apk in your device test using unity profiler unity has also introduced a new module in its profiler that allows you to monitor the scalers changes and extract frame time information to your editor the profiler allows you to get real-time information from the adaptive performance plugin navigate to windows > analysis > profiler set the play mode to your connected galaxy device once connected, you can analyze the frame time stats and check the state of the scalers during runtime scalers in this profiler are reacting to the thermal trend and are being raised or lowered in response to the thermal warning observe the scalers when enabled or disabled texture scaler the texture scaler is the custom script to lower the texture quality based on thermal levels when enabled and maxed out, the texture fidelity has been lowered this lowers the gpu load and memory usage lod scaler you may notice a subtle change in the appearance of the rocks, trees, and umbrella models this is due to the adaptation of the lod bias that determines which version of the models to use as the thermal levels rise, lower poly models are selected to reduce the number of triangles rendered this reduces the gpu load and minimizes thermal build-up shadow map resolution shadow map resolution creates a blurring effect on the shadows which incidentally lowers the performance load required to render them basically, lower shadow detail means fewer gpu calculations, which lead to less heat build-up check the game performance using gpuwatch gpuwatch is a profiling tool for observing gpu activity in your app the following are the common information shown by the gpuwatch fps counters current average cpu and gpu load cpu load gpu load it is very helpful in profiling your game during the post-development stage so you can further optimize to enable gpuwatch on your galaxy device you can easily reposition the gpuwatch widgets on the screen by enabling unlock widgets under the notification menu of gpuwatch you’re done! congratulations! you have successfully achieved the goal of this code lab now, you can improve and optimize your android game on samsung galaxy devices using adaptive performance in unity by learning about scalers, thermal warnings, and bottleneck apis the performance of your mobile games can be pushed further by utilizing these tools if you face any trouble, you may download this file adaptive performance complete project 1 06 gb to learn more, visit developer samsung com/galaxy-gamedev
Learn Code Lab
codelabtransfer heart rate data from galaxy watch to a mobile device objective create a health app for galaxy watch, operating on wear os powered by samsung, to measure heart rate and inter-beat interval ibi , send data to a paired android phone, and create an android application for receiving data sent from a paired galaxy watch overview with this code lab, you can measure various health data using samsung health sensor sdk and send it to a paired android mobile device for further processing samsung health sensor sdk tracks various health data, but it cannot save or send collected results meanwhile, wearable data layer allows you to synchronize data from your galaxy watch to an android mobile device using a paired mobile device allows the data to be more organized by taking advantage of a bigger screen and better performance see samsung health sensor sdk descriptions for detailed information set up your environment you will need the following galaxy watch4 or newer android mobile device android studio latest version recommended java se development kit jdk 17 or later sample code here is a sample code for you to start coding in this code lab download it and start your learning experience! heart rate data transfer sample code 213 7 kb connect your galaxy watch to wi-fi go to settings > connection > wi-fi and make sure that the wi-fi is enabled from the list of available wi-fi networks, choose and connect to the same one as your pc turn on developer mode and adjust its settings on your watch, go to settings > about watch > software and tap on software version 5 times upon successful activation of developer mode, a toast message will display as on the image below afterwards, developer options will be visible under settings tap developer options and enable the following options adb debugging in developer options find wireless debugging turn on wireless debugging check always allow on this network and tap allow go back to developer options and click turn off automatic wi-fi notethere may be differences in settings depending on your one ui version connect your galaxy watch to android studio go to settings > developer options > wireless debugging and choose pair new device take note of the wi-fi pairing code, ip address & port in android studio, go to terminal and type adb pair <ip address> <port> <wi-fi pairing code> when prompted, tap always allow from this computer to allow debugging after successfully pairing, type adb connect <ip address of your watch> <port> upon successful connection, you will see the following message in the terminal connected to <ip address of your watch> now, you can run the app directly on your watch turn on developer mode for health platform to use the app, you need to enable developer mode in the health platform on your watch go to settings > apps > health platform quickly tap health platform title for 10 times this enables developer mode and displays [dev mode] below the title to stop using developer mode, quickly tap health platform title for 10 times to disable it set up your android device click on the following links to setup your android device enable developer options run apps on a hardware device connect the galaxy watch with you samsung mobile phone start your project in android studio, click open to open an existing project locate the downloaded android project hrdatatransfer-code-lab from the directory and click ok you should see both devices and applications available in android studio as in the screenshots below initiate heart rate tracking noteyou may refer to this blog post for more detailed analysis of the heart rate tracking using samsung health sensor sdk first, you need to connect to the healthtrackingservice to do that create connectionlistener, create healthtrackingservice object by invoking healthtrackingservice connectionlistener, context invoke healthtrackingservice connectservice when connected to the health tracking service, check the tracking capability the available trackers may vary depending on samsung health sensor sdk, health platform versions or watch hardware version use the gettrackingcapability function of the healthtrackingservice object obtain heart rate tracker object using the function healthtrackingservice gethealthtracker healthtrackertype heart_rate_continuous define event listener healthtracker trackereventlistener, where the heart rate values will be collected start tracking the tracker starts collecting heart rate data when healthtracker seteventlistener updatelistener is invoked, using the event listener collect heart data from the watch the updatelistener collects datapoint instances from the watch, which contains a collection of valuekey objects those objects contain heart rate, ibi values, and ibi statuses there's always one value for heart rate while the number of ibi values vary from 0-4 both ibi value and ibi status lists have the same size go to wear > java > data > com samsung health hrdatatransfer > data under ibidataparsing kt, provide the implementation for the function below /******************************************************************************* * [practice 1] get list of valid inter-beat interval values from a datapoint * - return arraylist<int> of valid ibi values validibilist * - if no ibi value is valid, return an empty arraylist * * var ibivalues is a list representing ibivalues up to 4 * var ibistatuses is a list of their statuses has the same size as ibivalues ------------------------------------------------------------------------------- * - hints * use local function isibivalid status, value to check validity of ibi * ****************************************************************************/ fun getvalidibilist datapoint datapoint arraylist<int> { val ibivalues = datapoint getvalue valuekey heartrateset ibi_list val ibistatuses = datapoint getvalue valuekey heartrateset ibi_status_list val validibilist = arraylist<int> //todo 1 return validibilist } check data sending capabilities for the watch once the heart rate tracker can collect data, set up the wearable data layer so it can send data to a paired android mobile device wearable data layer api provides data synchronization between wear os and android devices noteto know more about wearable data layer api, go here to determine if a remote mobile device is available, the wearable data layer api uses concept of capabilities not to be confused with samsung health sensor sdk’s tracking capabilities, providing information about available tracker types using the wearable data layer's capabilityclient, you can get information about nodes remote devices being able to consume messages from the watch go to wear > java > com samsung health hrdatatransfer > data in capabilityrepositoryimpl kt, and fill in the function below the purpose of this part is to filter all capabilities represented by allcapabilities argument by capability argument and return the set of nodes set<node> having this capability later on, we need those nodes to send the message to them /************************************************************************************** * [practice 2] check capabilities for reachable remote nodes devices * - return a set of node objects out of all capabilities represented by 2nd function * argument, having the capability represented by 1st function argument * - return empty set if no node has the capability -------------------------------------------------------------------------------------- * - hints * you might want to use filtervalues function on the given allcapabilities map * ***********************************************************************************/ override suspend fun getnodesforcapability capability string, allcapabilities map<node, set<string>> set<node> { //todo 2 } encode message for the watch before sending the results of the heart rate and ibi to the paired mobile device, you need to encode the message into a string for sending data to the paired mobile device we are using wearable data layer api’s messageclient object and its function sendmessage string nodeid, string path, byte[] message go to wear > java > com samsung health hrdatatransfer > domain in sendmessageusecase kt, fill in the function below and use json format to encode the list of results arraylist<trackeddata> into a string /*********************************************************************** * [practice 3] - encode heart rate & inter-beat interval into string * - encode function argument trackeddata into json format * - return the encoded string ----------------------------------------------------------------------- * - hint * use json encodetostring function **********************************************************************/ fun encodemessage trackeddata arraylist<trackeddata> string { //todo 3 } notetrackeddata is an object, containing data received from heart rate tracker’s single datapoint object @serializable data class trackeddata var hr int, var ibi arraylist<int> = arraylist run unit tests for your convenience, you will find an additional unit tests package this will let you verify your code changes even without using a physical watch or mobile device see the instruction below on how to run unit tests right click on com samsung health hrdatatransfer test , and execute run 'tests in 'com samsung health hrdatatransfer" command if you have completed all the tasks correctly, you will see all the unit tests pass successfully run the app after building the apks, you can run the applications on your watch to measure heart rate and ibi values, and on your mobile device to collect the data from your watch once the app starts, allow the app to receive data from the body sensors afterwards, it shows the application's main screen to get the heart rate and ibi values, tap the start button tap the send button to send the data to your mobile device notethe watch keeps last ~40 values of heart rate and ibi you’re done! congratulations! you have successfully achieved the goal of this code lab now, you can create a health app on a watch to measure heart rate and ibi, and develop a mobile app that receives that health data! if you face any trouble, you may download this file heart rate data transfer complete code 213 9 kb to learn more about samsung health, visit developer samsung com/health
We use cookies to improve your experience on our website and to show you relevant advertising. Manage you settings for our cookies below.
These cookies are essential as they enable you to move around the website. This category cannot be disabled.
These cookies collect information about how you use our website. for example which pages you visit most often. All information these cookies collect is used to improve how the website works.
These cookies allow our website to remember choices you make (such as your user name, language or the region your are in) and tailor the website to provide enhanced features and content for you.
These cookies gather information about your browser habits. They remember that you've visited our website and share this information with other organizations such as advertisers.
You have successfully updated your cookie preferences.