• Learn
    • Code Lab
    • Foldables and Large Screens
    • One UI Beta
    • Samsung Developers Podcasts
  • Develop
    • Mobile/Wearable
    • Galaxy GameDev
    • Galaxy Themes
    • Galaxy Watch
    • Health
    • Samsung Blockchain
    • Samsung DeX
    • Samsung IAP
    • Samsung Internet
    • Samsung Pay
    • Samsung Wallet
    • View All
      • Galaxy AR Emoji
      • Galaxy Accessory
      • Galaxy Edge
      • Galaxy Z
      • Galaxy Performance
      • Galaxy FM Radio
      • Galaxy S Pen Remote
      • Galaxy Sensor Extension
      • PENUP
      • Samsung Automation
      • Samsung Neural
      • Samsung TEEGRIS
      • Samsung eSE SDK
    • Visual Display
    • Smart TV
    • Smart Hospitality Display
    • Smart Signage
    • Digital Appliance
    • Family Hub
    • Platform
    • Bixby
    • Knox
    • SmartThings
    • Tizen.NET
  • Design
    • Design System
    • One UI
    • One UI Watch
    • Smart TV
  • Distribute
    • Galaxy Store
    • TV Seller Office
    • Galaxy Store Games
    • Samsung Podcasts
  • Support
    • Developer Support
    • Remote Test Lab
    • Issues and Bugs Channel
    • Samsung Android USB Driver
    • Galaxy Emulator Skin
  • Connect
    • Blog
    • News
    • Forums
    • Events
    • Samsung Developer Conference
    • SDC22
    • SDC21
    • SDC19 and Previous Events
  • Sign In
Top Global Search Form
Recommendation
  • Blog
  • Code Lab
  • Foldable and Large Screen Optimization
  • Forums
  • Galaxy Emulator Skin
  • Galaxy GameDev
  • Health
  • Remote Test Lab
  • Samsung Developer Conference
  • SDC22
  • Watch Face Studio
All Search Form
Recommendation
    Suggestion
      All Search Form
      Filter
      Filter
      Filter
      • ALL
      • DOCS
      • SDK
      • API REFERENCE
      • CODE LAB (7)
      • BLOG
      • NEWS/EVENTS
      • CODE LAB
        api reference code lab blog news/events
      1. Learn
      2. Code Lab

      codelab

      Implement Multi-Window Picture-in-Picture on a Video Player

      implement multi-window picture-in-picture on a video player objective learn how to implement multi-window picture-in-picture (pip) feature on a video player app for a seamless viewing experience on foldable devices. overview android allows activities of apps to launch in a small window called picture-in-picture (pip) since android 8.0. it is a type of multi-window mode primarily used for video playback. pip allows the user to continue watching in a small window pinned to a corner of the screen while browsing content on the main screen or navigating between apps. set up your environment you will need the following: java se development kit (jdk) 8 or later android studio (latest version recommended) samsung galaxy fold, z fold2, z fold3, or z fold4 remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab. download it and start your learning experience! pip video player sample code (13.30 mb) start your project after downloading the sample project files, follow the steps below to open your project: in android studio, click open. locate the downloaded android project (videoplayer_challenge) from the directory and click ok. configure android manifest to support pip notetake a look and check the files of your project before starting to code. you can implement picture-in-picture in your app by configuring the manifest file and calling the enterpictureinpicturemode method. moreover, you need to consider that the multi-window and app continuity features work correctly in pip mode. to prevent the activity from restarting, handle the layout configuration change for mainactivity in the androidmanifest.xml file. this will ensure that it will not restart when there are layout changes from pip mode to full screen and vice versa. android:configchanges="screensize|smallestscreensize|screenlayout|orientation" picture-in-picture is not supported by default in android apps. set android:supportspictureinpicture to true in the manifest file. android:supportspictureinpicture="true" switch your video player to pip mode you want to trigger pip mode in your video player app when the user taps the home button. to implement pip mode, go to java > com.xyz.codelab and make the following changes to the mainactivity.kt file: override fun onuserleavehint() { enterpipmode() } noteenterpipmode() is a user-defined method. you can view its full implementation in the provided sample code. call enterpictureinpicturemode method in enterpipmode(), you need to call enterpictureinpicturemode(pictureinpictureparams) method with params set in the previous lines. val pictureinpictureparams = pictureinpictureparams.builder().setaspectratio(rational).build() enterpictureinpicturemode(pictureinpictureparams) run the app after building the apk, you can now run the video player app, enhanced with a pip mode. test it by playing a video while browsing the main screen or navigating between apps in multi-window mode. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement multi-window picture-in-picture on a video player app for foldable devices by yourself! if you're having trouble, you may download this file: pip video player complete code (13.19 mb) to learn more about developing apps for galaxy foldable devices, visit: developer.samsung.com/galaxy-z

      https://developer.samsung.com/codelab/galaxy-z/picture-in-picture.html
      1. Learn
      2. Code Lab

      codelab

      Configure an App to Enable Drag and Drop in Multi-Window

      configure an app to enable drag and drop in multi-window objective learn how to implement drag and drop on a note-taking app when in multi-window mode. overview in galaxy fold and it's latest versions, the advantage of its larger display is to split its screen and simultaneously use up to three apps. in multi-window mode, you can split the screen with one window being the main focus, and the other two windows off to the side. all three windows are active, not just the largest one. you can multitask in either landscape or portrait mode, giving you even more flexibility. when using multi-window, drag & drop is one of the useful features when multitasking. drag and drop allows you to easily move data from one app to another. to provide users with better multitasking experience on samsung's foldable devices, developers need to optimize their apps to work on multi-window mode. set up your environment you will need the following: java se development kit (jdk) 8 or later android studio (latest version recommended) samsung galaxy fold, z fold2, z fold3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab. download it and start your learning experience! multi-window (drag & drop) sample code (19.27 mb) start your project open android studio and click open an existing project. locate the downloaded android project (simplenotes_dragdrop) from the directory and click ok. make the app resizable to ensure that the app works in multi-window mode, you need to add an attribute in the manifest’s <activity> element. if you set android:resizeableactivity to true, the activity can be launched in multi-window or in pop-up view, and adapt different screen size. android:resizeableactivity= "true" noteif this attribute is set to true, the activity can be launched in split-screen and freeform modes. otherwise, it will disable multi-window display. remember to handle the changes required to fit your ux in small windows when in multi-window mode. for this code lab, you do not need to worry about the ux as it is already handled. register a drag event listener in newnote.kt, register a drag event listener object by calling setondraglistener for both title and description view of the app. title.setondraglistener(ondraglistener) desc.setondraglistener(ondraglistenerdescription) store the action type to a variable declare a variable to store the action type for the incoming event. val action: int = event.getaction() get and drop the text data inside the dragevent.action_drop, get the item from clipdata and check the mime type. if the mime type is set to text/plain, get the text value from the item object and allow the operation of drop. otherwise, simply show a toast message. val item: item = event.getclipdata().getitemat(0) var mtype = event.clipdescription.getmimetype(0) if(mtype == "text/plain" || mtype== "text/html"){ // gets the text data from the item. dragdata = item.text.tostring() }else{ toast.maketext(applicationcontext,"operation not allowed"+mtype,toast.length_long).show() return@ondraglistener true } tipin this code lab, to simplify the demonstration of implementing drag and drop, mime type or media type is set to plain text only. run the app after building the apk, you can run the optimized note-taking app and start dragging and dropping texts between apps in multi-window mode. if you don’t have any physical device, you can also test it on a remote test lab device. notewatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement drag and drop in your app for your foldable device by yourself! if you're having trouble, you may download this file: multi-window (drag & drop) complete code (19.33 mb) to learn more about developing apps for galaxy foldable devices, visit: www.developer.samsung.com/galaxy-z

      https://developer.samsung.com/codelab/galaxy-z/multi-window-drag-drop.html
      1. Learn
      2. Code Lab

      codelab

      Configure an App to Enable Copy and Paste in Multi-Window

      configure an app to enable copy and paste in multi-window objective learn how to implement copy and paste on a note-taking app when in multi-window mode. overview in galaxy fold and it's latest versions, the advantage of its larger display is to split its screen and simultaneously use up to three apps. in multi-window mode, you can split the screen with one window being the main focus, and the other two windows off to the side. all three windows are active, not just the largest one. you can multitask in either landscape or portrait mode, giving you even more flexibility. when using multi-window, copy & paste is one of the useful features when multitasking. copy and paste is a common feature used by many, to duplicate an object and place it in a desired location. to provide users with better multitasking experience on samsung's foldable devices, developers need to optimize their apps to work on multi-window mode. set up your environment you will need the following: java se development kit (jdk) 8 or later android studio (latest version recommended) samsung galaxy fold, z fold2, z fold3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab. download it and start your learning experience! multi-window (copy & paste) sample code (19.48 mb) start your project open android studio and click open an existing project. locate the downloaded android project (simplenotes_copypaste) from the directory and click ok. make the app resizable to ensure that the app works in multi-window mode, you need to add an attribute in the manifest’s <activity> element. if you set android:resizeableactivity to true, the activity can be launched in multi-window, pop-up view and adapt different screen size. android:resizeableactivity= "true" noteif this attribute is set to true, the activity can be launched in split-screen and free-form modes. otherwise, it will disable multi-window display. remember to handle the changes required to fit your ux in small windows when in multi-window mode. for this code lab, you do not need to worry about the ux as it is already handled. get a reference to clipboardmanager class clipboardmanager provides methods to get and set the current primary clipboard data expressed as a clipdata object, which defines the protocol for data exchange between applications. in newnote.kt, get a reference to clipboardmanager class by invoking getsystemservice(clipboard_service). val clipboard = getsystemservice(context.clipboard_service) as clipboardmanager complete the copy operation now, create a new clipdata of simple text using newplaintext of clipboardmanager and pass it on setprimaryclip to complete the copy operation. val clip: clipdata = clipdata.newplaintext("simple text", textdata) clipboard.setprimaryclip(clip) get the text data from the clipboard return the clipdata by getting the primaryclip data from the clipboard. from the primaryclip, acquire the item using getitemat(0). val itemtext = clipboard.primaryclip?.getitemat(0)?.text run the app after building the apk, you can run the optimized app and test it by copying and pasting texts between apps when in multi-window mode. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement multi-window features such as copy and paste in your app by yourself! if you're having trouble, you may download this file: multi-window (copy & paste) complete code (19.53 mb) to learn more about developing apps for galaxy foldable devices, visit: www.developer.samsung.com/galaxy-z

      https://developer.samsung.com/codelab/galaxy-z/multi-window-copy-paste.html
      1. Learn
      2. Code Lab

      codelab

      Develop a Camera Web App on Foldables

      develop a camera web app on foldables objective learn how to develop a camera web application that detects partially folded posture and adjusts its layout accordingly to improve user's experience on samsung foldable devices. in order to create a camera web app for foldable devices, you will use the following: html5 and css modern features webrtc device posture api viewport segment api overview foldable devices are here! these smartphones allow different form factors with just one device, which can lead to new opportunities for innovation. as developers should be able to detect what is the current posture of the device, the physical position in which a device holds, samsung is putting an effort within the w3c on making a new web standard called device posture api. this api allows web applications to request and be notified of changes of the device posture. together with the device posture api, the viewport segment api enables web developers to build layouts that are optimized for the dual-screen form factor of foldables. web apps can take advantage of the new form factors and improve user's experience. set up your environment you will need the following: samsung internet (at least v19.0) samsung galaxy foldable device: galaxy z fold2, z fold3, or z fold4 galaxy z flip, z flip3, or z flip4 if physical device is not available for testing, use: a. remote test lab requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available b. device posture api polyfill for testing using a non-foldable device, just implement a polyfill to the project. start your project open a new browser tab, go to https://glitch.com, and log in. click new project, then select import from github. enter the url of this repository. add html video markup notetake a look and check the files of your project before starting to code. in index.html, you will have two main sections, one is dedicated to the video camera and the other one to the camera controls. you can identify the video section as it contains the class video-container. complete the html markup by adding the <video> element, width, height, and id attributes. <div class="video-container"> <video id="video" width="1280" height="200" > video stream not available. </video> </div> once the video markup is added, the html file is complete. besides this, you can find: a <canvas> element into which the captured frames are stored and kept hidden, since users don’t need to see it. an <img> element where the pictures will be displayed. enable front and back cameras with facingmode in index.js, you'll find a startup() function. here is where you: initialize most values. grab element references to use later. set up the camera and image. start by setting up the camera. you will have access to both front and back cameras by using facingmode, a domstring that indicates the direction on which camera is being used to capture images. it's always a good practice to check if this function is available for use. add the following lines of code in startup(): let supports = navigator.mediadevices.getsupportedconstraints(); if (supports["facingmode"] === true) { flip_button.disabled = false; } link the media stream to the video element if you are able to use facingmode, the camera web app will use this option in capture function to detect which is the current camera used and later send it to the flip button. now, the camera should be activated, insert this block of code after retrieving defaultopts.video value using facingmode: navigator.mediadevices .getusermedia(defaultsopts) .then(_stream => { stream = _stream; video.srcobject = stream; video.play(); }) .catch(error => console.error(error)); in order to get the media stream, you call navigator.mediadevices.getusermedia() and request a video stream which returns a promise. the success callback receives a stream object as input. it is the <video> element's source to the new stream. once the stream is linked to the <video> element, start it playing by calling video.play(). it's always a good practice to include the error callback too, just in case the camera is not available or the permissions are denied. take a look at the implementations in index.js at this point, the functionality of the web app is complete. before moving to the next step, let’s review the rest of the javascript code: there is an event listener video for the canplay event, which will check when the video playback begins. if it's the first time running, it will set up video attributes like width and height. for the snip button, there is an event listener for click that will capture the picture. the flip button will be waiting for a click event to assign the flag about which camera is being used, front or back camera, within the variable shouldfaceuser, and initialize the camera again. clearpicture() creates an image and converts it to a format that will be displayed in the <img> element. finally, takepicture() captures the currently displayed video frame, converts it into a png file, and displays it in the captured frame box. use device posture and viewport segment apis at this point, you should have a working prototype of a camera web app. the video camera should be displayed, and a picture may be taken using the snip button. in addition, it will show a preview of the picture taken through the small preview display. the flip button allows you also to switch between front and back cameras. now, it's time to play around the layout of the web app and take advantage of the features available on a foldable device. in order to do that, you will implement the device posture api that allows developers to detect what is the current posture of the phone. in order to change the layout when the device is partially folded, the device posture that you will look for is in a form of a book or laptop. the viewport segments api, in the other hand, is an experimental media feature designed to detect if your website is being displayed on a dual-screen device. the media feature for this is called viewport-segments. there are two types: horizontal-viewport-segments and vertical-viewport-segments. you can use these media queries combined with the device posture api to also check if the device is currently in a horizontal or vertical orientation within its hinge, which can be really helpful for samsung devices. the galaxy z fold devices have a vertical hinge orientation while the galaxy z flip devices have a horizontal hinge. with the viewport segment api, you can use horizontal-viewport-segment for galaxy z fold devices and vertical-viewport-segments for galaxy z flip devices, as illustrated on the diagram below. apply the following media query in style.css: @media (vertical-viewport-segments: 2) and (device-posture: folded) { body { display: flex; flex-flow: column nowrap; display: block; } .camera-controls { top: env(viewport-segment-top 0 1); height: env(viewport-segment-height 0 1); } .msg { display: block; margin: 3em; } } using modern css features like display:flex, you can change the layout of the body element and the elements with the class video-container and camera-controls when the device is in flex mode. test your app whether you test on a real foldable phone or on a remote test lab device, you need to enable the device posture api in the latest version of samsung internet. to do this, open the browser and type internet://flags in the url bar and search for either device posture api or screen fold api, then select enable. test in a real device if you have a real physical device, you can test it directly in samsung internet using the url that glitch provides you by clicking on show in a new window menu. just partially fold your phone, and you will see how the layout changes and even discover a hidden message! use remote test lab the other option, if you don’t have a physical device, is by using remote test lab. you can choose any galaxy foldable devices from the list and follow the same instructions as you would with a real device. just make sure to enable the device posture api and have the latest version of samsung internet. use the buttons provided by remote test lab to partially fold your remote galaxy device. implement polyfill polyfill allows you to emulate behavior on devices that do not have folding capabilities. it helps you visualize how the content responds to the different angle and posture configurations. just include sfold-polyfill.js directly into your code and use the polyfill settings (screenfold-settings.js) of the web component that will emulate the angle of the device and therefore, it will change its posture. moreover, add the following code in index.html. <head> … <script type='module' defer src="screenfold-settings.js"></script> <script src="sfold-polyfill.js"></script> … </head> <body> <section> <screenfold-settings></screenfold-settings> </section> … </body> noteas the current polyfill has a previous version of api just replace the media query with following situations: when testing using a personal computer either a laptop or a desktop, use @media (screen-fold-posture: laptop) when testing using a regular phone that's not a foldable, use @media (screen-fold-posture: book) you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can create a camera web app that changes its layout when a device is partially folded. if you're having trouble, you may check the complete code here. learn more with these resources: overview of galaxy z web responsive design of foldable devices folding the web current web on galaxy fold viewport segment api

      https://developer.samsung.com/codelab/internet/foldable-camera-app.html
      1. Learn
      2. Code Lab

      codelab

      Implement App Continuity and Optimize Large Screen UI of a Gallery App

      implement app continuity and optimize large screen ui of a gallery app objective learn how to apply app continuity and large screen optimization on a gallery app for seamless experience on foldable devices. overview samsung foldable devices have two different physical screens: the cover display and the larger main display. app continuity lets you experience a seamless transition between these displays. when using an app that supports app continuity, you can unfold the device and continue where you left off. with this feature, apps can run without interruption when it receives a configuration change event, such as when a device is folded or unfolded. it is essential to continue the operation by keeping the same state and location of the app after the transition. large screen optimization plays another significant role in terms of ux designing for foldable devices. the screens of foldable devices have a wide range of aspect ratios, careful ux design is needed to suit two different size displays. set up your environment you will need the following: java se development kit (jdk) 8 or later android studio (latest version recommended) samsung galaxy fold, z fold2, z fold3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available sample code here is a sample code for you to start coding in this code lab. download it and start your learning experience! app continuity sample code (2.80 mb) start your project after downloading the sample code containing the project file for this code lab, open your android studio and click open an existing project. locate the downloaded android project from the directory and click ok. handle the configuration change to have a seamless experience when using a gallery app, the photo that is displayed after folding or unfolding the device should be the same as the previous. you need to store the position or id of the photo, as android destroys and recreate the activity when folding and unfolding the phone. use bundle with its key/value pair to store and restore the values. in addition, you need to consider the screen rotation in order to keep the app running, as android restarts the activity of the app after the configuration changes. let's first start to prevent the activity from restarting, by handling the configuration change for mainactivity in androidmanifest.xml file. android:configchanges="keyboardhidden|orientation|screensize" save the instance state by storing the data before the activity is destroyed when the screen changes from main to cover display or vice versa, android provides us an override method called onsaveinstancestate. it accepts a bundle as a parameter. use putint method to store data into key/value pair. override fun onsaveinstancestate(outstate: bundle) { outstate.putint("currentposition", currentimageposition) log.i("tag", "onsave "+currentimageposition) super.onsaveinstancestate(outstate) } retrieve the stored values when the app is first developed, the savedinstancestate will be null inside the oncreate method. if it is not null, you can retrieve the stored values from the bundle using getint and the key name you set up in the previous step while storing. if(savedinstancestate != null) { selectedimageview!!.setimageresource(images[savedinstancestate.getint("currentposition")]) customgalleryadapter!!.updateposition(images[savedinstancestate.getint("currentposition")]) currentimageposition = savedinstancestate.getint("currentposition") log.i("tag", "onsaved " + savedinstancestate.getint("currentposition")) } create a layout file for the main display the next thing to do, is to optimize the app's large screen ui. here, you will add a navigator for the photos when using the app in the main display, since it has more space. android offers an alternate resources option based on various criteria like display size, density, and other more. you can use two different layout files for each display using alternate resources of android. create a new directory in res in the form of [resource]-[qualifier] for cover display and main display. the appropriate version of the resource is shown automatically by the system based on the smallest width qualifier of sw. the cover display will be between sw240 and sw480, while the main display will be sw600. create a new directory named layout-sw600dp under the res directory and then create an xml file named activity_main. res/ layout/ activity_main.xml layout-sw600dp/ activity_main.xml add the layout code add the gallery layout code shown below, in layout-sw600dp/activity_main.xml file. <?xml version="1.0" encoding="utf-8"?> <androidx.constraintlayout.widget.constraintlayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:app="http://schemas.android.com/apk/res-auto" xmlns:tools="http://schemas.android.com/tools" android:id="@+id/linearlayout" android:layout_width="match_parent" android:layout_height="match_parent" tools:context="com.xyz.gallery.mainactivity"> <imageview android:id="@+id/imageview" android:layout_width="wrap_content" android:layout_height="match_parent" android:gravity="center_vertical" app:layout_constraintend_toendof="parent" app:layout_constraintstart_tostartof="parent" app:layout_constrainttop_totopof="parent" /> <gallery android:id="@+id/allgallery" android:layout_width="fill_parent" android:layout_height="108dp" android:layout_marginbottom="4dp" android:gravity="fill" android:orientation="horizontal" android:padding="10dp" android:spacing="5dp" app:layout_constraintbottom_tobottomof="parent" app:layout_constraintend_toendof="parent" app:layout_constrainthorizontal_bias="0.0" app:layout_constraintstart_tostartof="parent" /> </androidx.constraintlayout.widget.constraintlayout> run the app after building the apk, you can run the gallery app and see how it change when you fold and unfold the device. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. noteapp continuity when folding the device, can be turned on for selected apps in settings (if the app supports app continuity). to continue using the gallery app on the cover display when you close your phone, go to settings> display > continue apps on cover screen > toggle on gallery app. foldable devices let you decide whether you want to see less or more content at the same time in the main display. to see the photo navigator added in the app, go to settings> display > screen layout and zoom > see more content at the same time. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can optimize your app to support continuity and large screen layout by yourself! if you're having trouble, you may download this file: app continuity complete code (24.44 mb) to learn more about developing apps for galaxy foldable devices, visit: www.developer.samsung.com/galaxy-z

      https://developer.samsung.com/codelab/galaxy-z/app-continuity.html
      1. Learn
      2. Code Lab

      codelab

      Implement Flex Mode on an Unreal Engine Game

      implement flex mode on an unreal engine game objective learn how to implement flex mode on an unreal engine game using android jetpack windowmanager and raw java native interface (jni). overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps. when the phone is partially folded, it will go into flex mode. apps will reorient to fit the screen, letting you watch videos or play games without holding the phone. for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate. unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode. to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard. set up your environment you will need the following: epic games launcher with unreal engine 4 or later visual studio or any source code editor samsung galaxy foldable device: galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available create and set up your project after launching unreal engine from the epic games launcher, follow the steps below to start your project: in the select or create new project window, choose games as a new project category and click next. select third person as template, then click next to proceed. noteyou can implement flex mode on any template or existing projects and use this code lab activity as a reference. in the project settings window, set the following: type of project: c++ target platform: mobile / tablet performance characteristics: scalable 3d or 2d real-time raytracing: raytracing disabled include an additional content pack: no starter content project name: tutorial_project click create project. wait for the engine to finish loading and open the unreal editor. once the project is loaded, go to edit > project settings > platforms > android. click the configure now button if the project is not yet configured for the android platform. then, proceed with the following apk packaging and build settings: a. apk packaging set target sdk version to 30. set orientation to full sensor. change the maximum supported aspect ratio to 2.8 (aspect ratio of galaxy z fold3 in decimal) to prevent black bars from appearing on the cover display. leave it if your game does not need to use the cover display. enable use display cutout region?, to prevents black bars at the edge of the main screen. otherwise, leave it unchecked. b. build disable support opengl es3.1 and enable support vulkan. notecurrently, there is a problem with opengl es and the split-screen system being investigated. the only option right now is to turn off opengl es and use vulkan instead. enable native resize event the resize event of a game when switching between displays is disabled in the engine by default. however, this behavior can be easily enabled by setting android.enablenativeresizeevent=1 in the deviceprofile. currently, the only way to create a profile for foldable devices is by creating a specific rule for each device. to save time in this code lab, enable the native resize event for all android devices instead. locate and open the tutorial_project > config folder in file explorer. inside the config folder, create a new folder named android. create a new file called androiddeviceprofiles.ini and open this file in a text editor, such as visual studio. copy below deviceprofile code to the newly created androiddeviceprofiles.ini file. [deviceprofiles] +deviceprofilenameandtypes=android,android [android deviceprofile] devicetype=android baseprofilename= +cvars=r.mobilecontentscalefactor=1.0 +cvars=slate.absoluteindices=1 +cvars=r.vulkan.delayacquirebackbuffer=2 +cvars=r.vulkan.robustbufferaccess=1 +cvars=r.vulkan.descriptorsetlayoutmode=2 ; don't enable vulkan by default. specific device profiles can set this cvar to 0 to enable vulkan. +cvars=r.android.disablevulkansupport=1 +cvars=r.android.disablevulkansm5support=1 ; pf_b8g8r8a8 +cvars=r.defaultbackbufferpixelformat=0 +cvars=android.enablenativeresizeevent=1 ; previewallowlistcvars and previewdenylistcvars are arrays of cvars that are included or excluded from being applied in mobile preview. ; if any previewallowlistcvars is set, cvars are denied by default. previewallowlistcvars=none this is a copy of the default android deviceprofile from the existing basedeviceprofiles.ini file but with the enabled nativeresizeevent console variable (cvars). notethis step is not required when you only want to implement flex mode. yet, it's recommended, to allow applications to run seamlessly from main to cover display without stretching and squashing the game, by enabling the nativeresizeevent. create a new plugin and import the foldablehelper foldablehelper is a java file that you can use in different projects. it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments. before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device (flat for normal mode and half-opened for flex mode), window size, and orientation of the fold on the screen. download the foldablehelper.java file here: foldablehelper.java (5.64 kb) to import the foldablehelper.java file to the project, follow the steps below: go to edit > plugins in the unreal editor. click the new plugin button and select blank to create a blank plugin. in the name field, type foldables_tutorial and click the create plugin button. in file explorer, locate and open tutorial_project > plugins folder. go to plugins > foldables_tutorial > source> foldables_tutorial > private and create a new folder called java. copy the foldablehelper.java file into java folder. open the tutorial_project.sln file in visual studio. in the same private folder path, add a new filter called java. right-click on the java filter and click add > existing item. locate the foldablehelper.java file, then click add to include this java file in the build. modify java activity to use foldablehelper unreal plugin language (upl) is a simple xml-based language created by epic games for manipulating xml and returning strings. using upl, you can utilize the foldablehelper.java file by modifying the java activity and related gradle files as follows: in visual studio, right-click on source > foldables_tutorial folder, then click add > new item > web > xml file (.xml). create an xml file called foldables_tutorial_upl.xml. ensure that the file location is correct before clicking add. in the newly created xml file, include the foldablehelper.java file in the build by copying the java folder to the build directory. <root xmlns:android="http://schemas.android.com/apk/res/android"> <prebuildcopies> <copydir src="$s(plugindir)/private/java" dst="$s(builddir)/src/com/samsung/android/gamedev/foldable" /> </prebuildcopies> set up the gradle dependencies in the build.gradle file by adding the following in the xml file: <buildgradleadditions> <insert> dependencies { implementation filetree(dir: 'libs', include: ['*.jar']) implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.0" implementation "androidx.core:core:1.7.0" implementation "androidx.core:core-ktx:1.7.0" implementation "androidx.appcompat:appcompat:1.4.0" implementation "androidx.window:window:1.0.0" implementation "androidx.window:window-java:1.0.0" } android{ compileoptions{ sourcecompatibility javaversion.version_1_8 targetcompatibility javaversion.version_1_8 } } </insert> </buildgradleadditions> next, modify the gameactivity: <gameactivityimportadditions> <insert> <!-- package name of foldablehelper --> import com.samsung.android.gamedev.foldable.foldablehelper; </insert> </gameactivityimportadditions> <gameactivityoncreateadditions> <insert> foldablehelper.init(this); </insert> </gameactivityoncreateadditions> <gameactivityonstartadditions> <insert> foldablehelper.start(this); </insert> </gameactivityonstartadditions> <gameactivityonstopadditions> <insert> foldablehelper.stop(); </insert> </gameactivityonstopadditions> </root> gameactivityimportadditions adds the com.samsung.android.gamedev.foldable.foldablehelper into the gameactivity with the existing imports. gameactivityoncreateadditions adds the code to the oncreate() method inside the gameactivity. gameactivityonstartadditions adds the code to the onstart() method inside the gameactivity. gameactivityonstopadditions adds the code to the onstop() method inside the gameactivity. save the xml file. then, ensure that the engine uses the upl file by modifying the foldables_tutorial.build.cs script, located in the same folder as the foldables_tutorial_upl.xml file. after the dynamicallyloadedmodulenames.addrange call, add the following: if (target.platform == unrealtargetplatform.android) { additionalpropertiesforreceipt.add("androidplugin", moduledirectory + "\\foldables_tutorial_upl.xml"); } this means that the game engine will use the upl file if the platform is android. otherwise, the foldablehelper won’t work. implement a storage struct the next thing to implement is a struct, the native version of java’s foldablelayoutinfo class. to store the data retrieved from the java code using a struct, do the following: in content browser of unreal editor, right-click on c++ classes > add/import content. then, click new c++ class. select none for the parent class and click next. name the new class as foldablelayoutinfo. assign it to the foldables_tutorial plugin. then, click create class. delete the created foldablelayoutinfo.cpp file and only keep its header file. in the header file called foldablelayoutinfo.h, set up a struct to store all needed data from the windowmanager. #pragma once #include "core.h" enum efoldstate { undefined_state, flat, half_opened }; enum efoldorientation { undefined_orientation, horizontal, vertical }; enum efoldocclusiontype { undefined_occlusion, none, full }; struct ffoldablelayoutinfo { efoldstate state; efoldorientation orientation; efoldocclusiontype occlusiontype; fvector4 foldbounds; fvector4 currentmetrics; fvector4 maxmetrics; bool isseparating; ffoldablelayoutinfo() : state(efoldstate::undefined_state), orientation(efoldorientation::undefined_orientation), occlusiontype(efoldocclusiontype::undefined_occlusion), foldbounds(-1, -1, -1, -1), currentmetrics(-1, -1, -1, -1), maxmetrics(-1, -1, -1, -1), isseparating(false) { } }; implement jni code to implement jni, create a new c++ class with no parent and name it foldables_helper. assign the class to the same plugin, then modify the c++ header and source files as follows: in the created header file (foldables_helper.h), include foldablelayoutinfo.h. #include "foldablelayoutinfo.h" then, declare a multicast_delegate to serve as a listener for passing the data from the java implementation to the rest of the engine. declare_multicast_delegate_oneparam(fonlayoutchangeddelegate, ffoldablelayoutinfo); lastly, set up the methods and member variables. class foldables_tutorial_api ffoldables_helper { public: static void init(); static bool haslistener; static fonlayoutchangeddelegate onlayoutchanged; }; moving to the source file (foldables_helper.cpp), set up the definitions for the methods and member variables created in the header file. bool ffoldables_helper::haslistener = false; fonlayoutchangeddelegate ffoldables_helper::onlayoutchanged; void ffoldables_helper::init() { haslistener = true; } now, in the same source file, create the native version of the onlayoutchanged() function created in the foldablehelper.java file. since the java onlayoutchanged() function only works on android, surround the function with an #if directive to ensure that it compiles only on android. #if platform_android #endif within this directive, copy the code below to use the jni definition of the java onlayoutchanged() function. extern "c" jniexport void jnicall java_com_samsung_android_gamedev_foldable_foldablehelper_onlayoutchanged(jnienv * env, jclass clazz, jobject jfoldablelayoutinfo) { create the ffoldablelayoutinfo to store the data retrieved from java. ffoldablelayoutinfo result; retrieve the field ids of the foldablelayoutinfo and rect objects created in the java file. //java foldablelayoutinfo field ids jclass jfoldablelayoutinfocls = env->getobjectclass(jfoldablelayoutinfo); jfieldid currentmetricsid = env->getfieldid(jfoldablelayoutinfocls, "currentmetrics", "landroid/graphics/rect;"); jfieldid maxmetricsid = env->getfieldid(jfoldablelayoutinfocls, "maxmetrics", "landroid/graphics/rect;"); jfieldid hingeorientationid = env->getfieldid(jfoldablelayoutinfocls, "hingeorientation", "i"); jfieldid stateid = env->getfieldid(jfoldablelayoutinfocls, "state", "i"); jfieldid occlusiontypeid = env->getfieldid(jfoldablelayoutinfocls, "occlusiontype", "i"); jfieldid isseparatingid = env->getfieldid(jfoldablelayoutinfocls, "isseparating", "z"); jfieldid boundsid = env->getfieldid(jfoldablelayoutinfocls, "bounds", "landroid/graphics/rect;"); jobject currentmetricsrect = env->getobjectfield(jfoldablelayoutinfo, currentmetricsid); //java rect object field ids jclass rectcls = env->getobjectclass(currentmetricsrect); jfieldid leftid = env->getfieldid(rectcls, "left", "i"); jfieldid topid = env->getfieldid(rectcls, "top", "i"); jfieldid rightid = env->getfieldid(rectcls, "right", "i"); jfieldid bottomid = env->getfieldid(rectcls, "bottom", "i"); retrieve the current windowmetrics and store it in the ffoldablelayoutinfo as an fintvector4. // currentmetrics int left = env->getintfield(currentmetricsrect, leftid); int top = env->getintfield(currentmetricsrect, topid); int right = env->getintfield(currentmetricsrect, rightid); int bottom = env->getintfield(currentmetricsrect, bottomid); // store currentmetrics rect to fvector4 result.currentmetrics = fintvector4{ left, top, right, bottom }; do the same for the other variables in the java foldablelayoutinfo. // maxmetrics jobject maxmetricsrect = env->getobjectfield(jfoldablelayoutinfo, maxmetricsid); left = env->getintfield(maxmetricsrect, leftid); top = env->getintfield(maxmetricsrect, topid); right = env->getintfield(maxmetricsrect, rightid); bottom = env->getintfield(maxmetricsrect, bottomid); //store maxmetrics rect to fvector4 result.maxmetrics = fintvector4{ left, top, right, bottom }; int hingeorientation = env->getintfield(jfoldablelayoutinfo, hingeorientationid); int state = env->getintfield(jfoldablelayoutinfo, stateid); int occlusiontype = env->getintfield(jfoldablelayoutinfo, occlusiontypeid); bool isseparating = env->getbooleanfield(jfoldablelayoutinfo, isseparatingid); // store the values to an object for unreal result.orientation = tenumasbyte<efoldorientation>(hingeorientation + 1); result.state = tenumasbyte<efoldstate>(state + 1); result.occlusiontype = tenumasbyte<efoldocclusiontype>(occlusiontype + 1); result.isseparating = isseparating; // boundsrect jobject boundsrect = env->getobjectfield(jfoldablelayoutinfo, boundsid); left = env->getintfield(boundsrect, leftid); top = env->getintfield(boundsrect, topid); right = env->getintfield(boundsrect, rightid); bottom = env->getintfield(boundsrect, bottomid); // store maxmetrics rect to fvector4 result.foldbounds = fintvector4{ left, top, right, bottom }; broadcast the result via the onlayoutchanged delegate for use in the engine. if (ffoldables_helper::haslistener) { ue_log(logtemp, warning, text("broadcast")); ffoldables_helper::onlayoutchanged.broadcast(result); } } create a player controller and two ui states this section focuses on adding a player controller and creating two user interface (ui) states for flat and flex modes. these objects are needed for the flex mode logic implementation. following are the steps to add a player controller and create two ui states : add a new player controller blueprint. in content browser, go to content > thirdpersoncpp and right-click on blueprints > add/import content > blueprint class. pick player controller as its parent class. rename it as flexplayercontroller. notethe flexplayercontroller added is generic and can be replaced by your custom player controller in an actual project. add a new c++ class with actor component as its parent class. name it as foldables_manager and assign it to the foldables_tutorial plugin. click the create class button. open the flexplayercontroller blueprint by double-clicking it. click open full blueprint editor. attach the actor component to the flexplayercontroller. in the left pane, click add component, then find and select the foldables_manager. next, create a pair of userwidget classes for the ui layouts needed: flat mode ui for the full screen or normal mode; and flex mode ui for split-screen. in add c++ class window, select the show all classes checkbox. find and pick userwidget as the parent class. then, click next. name the new user widget as flatui and attach it to the plugin. click next. repeat the process but name the new user widget as flexui. you might get an error when trying to compile stating that the userwidget is an undeclared symbol. to fix this, open the foldables_tutorial.build.cs file, and in the publicdependencymodulenames.addrange call, add "inputcore" and "umg" to the list. create a pair of blueprints made from subclasses of these two user widgets. right-click on content and create a new folder called foldui. inside the newly created folder, right-click to add a new blueprint class. in all classes, choose flatui and click the select button. rename the blueprint as bp_flatui. in the same folder, repeat the process but choose the flexui class and rename the blueprint as bp_flexui. double-click on bp_flatui and bp_flexui, then design your two uis like below to visualize switching between flat and flex mode: flat ui flex ui notethis code lab activity does not cover the steps on how to create or design ui. if you want to learn about how to create your own game design in unreal engine 4, refer to unreal motion graphics ui designer guide. implement the flex mode logic after creating the flexplayercontroller and the two ui states (bp_flatui and bp_flexui), you can now implement flex mode logic in the foldables_manager. open the foldables_manager.h and include the necessary c++ header files: #pragma once #include "coreminimal.h" #include "components/actorcomponent.h" #include "engine.h" #include "flatui.h" #include "flexui.h" #include "foldables_helper.h" #include "foldables_manager.generated.h" remove the line below to save a little bit of performance as this component doesn't need to tick. public: // called every frame virtual void tickcomponent(float deltatime, eleveltick ticktype, factorcomponenttickfunction* thistickfunction) override; set up the functions needed in foldables_manager: the constructor, a function to create the ui widgets the implementation of onlayoutchanged delegate public: // sets default values for this component's properties ufoldables_manager(); void createwidgets(); protected: // called when the game starts virtual void beginplay() override; protected: void onlayoutchanged(ffoldablelayoutinfo foldablelayoutinfo); then, set up the variables needed: references to the flat and flex ui classes references to the flat and flex ui objects mark the pointers as uproperty to ensure that garbage collection does not delete the objects they point to. tsubclassof<uuserwidget> flatuiclass; tsubclassof<uuserwidget> flexuiclass; uproperty() class uflatui* flatui; uproperty() class uflexui* flexui; finally, define a new private function restoreflatmode(), to disable flex mode and return to normal mode. private: void restoreflatmode(); moving over to foldables_manager.cpp, implement the constructor. using the constructorhelpers, find the ui classes and set the variables to store these classes. also, set the bcanevertick to false to prevent the component from ticking and remove the code block of tickcomponent() function. // sets default values for this component's properties ufoldables_manager::ufoldables_manager() { primarycomponenttick.bcanevertick = false; static constructorhelpers::fclassfinder<uflatui> flatuibpclass(text("/game/foldui/bp_flatui")); static constructorhelpers::fclassfinder<uflexui> flexuibpclass(text("/game/foldui/bp_flexui")); if (flatuibpclass.succeeded()) { flatuiclass = flatuibpclass.class; } if (flexuibpclass.succeeded()) { flexuiclass = flexuibpclass.class; } } next, set up the beginplay() function to link the delegate to the onlayoutchanged() function, to initialize the foldables_helper, and to create the widgets ready for use in the first frame. // called when the game starts void ufoldables_manager::beginplay() { super::beginplay(); ffoldables_helper::onlayoutchanged.adduobject(this, &ufoldables_manager::onlayoutchanged); ffoldables_helper::init(); createwidgets(); } set up the createwidgets() function to create the widgets using the ui classes acquired in the constructor. add the flatui widget to the viewport, assuming the app opens in normal mode until it receives the foldablelayoutinfo. void ufoldables_manager::createwidgets() { flatui = createwidget<uflatui>((aplayercontroller*)getowner(), flatuiclass, fname(text("flatui"))); flexui = createwidget<uflexui>((aplayercontroller*)getowner(), flexuiclass, fname(text("flexui"))); flatui->addtoviewport(); } afterward, create the onlayoutchanged() function, which will be called via a delegate. inside this function, check whether the device’s folded state is half_opened. if so, check whether the orientation of the fold is horizontal. void ufoldables_manager::onlayoutchanged(ffoldablelayoutinfo foldablelayoutinfo) { //if state is now flex if (foldablelayoutinfo.state == efoldstate::half_opened) { if (foldablelayoutinfo.orientation == efoldorientation::horizontal) { notefor this third person template, splitting the screen vertically isn’t ideal from a user experience (ux) point of view. for this code lab activity, split the screen only on the horizontal fold (top and bottom screen). if you want it vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis. you must also ensure that you have a flex ui object for the vertical layout. if the device is both on flex mode and horizontal fold, change the viewport to only render on the top screen using the normalized position of the folding feature. then in an asynctask on the game thread, disable the flatui and enable the flexui. however, if the device is on normal mode, then return to flat ui using restoreflatmode() function. //horizontal split float foldratio = (float)foldablelayoutinfo.foldbounds.y / (foldablelayoutinfo.currentmetrics.w - foldablelayoutinfo.currentmetrics.y); gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizex = 1.0f; gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizey = foldratio; asynctask(enamedthreads::gamethread, [=]() { if (flatui->isinviewport()) { flatui->removefromparent(); } if (!flexui->isinviewport()) { flexui->addtoviewport(0); } }); } else { restoreflatmode(); } } else { restoreflatmode(); } } reverse the flex mode implementation logic to create the restoreflatmode() function by setting the viewport to fill the screen, then disable the flexui and enable the flatui. void ufoldables_manager::restoreflatmode() { gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizex = 1.0f; gengine->gameviewport->splitscreeninfo[0].playerdata[0].sizey = 1.0f; asynctask(enamedthreads::gamethread, [=]() { if (!flatui->isinviewport()) { flatui->addtoviewport(0); } if (flexui->isinviewport()) { flexui->removefromparent(); } }); } set up a game mode and attach the flexplayercontroller the game mode defines the game rules, scoring, and any game-specific behavior. set up the game mode in unreal editor by creating a blueprint class in the content > thirdpersoncpp > blueprints folder. pick game mode base as the parent class and rename it as flexgamemode. double-click on flexgamemode. in the drop-down menu next to player controller class, choose the flexplayercontroller. lastly, go to edit > project settings > project > maps & modes and select flexgamemode as the default gamemode. build and run the app go to edit > package project > android to build the apk. ensure that the android development environment for unreal is already set up to your computer. noteif the build failed due to corrupted build tools, consider downgrading the version to 30 or lower using the sdk manager. or, simply rename d8.bat to the name of the missing file (dx.bat) in (sdk path) > build-tools > (version number) directory and, in lib folder of the same directory, rename d8.jar to dx.jar. after packaging your android project, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement flex mode in your unreal engine game app by yourself! if you're having trouble, you may download this file: flex mode on unreal complete code (20.16 mb) to learn more, visit: www.developer.samsung.com/galaxy-z www.developer.samsung.com/galaxy-gamedev

      https://developer.samsung.com/codelab/gamedev/flex-mode-unreal.html
      1. Learn
      2. Code Lab

      codelab

      Implement Flex Mode on a Unity Game

      implement flex mode on a unity game objective learn how to implement flex mode on a unity game using android jetpack windowmanager and unity's java native interface (jni) wrapper. overview the flexible hinge and glass display on galaxy foldable devices, such as the galaxy z fold4 and galaxy z flip4, let the phone remains propped open while you use apps. when the phone is partially folded, it will go into flex mode. apps will reorient to fit the screen, letting you watch videos or play games without holding the phone. for example, you can set the device on a flat surface, like on a table, and use the bottom half of the screen to navigate. unfold the phone to use the apps in full screen mode, and partially fold it again to return to flex mode. to provide users with a convenient and versatile foldable experience, developers need to optimize their apps to meet the flex mode standard. set up your environment you will need the following: unity hub with unity 2020.3.31f1 or later (must have android build support) visual studio or any source code editor samsung galaxy foldable device: galaxy z fold2, z fold3, or newer galaxy z flip, z flip3, or newer remote test lab (if physical device is not available) requirements: samsung account java runtime environment (jre) 7 or later with java web start internet environment where port 2600 is available sample code here is a sample project for you to start coding in this code lab. download it and start your learning experience! flex mode on unity sample code (43.39 kb) start your project after downloading the sample project files, follow the steps below to open your project: launch the unity hub. click projects > open. locate the unzipped project folder and click open to add the project to the hub and open in the editor. notethe sample project was created in unity 2020.3.31f1. if you prefer using a different unity version, click choose another editor version when prompted and select a higher version of unity. configure android player settings to ensure that the project runs smoothly on the android platform, configure the player settings as follows: go to file > build settings. under platform, choose android and click switch platform. wait until this action finishes importing necessary assets and compiling scripts. then, click player settings to open the project settings window. go to player > other settings and scroll down to see target api level. set it to api level 31 as any less than this will result in a dependency error regarding an lstar variable. you can set the minimum api level on lower levels without any problem. next, in the resolution and presentation settings, enable resizable window. it is also recommended that render outside safe area is enabled to prevent black bars on the edges of the screen. lastly, enable the custom main manifest, custom main gradle template, and custom gradle properties template in the publishing settings. after closing the project settings window, check for the new folder structure created within your assets in the project window. the newly created android folder contains androidmanifest.xml, gradletemplate.properties, and maintemplate.gradle files. import the foldablehelper and add dependencies foldablehelper is a java file that you can use in different projects. it provides an interface to the android jetpack windowmanager library, enabling application developers to support new device form factors and multi-window environments. before proceeding, read how to use jetpack windowmanager in android game dev and learn the details of how foldablehelper uses windowmanager library to retrieve information about the folded state of the device (flat for normal mode and half-opened for flex mode), window size, and orientation of the fold on the screen. download the foldablehelper.java file here: foldablehelper.java (171.80 kb) to import the foldablehelper.java file and add dependencies to the project, follow the steps below: in assets > plugins > android, right-click and select import new asset. locate and choose the foldablehelper.java file, then click import. next, open the gradletemplate.properties file to any source code editor like visual studio and add the following lines below the **additional_properties** marker. android.useandroidx = true android.enablejetifier = true useandroidx sets the project to use the appropriate androidx libraries instead of support libraries. enablejetifier automatically migrates existing third-party libraries to use androidx by rewriting their binaries. lastly, open the maintemplate.gradle file and add the dependencies for the artifacts needed for the project. **apply_plugins** dependencies { implementation filetree(dir: 'libs', include: ['*.jar']) implementation "androidx.appcompat:appcompat:1.4.1" implementation "androidx.core:core:1.7.0" implementation "androidx.core:core-ktx:1.7.0" implementation "androidx.window:window:1.0.0" implementation "androidx.window:window-java:1.0.0" implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.6.0" **deps**} noteyou may update the version of these dependencies when necessary, but be aware that there might be significant changes. create a new playeractivity to implement flex mode on your applications, you must make necessary changes to the activity. since it is impossible to access and change the original unityplayeractivity, you need to create a new playeractivity that inherits from the original. to do this: create a new file named foldableplayeractivity.java and import it into the android folder, same as when you imported the foldablehelper.java file. to extend the built-in playeractivity from unity, write below code in the foldableplayeractivity.java file. package com.unity3d.player; import android.os.bundle; import com.unity3d.player.unityplayeractivity; import com.samsung.android.gamedev.foldable.foldablehelper; import android.util.log; public class foldableplayeractivity extends unityplayeractivity { @override protected void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); foldablehelper.init(this); } @override protected void onstart() { super.onstart(); foldablehelper.start(this); } @override protected void onstop() { super.onstop(); foldablehelper.stop(); } } oncreate() calls the foldablehelper.init() to ensure that the windowinfotracker and metrics calculator gets created as soon as possible. onstart() calls the foldablehelper.start() since the first windowlayoutinfo doesn't get created until onstart(). onstop() calls the foldablehelper.stop() to ensure that when the application closes, the listener gets cleaned up. after creating the foldableplayeractivity, ensure that the game uses it. open the androidmanifest.xml file and change the activity name to the one you've just created. <activity android:name="foldableplayeractivity" android:theme="@style/unitythemeselector"> … </activity> store foldablelayoutinfo data to flexproxy implement a native listener that receives calls from java when the device state changes by following these steps: use the androidjavaproxy provided by unity in its jni implementation. androidjavaproxy is a class that implements a java interface, so the next thing you need to do is create an interface in the foldablehelper.java file. public interface windowinfolayoutlistener { void onchanged(foldablelayoutinfo layoutinfo); } this interface replaces the temporary native function. therefore, remove the code below: public static native void onlayoutchanged(foldablelayoutinfo resultinfo); then, go to assets > scripts, and right-click to create a new c# script inside called flexproxy.cs. inside this script, create the flexproxy class inheriting from androidjavaproxy. public class flexproxy : androidjavaproxy { } in flexproxy class, define the variables needed to store the data from foldablelayoutinfo and use enumerators for the folded state, hinge orientation, and occlusion type. for the various bounds, use unity's rectint type. also, use a boolean to store whether the data has been updated or not. public enum state { undefined, flat, half_opened }; public enum orientation { undefined, horizontal, vertical }; public enum occlusiontype { undefined, none, full }; public state state = state.undefined; public orientation orientation = orientation.undefined; public occlusiontype occlusiontype = occlusiontype.undefined; public rectint foldbounds; public rectint currentmetrics; public rectint maxmetrics; public bool hasupdated = false; next, define what java class the flexproxy is going to implement by using the interface's fully qualified name as below: public flexproxy() : base("com.samsung.android.gamedev.foldable.foldablehelper$windowinfolayoutlistener") { } com.samsung.android.gamedev.foldable is the package name of the foldablehelper.java file. foldablehelper$windowinfolayoutlistener is the class and interface name separated by a $. after linking the proxy to the java interface, create a helper method to simplify java to native conversions. private rectint converttorectint(androidjavaobject rect) { if(rect != null) { var left = rect.get<int>("left"); var top = rect.get<int>("top"); var width = rect.call<int>("width"); var height = rect.call<int>("height"); return new rectint(xmin: left, ymin: top, width: width, height: height); } else { return new rectint(-1, -1, -1, -1); } } this method takes a java rect object and converts it into a unity c# rectint. now, use this converttorectint() function for the onchanged() function: public void onchanged(androidjavaobject layoutinfo) { foldbounds = converttorectint(layoutinfo.get<androidjavaobject>("bounds")); currentmetrics = converttorectint(layoutinfo.get<androidjavaobject>("currentmetrics")); maxmetrics = converttorectint(layoutinfo.get<androidjavaobject>("maxmetrics")); orientation = (orientation)(layoutinfo.get<int>("hingeorientation") + 1); state = (state)(layoutinfo.get<int>("state") + 1); occlusiontype = (occlusiontype)(layoutinfo.get<int>("occlusiontype") + 1); hasupdated = true; } attach flexproxy to foldablehelper in this step, you need to attach the flexproxy to the java implementation. modify the foldablehelper.java and foldableplayeractivity.java files as follows: in the foldablehelper.java file, create a variable in foldablehelper class where you can store the listener. private static windowinfolayoutlistener listener = null; create a method to receive the native listener. public static void attachnativelistener(windowinfolayoutlistener nativelistener){ listener = nativelistener; } to ensure that the listener is actually used, modify the layoutstatechangecallback to: call the listener.onchanged, the androidjavaproxy version of onchanged function, instead of calling the existing onchanged function in the java file; and check if the listener exists at the time of calling. @override public void accept(windowlayoutinfo windowlayoutinfo) { if (listener != null){ foldablelayoutinfo resultinfo = updatelayout(windowlayoutinfo, activity); listener.onchanged(resultinfo); } } finally, in the foldableplayeractivity.java file, import windowinfolayoutlistener. import com.samsung.android.gamedev.foldable.foldablehelper.windowinfolayoutlistener; then, create a new method in foldableplayeractivity to pass the native listener to foldablehelper. public void attachunitylistener(windowinfolayoutlistener listener){ foldablehelper.attachnativelistener(listener); } implement native flex mode this section focuses on creating the flex mode split-screen effect on the game’s user interface (ui). create a new c# script in the scripts folder called flexmodehelper.cs. after creating the script, define the variables you need for this implementation. public class flexmodehelper : monobehaviour { private flexproxy windowmanagerlistener; [serializefield] private camera maincamera; [serializefield] private camera subcamera; [serializefield] private gameobject flat_ui_panel; [serializefield] private gameobject flex_ui_panel; private recttransform flex_ui_top; private recttransform flex_ui_bottom; screenorientation currentorientation; bool isfold = false; bool isflip = false; float landscapefov = 65; float portraitfov; flexproxy object is the callback object which receives the foldablelayoutinfo from java. maincamera and subcamera are two cameras creating the split-screen effect. however, flex mode does not require the use of two cameras. so, if you only need one viewport, you can remove the subcamera and replace it with any component you want. gameobjects, namely flat_ui_panel and flex_ui_panel, are the parent objects of two ui designs: flat mode (full screen) and flex mode (split screen). recttransforms, namely flex_ui_top and flex_ui_bottom, are the two child objects within flex_ui_panel to serve as the top screen and bottom screen. screenorientation object handles the field of view (fov) changes on the cover screen of the galaxy z fold series (except galaxy fold) and the screen of non-foldable devices. isfold and isflip are booleans to store whether the app runs on a galaxy z fold or z flip device and help create a consistent fov across both devices. landscapefov and portraitfov are two fov values to keep a consistent fov across both orientations of the device. noteidentification of the device where the app is running is not necessary. yet, it is recommended if you want to know how to customize the fov of the game concerning the device model. next, construct a start() method where you: create a flexproxy object and pass it into attachunitylistener on the activity using unity’s jni implementation; turn off the subcamera initially, assuming that the game starts on normal mode; identify which device the app is running using systeminfo.devicemodel. the model number of the galaxy z fold series starts with "sm-f9", while it's "sm-f7" for the galaxy z flip series; calculate the portraitfov from the landscapefov and the camera aspect ratio; set the initial fov depending on the device's orientation using unity's screen.orientation; and retrieve the transforms of the flex ui top and bottom panels. void start() { windowmanagerlistener = new flexproxy(); using (androidjavaclass javaclass = new androidjavaclass("com.unity3d.player.unityplayer")) { using (androidjavaobject activity = javaclass.getstatic<androidjavaobject>("currentactivity")) { activity.call("attachunitylistener", windowmanagerlistener); } } subcamera.enabled = false; string devicemodel = systeminfo.devicemodel; if (devicemodel.contains("sm-f9")) { isfold = true; } else { isflip = devicemodel.contains("sm-f7"); } portraitfov = camera.horizontaltoverticalfieldofview(landscapefov, maincamera.aspect); currentorientation = screen.orientation; if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { maincamera.fieldofview = portraitfov; } flex_ui_top = (recttransform)flex_ui_panel.transform.getchild(0); flex_ui_bottom = (recttransform)flex_ui_panel.transform.getchild(1); } after the game starts, assign a task to flexmodehelper to check if there is an update in the windowmanagerlistener. the windowmanagerlistener receives a call from java when there is a change in the folded state. if a change occurs, update the currentorientation and run the updateflexmode() method. alternatively, if the listener hasn't updated, check to see if the screen has changed orientation. since the cover screen has no folding feature, flexproxy will not update as the callback won't trigger. instead, store the screenorientation and compare if it matches the current screen orientation. otherwise, change the fov because the device has just rotated. once we have figured out if the device has rotated, update the fov of the maincamera based on whether it's in landscape or portrait mode. void update() { if (windowmanagerlistener.hasupdated) { currentorientation = screen.orientation; updateflexmode(); } else { if (screen.orientation != currentorientation) { currentorientation = screen.orientation; if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { maincamera.fieldofview = portraitfov; } } } } create the updateflexmode() method to adjust the game ui according to the folded state of the device. void updateflexmode() { } in this method, check if the folded state is half_opened. if so, enable the subcamera for it to start rendering to the bottom screen and switch the active ui panel to the one created for flex mode (flex_ui_panel). if(windowmanagerlistener.state == flexproxy.state.half_opened) { subcamera.enabled = true; flex_ui_panel.setactive(true); flat_ui_panel.setactive(false); then, check whether the orientation of the fold is horizontal. if (windowmanagerlistener.orientation == flexproxy.orientation.horizontal) { notefor this sample game, splitting the screen isn’t ideal vertically from a user experience (ux) point of view. for this code lab activity, split the screen only on the horizontal fold (top and bottom screen). if you want to split the screen vertically, you need to use the same principle in the next step but for the x-axis instead of the y-axis. so, if the device is on flex mode and horizontal fold, adjust the ui to place the maincamera at the top and subcamera at the bottom of the screen. locate the normalized location of the foldbounds. float foldratiotop = (float)windowmanagerlistener.foldbounds.ymin / windowmanagerlistener.currentmetrics.height; float foldratiobot = (float)windowmanagerlistener.foldbounds.ymax / windowmanagerlistener.currentmetrics.height; use these to set the render areas of the maincamera and subcamera above and below the foldbounds. maincamera.rect = new rect(0, foldratiotop, 1, foldratiotop); subcamera.rect = new rect(0, 0, 1, foldratiobot); next, ensure that the fovs are consistent across the two uis. reset the field of view of each camera, and use the foldratio to ensure that the size of objects in flex mode appears roughly the same as in flat mode. also, use two slightly different fovs for the galaxy fold and flip devices. if (isfold) { maincamera.fieldofview = (landscapefov * foldratiotop); subcamera.fieldofview = (landscapefov * foldratiobot); } else { if(isflip) { maincamera.fieldofview = (landscapefov); subcamera.fieldofview = (landscapefov); } } finally, update the two child objects of the flex_ui to ensure that they line up with the position of the fold. flex_ui_top.anchormin = new vector2(0, foldratiotop); flex_ui_top.anchormax = new vector2(1, 1); flex_ui_bottom.anchormin = new vector2(0, 0); flex_ui_bottom.anchormax = new vector2(1, foldratiobot); however, if the device is not on flex mode and the orientation of the fold is not horizontal, then run the restoreflatmode() method. notify the flexproxy object that its data has been used. } else { restoreflatmode(); } } else { restoreflatmode(); } windowmanagerlistener.hasupdated = false; } create the restoreflatmode() function where you: set both cameras to render to the entire screen; disable the subcamera; and disable the flex ui and enable the flat ui. void restoreflatmode() { maincamera.rect = new rect(0, 0, 1, 1); subcamera.rect = new rect(0, 0, 1, 1); subcamera.enabled = false; flex_ui_panel.setactive(false); flat_ui_panel.setactive(true); also, check to see if the folded state is undefined, which means that the app is running on the cover screen of a galaxy z fold device. if so, treat the screen as a rectangle and use either the landscapefov or the portraitfov depending on the orientation. additionally, since the app might have been initially opened on the main screen of a galaxy fold device, recalculate the portraitfov using the aspect ratio of the cover screen. if the folded state is not undefined, then the app is opened on the main screen of either a galaxy z fold or z flip. in this case, set the fov accordingly by using the isfold boolean to check. if the boolean returns true, treat the screen as a square. otherwise, treat it as a rectangle and set the fov based on the orientation. if (windowmanagerlistener.state == flexproxy.state.undefined) { if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { if(isfold) portraitfov = camera.horizontaltoverticalfieldofview(landscapefov, maincamera.aspect); maincamera.fieldofview = portraitfov; } } else { if (isfold) { maincamera.fieldofview = landscapefov; } else { if (currentorientation == screenorientation.landscape || currentorientation == screenorientation.landscapeleft || currentorientation == screenorientation.landscaperight) { maincamera.fieldofview = landscapefov; } else { maincamera.fieldofview = portraitfov; } } } } set up flex scene return to the unity editor and create an empty gameobject in the scene. right-click on the sample scene > gameobject > create empty and name it flexmanager. select the flexmanager object, then drag and drop the flexmodehelper script into the inspector pane. then, select the cameras and ui panels like below: build and run the app go to file > build settings and ensure that the scenes/samplescene is selected in scenes in build. click build to build the apk. after building the apk, run the game app on a foldable galaxy device and see how the ui switches from normal to flex mode. if you don’t have any physical device, you can also test it on a remote test lab device. tipwatch this tutorial video and know how to easily test your app via remote test lab. you're done! congratulations! you have successfully achieved the goal of this code lab. now, you can implement flex mode in your unity game app by yourself! if you're having trouble, you may download this file: flex mode on unity complete code (631.05 mb) to learn more, visit: www.developer.samsung.com/galaxy-z www.developer.samsung.com/galaxy-gamedev

      https://developer.samsung.com/codelab/gamedev/flex-mode-unity.html
      No Search Results
      No Search results. Try using another keyword.
      • <<
      • <
      • 1
      • >
      • >>
      Samsung Developers
      Samsung Developers
      Quick Link
      • Android USB Driver
      • Code Lab
      • Galaxy Emulator Skin
      • Foldables and Large Screens
      • One UI Beta
      • Remote Test Lab
      • Samsung Developers Podcast
      Family Site
      • Bixby
      • Knox
      • Samsung Pay
      • SmartThings
      • Tizen
      • Samsung Research
      • Samsung Open Source
      • Samsung Dev Spain
      • Samsung Dev Brazil
      Legal
      • Terms
      • Privacy
      • Open Source License
      • Cookie Policy
      Social Communications
      • Facebook
      • Instagram
      • Twitter
      • YouTube
      • Buzzsprout
      • Rss
      • Linkedin
      • System Status
      • Site Map
      • System Status
      • Site Map
      • facebook
      • instagram
      • twitter
      • youtube
      • buzzsprout
      • rss
      • linkedin

      Copyright © 2023 SAMSUNG. All rights reserved.