Tutorials Game, Mobile

How to Use Jetpack WindowManager in Android Game Dev

Lochlann Henry Ramsay-Edwards

Galaxy GameDev Engineer

With the increasing popularity of foldable phones such as the Galaxy Z Fold3 and Galaxy Z Flip3, apps on these devices are adopting its foldable features. In this blog, you can get started on how to utilize these foldable features on Android game apps.

We focus on creating a Java file containing an implementation of the Android Jetpack WindowManager library that can be imported into game engines like Unity or Unreal Engine. This creates an interface allowing developers to retrieve information about the folding feature on the device.

At the end of this blog, you can go deeper in learning by going to Code Lab.

Android Jetpack WindowManager

Android Jetpack, in their own words, is "a suite of libraries to help developers follow best practices, reduce boilerplate code, and write code that works consistently across Android versions and devices so that developers can focus on the code they care about."

WindowManager is one of these libraries, and is intended to help application developers support new device form factors and multi-window environments. The library had its 1.0.0 release in January 2022 for targeted foldable devices. According to its documentation, future versions will be extended to more display types and window features.

Creating the Android Jetpack WindowManager setup

As previously mentioned, we are creating a Java file that can be imported into either Unity or Unreal Engine 4, to create an interface for retrieving information on the folding feature and pass it over to the native or engine side of your applications.

Set up the FoldableHelper class and data storage class

Create a file called FoldableHelper.java in Visual Studio or any source code editor. Let's start off by giving it a package name of package com.samsung.android.gamedev.foldable;

Next, let's import all the necessary libraries and classes in this file:

//Android Imports
import android.app.Activity;
import android.graphics.Rect;
import android.os.Handler;
import android.os.Looper;
import android.util.Log;

//Android Jetpack WindowManager Imports
import androidx.annotation.NonNull;
import androidx.core.util.Consumer;
import androidx.window.java.layout.WindowInfoTrackerCallbackAdapter;
import androidx.window.layout.DisplayFeature;
import androidx.window.layout.FoldingFeature;
import androidx.window.layout.WindowInfoTracker;
import androidx.window.layout.WindowLayoutInfo;
import androidx.window.layout.WindowMetrics;
import androidx.window.layout.WindowMetricsCalculator; 

//Java Imports
import java.util.List;
import java.util.concurrent.Executor;

Start by creating a class, FoldableHelper, that is going to contain all of our helper functions. Let's then create variables to store a callback object as well as WindowInfoTrackerCallbackAdapter and WindowMetricsCalculator. Let's also create a temporary declaration of the native function to pass the data from Java to the native side of application once we start working in the game engines.

public class FoldableHelper {
    private static LayoutStateChangeCallback layoutStateChangeCallback;
    private static WindowInfoTrackerCallbackAdapter wit;
    private static WindowMetricsCalculator wmc;

    public static native void OnLayoutChanged(FoldableLayoutInfo resultInfo);
}

Let's create a storage class to hold the data received from the WindowManager library. An instance of this class will also be passed to the native code to transfer the data.

public static class FoldableLayoutInfo {
    public static int UNDEFINED = -1;

    // Hinge Orientation
    public static int HINGE_ORIENTATION_HORIZONTAL = 0;
    public static int HINGE_ORIENTATION_VERTICAL = 1;

    // State
    public static int STATE_FLAT = 0;
    public static int STATE_HALF_OPENED = 1;

    // Occlusion Type
    public static int OCCLUSION_TYPE_NONE = 0;
    public static int OCCLUSION_TYPE_FULL = 1;

    Rect currentMetrics = new Rect();
    Rect maxMetrics = new Rect();

    int hingeOrientation = UNDEFINED;
    int state = UNDEFINED;
    int occlusionType = UNDEFINED;
    boolean isSeparating = false;
    Rect bounds = new Rect();
}

Initialize the WindowInfoTracker

Since we are working in Java and the WindowManager Library is written in Kotlin, we have to use the WindowInfoTrackerCallbackAdapter. This is an interface provided by Android to enable the use of the WindowInfoTracker from Java. The Window Info Tracker is how we receive information about any foldable features inside the window's bounds.

Next is to create WindowMetricsCalculator, which lets us retrieve the Window Metrics of an Activity. Window Metrics consists of the windows' current and maximum bounds.

We also create a new LayoutStateChangeCallback object. This object is passed into the Window Info Tracker as a listener object and is called every time the layout of the device changes (for our purposes this is when the foldable state changes).

public static void init(Activity activity) {
    //Create Window Info Tracker
    wit = new WindowInfoTrackerCallbackAdapter(WindowInfoTracker.Companion.getOrCreate(activity));
    //Create Window Metrics Calculator
    wmc = WindowMetricsCalculator.Companion.getOrCreate();
    //Create Callback Object
    layoutStateChangeCallback = new LayoutStateChangeCallback(activity);
}

Set up and attach the callback listener

In this step, let's attach the layoutStateChangeCallback to the WindowInfoTrackerCallbackAdapter as a listener.

The addWindowLayoutInfoListener function takes three parameters: the Activity to attach the listener to, an Executor, and a Consumer of WindowLayoutInfo. We will set up the Executor and Consumer in a moment.

The adding of the listener is kept separate from the initialization, since the first WindowLayoutInfo is not emitted until Activity.onStart has been called. As such, we'll likely not be needing to attach the listener until during or after onStart, but we can still set up the WindowInfoTracker and WindowMetricsCalculator ahead of time.

public static void start(Activity activity) {
    wit.addWindowLayoutInfoListener(activity, runOnUiThreadExecutor(), layoutStateChangeCallback);
}

Now, let's create the Executor for the listener. This Executor is straightforward and simply runs the command on the MainLooper of our Activity. It is possible to set this up to run on a custom thread, however this is not going to be covered in this blog. For more information, we recommend checking the official documentation for the Jetpack WindowManager.

static Executor runOnUiThreadExecutor() {
   return new MyExecutor();
}

static class MyExecutor implements Executor {
    Handler handler = new Handler(Looper.getMainLooper());

    @Override
    public void execute(Runnable command) {
        handler.post(command);
    }
}

We're going to create the basic layout of our LayoutStateChangeCallback. This consumes WindowLayoutInfo and implements Consumer<WindowLayoutInfo>. For now, let's simply lay out the class and give it some functionality a little bit later.

static class LayoutStateChangeCallback implements Consumer<WindowLayoutInfo> {
    private final Activity activity;

    public LayoutStateChangeCallback(Activity activity) {
        this.activity = activity;
    }
}

If the use of the listener is no longer needed, we want a way to remove it and the WindowInfoTrackerCallbackAdapter contains a function to do just that.

public static void stop() {
    wit.removeWindowLayoutInfoListener(layoutStateChangeCallback);
}

This just tidies things up for us and ensures that the listener is cleaned up when we no longer need it.

Next, we're going to add some functionality to the LayoutStateChangeCallback class. We are going to process WindowLayoutInfo into FoldableLayoutInfo we created previously. Using Java Native Interface (JNI), we are going to send that information over to the native side using the function OnLayoutChanged.

Note: This doesn't actually do anything yet, but we cover how to set this up in Unreal Engine and in Unity through Code Lab tutorials.

static class LayoutStateChangeCallback implements Consumer<WindowLayoutInfo> {
    @Override
    public void accept(WindowLayoutInfo windowLayoutInfo) {
            
        FoldableLayoutInfo resultInfo = updateLayout(windowLayoutInfo, activity);

        OnLayoutChanged(resultInfo);
    }
}

Let's implement the updateLayout function to process WindowLayoutInfo and return a FoldableLayoutInfo. Firstly, create a FoldableLayoutInfo that contains the processed information. Follow this up by getting the Window Metrics, both maximum metrics and current metrics.

private static FoldableLayoutInfo updateLayout(WindowLayoutInfo windowLayoutInfo, Activity activity)
{
    FoldableLayoutInfo retLayoutInfo = new FoldableLayoutInfo();

    WindowMetrics wm = wmc.computeCurrentWindowMetrics(activity);

    retLayoutInfo.currentMetrics = wm.getBounds();

    wm = wmc.computeMaximumWindowMetrics(activity);

    retLayoutInfo.maxMetrics = wm.getBounds();
}

Get the displayFeatures present in the current window bounds using windowLayoutInfo.getDisplayFeatures. Currently, the API only has one type of displayFeature: FoldingFeatures, however in the future there will likely be more as screen types evolve.

At this point, let's use a for loop to iterate through the resulting list until it finds a FoldingFeature. Once it detects a folding feature, it starts processing its data: Orientation, State, Seperation type, and its bounds. Then, store these data in FoldableLayoutInfo we've created at the start of the function call. You can learn more about these data by going to the Jetpack WindowManager documentation.

private static FoldableLayoutInfo updateLayout(WindowLayoutInfo windowLayoutInfo, Activity activity)
{
    FoldableLayoutInfo retLayoutInfo = new FoldableLayoutInfo();

    WindowMetrics wm = wmc.computeCurrentWindowMetrics(activity);
    retLayoutInfo.currentMetrics = wm.getBounds();

    wm = wmc.computeMaximumWindowMetrics(activity);
    retLayoutInfo.maxMetrics = wm.getBounds();

    List<DisplayFeature> displayFeatures = windowLayoutInfo.getDisplayFeatures();
    if (!displayFeatures.isEmpty())
    {
        for (DisplayFeature displayFeature : displayFeatures)
        {
            FoldingFeature foldingFeature = (FoldingFeature) displayFeature;
            if (foldingFeature != null)
            {  
                if (foldingFeature.getOrientation() == FoldingFeature.Orientation.HORIZONTAL)
                {
                   retLayoutInfo.hingeOrientation = FoldableLayoutInfo.HINGE_ORIENTATION_HORIZONTAL;
                }
                else
                {
                   retLayoutInfo.hingeOrientation = FoldableLayoutInfo.HINGE_ORIENTATION_VERTICAL;
                }

                if (foldingFeature.getState() == FoldingFeature.State.FLAT)
                {
                   retLayoutInfo.state = FoldableLayoutInfo.STATE_FLAT;
                }
                else
                {
                   retLayoutInfo.state = FoldableLayoutInfo.STATE_HALF_OPENED;
                }

                if (foldingFeature.getOcclusionType() == FoldingFeature.OcclusionType.NONE)
                {
                   retLayoutInfo.occlusionType = FoldableLayoutInfo.OCCLUSION_TYPE_NONE;
                }
                else
                {
                   retLayoutInfo.occlusionType = FoldableLayoutInfo.OCCLUSION_TYPE_FULL;
                }

                retLayoutInfo.isSeparating = foldingFeature.isSeparating();

                retLayoutInfo.bounds = foldingFeature.getBounds();

                return retLayoutInfo;
            }
        }
   }
  
   return retLayoutInfo;
}

If there's no folding feature detected, it simply returns the FoldableLayoutInfo without setting its data leaving it with UNDEFINED (-1) values.

Conclusion

The Java file you have now created should be usable in new or existing Unity and Unreal Engine projects, to provide access to the information on the folding feature. Continue learning about it by going to the Code Lab tutorials showing how to use the file created here, to implement flex mode detection and usage in game applications.

Additional resources on the Samsung Developers site

The Samsung Developers site has many resources for developers looking to build for and integrate with Samsung devices and services. Stay in touch with the latest news by creating a free account and subscribing to our monthly newsletter. Visit the Marketing Resources page for information on promoting and distributing your apps. Finally, our Developer Forum is an excellent way to stay up-to-date on all things related to the Galaxy ecosystem.

Recent Posts