AGDK tutorial step by step

Editor's Notes

• #2: Hello, everyone. Nice to meet you all and thank you for being here. And I am so happy for presenting this topic because I love this topic so much. My name is Jungsoo Nam. I’m based in South Korea and a former game engine engineer and now an android engineer. Today, the topic of my presentation is AGDK tutorial step by step.
• #3: Here you can see the test environment and reference materials that used in my presentation. There are many useful source codes and documentations in the official Google Android developer website and Github repositories. Basically, AGDK is also developed using NDK, so it is important to familiarize yourself with previous NDK materials. I think you are all familiar with using NDK.
• #4: I watched the AGDK release on July 13th, 2021, in real-time on YouTube. It was a library I was personally very excited about, and I look forward to applying it to my work. Well, What is AGDK? Android Game Development Kit (AGDK), a full range of tools and libraries to help you develop, optimize, and deliver high quality Android games. AGDK features follow three key tenets. C/C++ APIs: Game developers use C or C++ for high performance. Reduce fragmentation: Android engineers are always solving fragmentation problems. Built by Android, for Android: Also backward compatibility is important. Now in the metaverse era, not only games but also AR/VR so called XR app uses graphical features, so the app needs to cowork between general UI and graphics screen. For instance, an app using Unity as a library. It is mixed by normal Activities and Unity Activities.
• #5: Unlike Native Activity, Game Activity is compatible with fragments and extendable, making it easier to integrate some of your favorite SDKs. Because GameActivity inherits from AppCompatActivity. AppCompatActivity inherits from FragmentActivity which provides Fragments. FragmentActivity inherits from ComponentActivity which provides Lifecycle management. GameTextInput, In my experience in the past, it relied on java calls to show/hide the soft keyboard, and it was very inconvenient to manage Korean composition in java. Connecting gamer’s game controller is very important. For example, I’m currently using the Xbox Controller through Bluetooth on my gaming laptop, and it often disconnects itself and automatic reconnection is not supported, this is so inconvenient.
• #6: Let’s compare GameActivity (left) with NativeActivity (right) NativeActivity inherits from Activity, and implements SurfaceHolder.Callback2, and InputQueue.Callback. On the other hand, GameActivity inherits from AppCompatActivity, and implements SurfaceHolder.Callback2, gametextinput.Listener. The major differences are inherited from Activity vs AppCompatActivity, and GameActivity supports GameTextInput. NativeActivity has its own InputMethodManager instance in java side.
• #7: Another one, let's compare Unity3D 4.6 and Unity 2020.3 the latest LTS version. Unity 4.6 used to use UnityPlayerNativeActivity, on the other hand, Unity 2020.3 uses UnityPlayerActivity. The latest version no more uses NativeActivity and added implementing lifecycle callback interface. In the long run, in my opinion, Unity will adopt using GameActivity for improving their performance.
• #8: I will cover GameActivity, GameTextInput, GameTouchInput, Oboe, Game frame pacing, Android performance tuner, and OpenGL ES API in following slides. I won’t cover the excludes, but that doesn't mean those are not important. - Excluded things are probably more important for supporting our customers the game companies. - Vulkan's adoption is getting broader. - GPU profiling is important traditionally. - Still most game developers are using Visual Studio. - Almost no mobile games are possible without Unity and Unreal Engine.
• #9: Let me check about GameActivity offers again. Allowing to use Android Jetpack Architecture Components is most important. Other offers are similar to NativeActivity.
• #10: Ok, so launch our Android Studio and Goto menu choose ‘File > New > New Project’. Choose ‘Native C++’ project type in ‘Phone and Tablet tab’. Input project name as ‘GameActivityTutorial’.
• #11: This is the initial project screen. Since we selected ‘Native C++’, you can see the cpp folder on the left side, CMakeLists.txt was added, and native-related settings were set in gradle.
• #12: Now let's set up the GameActivity. 1. Add implementation games-activity:1.1.0-rc01 2. Change AppCompatActivity to GameActivity.
• #13: 3. Add game-activity to find_package in CMakeLists.txt 4. Add game-activity::game-activity to target_link_libraries 5. Set android.prefabVersion as 1.1.2 in gradle.properties 6. Set buildFeatures.prefab = true in build.gradle (module level) Prefab includes the headers and libraries of the native dependency.
• #14: More, add android.app.lib_name=”gameactivitytutorial” as AndroidManifest’s MainActivity meta-data This was the same way in the NDK. In GameActivity.onCreate, load library by given library name. Library loading is done dynamically by dlopen and dlsym in loadNativeCode_native.
• #15: Also, we need to include some source files as headers. Including files must be c or cpp source file and declared in CMakeList.txt for compile. We declare these later.
• #16: Next, we implement the android_main function in android_main.cpp If extension is cpp, we have to use extern “C” for C compatibility. (For the detail, as you know, cpp does mangle names, we prevent it.)
• #17: Open CMakeLists.txt again, we add our own c and cpp source files in here add_library. Plus, android, EGL, and GLESv3 libraries to target_link_libraries section. Android is android specific library, EGL is for EGL, GLESv3 for OpenGL ES library.
• #18: Now, let's move on to implementing NativeEngine. 1. Add NativeEngine.hpp/cpp files. 2. Add Constructor and Destructor, GameLoop, IsAnimating, and DoFrame. 3. Add android_app app pointer as a private member variable. 4. In GameLoop function, set this pointer to mApp’s userData, and loop a while block. When calling ALooper_pollAll, -1 is infinite. If not animating, we will block forever waiting for events. If animating, we loop until all events are read, then continue to draw the next frame of animation.
• #19: At this moment, let's find out how android_main function can be called. I know you've already known about this. Anyway, for reminding. In GameActivity.java, onCreate method calls loadNativeCode method, In JNI side, Java_com_google_androidgamessdk_GameActivity_loadNativeCode matches GameActivity.loadNativeCode. libname “main” is replaced with “gameactivitytutorial”, but funcname “GameActivity_onCreate” does not change.
• #20: In loadNativeCode_native function, we bind GameActivity_onCreate function and create a NativeCode instance for storing native states. First parameter is library handle, second one is GameActivity_onCreate function’s pointer.
• #21: GameActivity_onCreate is declared in android_native_app_glue.c But we included android_native_app_glue.c into gameactivitytutorial library. So we load GameActivity_onCreate dynamically through dlopen and dlsym. (I mentioned this in the previous page) The GameActivity pointer activity is a NativeCode instance.
• #22: Calling android_app_create by GameActivity_onCreate. (in the previous page) Then it creates a new thread with android_app_entry as a thread function. android_app_entry then runs android_app_main function. The reason why separating threads is for avoiding blocking the main thread.
• #23: So, let’s get first frame of our game. Register _handle_cmd_proxy callback function to mApp->onAppCmd In _handle_cmd_proxy, type cast app->userData to NativeEngine pointer, then call HandleCommand NativeEngine’s HandleCommand handles enum NativeAppGlueAppCmd declared in android_native_app_glue.h
• #24: 2nd, we add PrepareToRender function. It is called by DoFrame. PrepareToRender checks whether mEglDisplay, mEglSurface, mEglContext is set or not.
• #25: 3rd, we add InitDisplay, InitSurface, InitContext These are ordinary EGL initialization. In ConfigureOpenGL, we just set clear color and enable depth test and clear back buffer.
• #26: Finally, Remove setContentView calling in Java (kotlin) side. When we set content view, app state does not transit to APP_CMD_INIT_WINDOW, then we get always Init surface failed.
• #27: We need to handle touch now for the next step. Let’s add HandleGameActivityInput to NativeEngine and call it from GameLoop. Store surface’s width and height for determining touch area. When motionEvents exist, call _cook_game_activity_motion_event with motionEvent, surfWidth, suftHeight.
• #28: _cook_game_activity_motion_event function converts raw event to cooked event and passes the cooked event to the callback function. Cooked events are more simple. Logcat in the bottom shows callbacked touch events.
• #29: Another AGDK component, the GameTextInput. Let’s try to add it. 1. make NativeEngine as a Singleton for C callback. 2. add GameTextInputState as a member variable. 3. check mApp->textInputState in GameLoop.
• #30: 4. add OnTextInput. In OnTextInput, copy GameTextInputState callbacked by getTextInputState to NativeEngine local. 5. for testing, add UpdateInputMode function for toggling show/hide soft input keyboard. Now that I think about it, the name ToggleInputMode seems better than UpdateInputMode. Then we can now see the input string and the characters composing in Logcat.
• #31: Next, here is a high-performance audio library for game development, Oboe. Oboe Achieve the lowest latency, Use the best available native library, Avoid audio bugs. When I developed a mobile game, I used AudioTrack through JNI. But it’s slow and has some bugs depending on the manufacturer, so I made an audio pool myself and used it.
• #32: Let’s make a beep sound by using Oboe. I got the OboeSinePlayer source code from Google’s oboe repository. Make instance as a member of NativeEngine and play it when getting APP_CMD_INIT_WINDOW. That’s the end of playing through oboe.
• #33: There is another good library called SwappyGL. It controls the pace of the game’s frames.
• #34: Figure 2,3 are how short game frames work. It needs EGL_ANDROID_presentation_time extension to be smoother. By using it, Present B is guaranteed to have 2 frames. Figure 4,5 are how long frames work. It needs EGL_KHR_fence_sync extension to make frame C and D wait. By using it, Present B is also guaranteed to have 2 frames.
• #35: We can check EGL extensions from the Khronos group registry website. eglPresentationTimeANDROID delays frame time. eglClientWaitSyncKHR waits until timeout.
• #36: Left side is SwappyGL source code in gamesdk repository, Right side is how to setup games-frame-pacing library. Look at the swapInternal function, onPreSwap, onPostSwap functions are calculating swap timing automatically using fence and eglPresentationTimeANDROID.
• #37: Integrating SwappyGL is so simple. In the constructor, initialize the SwappyGL, and set swap interval. Simple however we need to implement GetJniEnv for passing JNIEnv variable to SwappyGL.
• #38: In destructor, we will call SwappyGL_destroy. And when APP_CMD_INIT_WINDOW command received, call SwappyGL_setWindow with mApp->window. Finally we replace eglSwapBuffers to SwappyGL_swap. That’s all.
• #39: The final one, Android performance tuner. Android performance tuner works with android vital in Google play. Each user upload the data with game annotation and fidelity parameters like Firebase Analytics.
• #40: The android performance tuner’s name is TuningFork. Let’s setup samples into GameActivityTutorial app. We decided to copy protobuf files from agdktunnel sample app. 1. make assets and proto folder. 2. copy proto files, fidelity params text files, and settings text file. 3. add custom task to build.gradle for build proto files.
• #41: 4. add implementation games-performance-tuner library in build.gradle. 5. and CMakeLists.txt build setup for proto gen files and performance-tuner library. 6. set permissions for application and enable usesCleartextTrafic for http.
• #42: Next let’s make TuningManager.hpp/cpp. Just copied them for testing from agdktunnel.
• #43: StartLoading is called by TuningManager constructor, EndLoading is called manually at the first frame rendered.
• #44: When using SwappyGL we can avoid using Choreographer directly, However, unfortunately, the TuningManager that used in agdktunnel needs to be used Choreographer directly. So minSdk requires 24. We have to set it in build.gradle. We can also have another chance to avoid using Choreographer by SwappyTracer. See the right side. But it is not applied yet in agdktunnel.
• #45: I checked swappy tracer’s usage from gamesdk sources. We call TuningFork_init with settings and store swappy tracer’s pointer to s_swappy_tracer in tf::Init. As a result, swappy tracer is stored, but nowhere to use it.
• #46: Finally configure the TuningManager and let's check if it's applied properly. Add TuningManager as a member variable to NativeEngine, create instance in NativeEngine constructor, and delete it in NativeEngine destructor. In DoFrame check first frame, and call FinishLoading of TuningManager. We can see the log to upload but got some issues with uploading. It needs time to find out the reason more.
• #47: This is end of my presentation. Thank you for your attention. If you have any questions, feel free to ask me.