What’s new in Android: Embracing era of Generative AI
0 likes63 views
The document discusses new features in Android related to the Gemini AI SDK, which supports on-device AI and Kotlin Multiplatform development. Key highlights include various functionalities such as code completion, user interface code generation, and function calling integration within apps. Additionally, it provides resources and links for setting up and utilizing Gemini in Android Studio, along with a demo application and tutorials for developers.
![- Multiplatform resources [link]
- Lifecycle [link]
- ViewModel [link]
- Navigation and Routing [link]
Official UI Framework
Libraries
Jetbrains
Jetbrains
Multiplatform](https://image.slidesharecdn.com/iox2024-whatsnewinandroid-241125042839-7d24efef/85/What-s-new-in-Android-Embracing-era-of-Generative-AI-67-320.jpg)
![- Paging [link]
- sqlite [link]
- room [link]
- datastore [link]
- Samples
https://github.com/android/kotlin-
multiplatform-samples
Official Multiplaform
Libraries (non-UI)
Android
Google
Multiplatform](https://image.slidesharecdn.com/iox2024-whatsnewinandroid-241125042839-7d24efef/85/What-s-new-in-Android-Embracing-era-of-Generative-AI-68-320.jpg)
What’s new in Android: Embracing era of Generative AI
- 1. Hassan Abid | GDE What’s new in Android: Embracing era of Generative AI
- 2. About myself GDE - Google Developers Expert for Android Since 2016 - CTO at BELIVE TECHNOLOGY - Mentor for AI First Singapore startup program - Mentor for Solution Challenge 2023 & 2024 - Speaker at GDG Events across Asia Pacific (GDG Hanoi 2019)
- 3. Agenda Agenda Gemini in Android Studio 01 Gemini SDK For Android 02 On Device AI 03 Kotlin Multiplatform 04
- 4. Link : https://developer.android.com/studio/preview/gemini Gemini in Android Studio Android Studio
- 5. AI Code Completion Error Explanation and Fixes Ask Development Questions Key Features Android Studio 01 02 03
- 6. Required for Gemini to use context from Project Authorize Gemini in Android Studio Android Studio
- 8. https://developer.android.com/studio/preview/gemini/aiexclude Configure context sharing with .aiexclude file Android Studio
- 9. Control privacy You can also control the data that Gemini can access through a custom .aiexclude file. Android Studio Android Studio
- 11. - This feature will be available in Android Studio later this year - Upload design image, and ask Gemini to generate compose UI code - For time being you can use Google AI Studio https://aistudio.google.com/ Use Google AI Studio to get Compose code (for UI) Google AI Studio
- 12. Prompt : Act as an Android app developer. For the image provided, use Jetpack Compose to build the screen so that the Compose Preview is as close to this image as possible. Also make sure to include imports and use Material3." Code here : https://gist.github.com/hassanabidpk/80041460aa 20884e6db18411f601a95a Google AI Studio : link here Use Google AI Studio to get Compose code Example 1
- 13. Prompt : Act as an Android app developer. For the image provided, use Jetpack Compose to build the screen so that the Compose Preview is as close to this image as possible. Also make sure to include imports and use Material3." Use Google AI Studio to get Compose code Example 2 Figma Design
- 14. Code here (CoffeeDetailScreen1.kt): https://gist.github.com/hassanabidpk/92cf3acf5b9 8d1fdb83e699db115663d Google AI Studio : link here UI Correction needed Generated Code (first iteration) Example 2 Iteration 1 result
- 15. Prompt : Given this code describing a UI screen and the image of its Compose Preview, please fix the part within the red outline so that the items are at bottom of screen. Buy now button should be on right side of Price Correction of UI Example 2 Highlighted part
- 16. Code here (CoffeeDetailScreen2.kt): https://gist.github.com/hassanabidpk/92cf3acf5b98d1 fdb83e699db115663d Google AI Studio : link here Generated Code (final iteration) Example 2 Final result
- 17. Integrating Gemini prompts in your app Visit Prompt Gallery, Try prompt in AI Studio and then get code for kotlin Google AI Studio
- 18. Reference Google IO 24 : https://android-developers.googleblog.com/2024/05/google-i o-2024-whats-new-in-android-development-tools.html Android Studio
- 20. Connect to devices anytime Android Device Streaming, powered by Firebase, lets you securely connect to remote physical Android devices hosted in Google's data centers. Device testing
- 21. Gemini SDK for Android Gemini API
- 22. Google AI for Android
- 24. Gemini API and Templates Gemini API
- 25. New Project Template You File > New Project. Select the new Gemini API starter template. Gemini API
- 27. Generate API Key Navigate to Google AI Studio https://aistudio.googl e.com/app/apikey Gemini API
- 29. How do I make production-grade applications on Android?
- 32. Link : https://firebase.google.com/docs/vertex-ai/get-started?platform=android Get Started with Gemini API Using Firebase Vertex AI SDK Firebase Vertex API
- 33. Github Repo
- 34. Note : Firebase supports upto 20MB payload Gemini Models use cases and properties Firebase Vertex API
- 37. Firebase : https://firebase.google.com/docs/vertex-ai/function-calling?platform=android Gen AI SDK : Function Calling Function Calling
- 38. - Function calling makes it easier for you to get structured data outputs from generative models - Function calling empowers users to interact with real-time information and various services, such as databases, customer relationship management systems etc. Why Function Calling? Function Calling
- 39. Android App getPastOrders() User Prompt 1 : What did I order recently? Generative Model Past orders & Preferences - 1 Monday : Latte - 2 Tuesday : Black Coffee - …. User Prompt 1 : I am feeling sleepy, order food to my current location Order Summary - 1 Vietnamese Coffee with Banh mi PlaceOrder() Function Calling
- 40. val placeOrderFunction = defineFunction( "placeOrder", "Place order of coffee or Tea", Schema.str("orderSummary", "Type of coffee or tea to order. Also add any additional food item"), Schema.str("address", "address to deliver food items") ) { order, address -> makePlaceOrderApiRequest(order, address) } Make API Request
- 41. val generativeModel = GenerativeModel( modelName = "gemini-1.5-flash", apiKey = BuildConfig.apiKey, generationConfig = config, systemInstruction = content { text(systemInstructions)}, tools = listOf(Tool(listOf(placeOrderFunction, pastOrdersFunction))) ) Add multiple functions
- 42. // Check if the model responded with a function call response.functionCalls.firstOrNull()?.let { functionCall -> // Try to retrieve the stored lambda from the model's tools and val matchedFunction = generativeModel.tools?.flatMap { it.functionDeclarations } ?.first { it.name == functionCall.name } ?: throw InvalidStateException("Function not found: ${functionCall.name}") // Call the lambda retrieved above val apiResponse: JSONObject = matchedFunction.execute(functionCall) // Send the API response back to the generative model response = chat.sendMessage( content(role = "function") { part(FunctionResponsePart(functionCall.name, apiResponse)) } ) } Send response back to on-going chat
- 43. On-Device AI On Device AI 1. The MediaPipe LLM Inference API (Public beta) 2. Google AI Edge SDK (Private preview)
- 44. 1. The LLM Inference API lets you run large language models (LLMs) completely on-device 2. Features : a. Text-to-text generation b. LLM selection c. LoRA Support Run Gemma 2B on Android & iOS On Device AI Wrote a poem about IOX Hanoi
- 45. 1. Use conversion scripts to format the model to be compatible with MediaPipe. a. E.g : Phi-2, Falcon, or StableLM b. Non-Kaggle models 2. The model conversion process requires the MediaPipe PyPI package Convert Models to Mediapipe Format On Device AI
- 46. Google AI Models
- 48. COST (Google AI Studio) - Pay-as-you go for project that has billing enabled - Free version (1500 requests per day)
- 50. COST (Firebase) Firebase : Pay as you go pricing only
- 54. “At I/O 24 we are excited to announce we are supporting Kotlin Multiplatform on Android, which enables sharing code across mobile, web, server, and desktop platforms” Android Dev Rel
- 56. 1. Use Kotlin Multiplatform for sharing Business Logic across mobile and Web 2. Use Flutter for sharing UI and business Logic across platform Recommendation from Google Multiplatform
- 58. Use Compose multi-platform to share UIs across Android and iOS Apps Cross platform Apps Multiplatform
- 59. Install Kotlin Multiplatform Plugin in Android Studio
- 61. - Select new project in Android Studio → Kotlin Multiplatform App Alternatively use https://kmp.jetbrains.com/ Step-1 Create Kotlin Multi-platform project Multiplatform
- 62. - Select Android and iOS Shared UI implementation using Compose Multiplatform - You can download template from Template gallery too (Recommended) Create Kotlin Multi-platform project online Multiplatform https://kmp.jetbrains.com/
- 63. - Project includes two modules - composeApp - iOSApp - The commonMain source set in composeApp uses the common Kotlin code, and platform source sets use Kotlin code specific to each target Step-2 Open project in Android Studio Multiplatform
- 64. In the list of run configurations, select composeApp. Choose your Android virtual / remote or real device and then click Run. Step-3a Run on Android Multiplatform
- 65. Step-3b Run on iOS Simulator Kotlin Multiplatform - Edit Configuration after selecting iosApp. Then Run project
- 66. No official Kotlin Multiplatform SDK available yet. However, there is an open source library for KMP https://github.com/PatilShreyas /generative-ai-kmp Let’s integrate Gemini Multiplatform
- 67. - Multiplatform resources [link] - Lifecycle [link] - ViewModel [link] - Navigation and Routing [link] Official UI Framework Libraries Jetbrains Jetbrains Multiplatform
- 68. - Paging [link] - sqlite [link] - room [link] - datastore [link] - Samples https://github.com/android/kotlin- multiplatform-samples Official Multiplaform Libraries (non-UI) Android Google Multiplatform
- 70. Summary Gemini in Android Studio 01 Gemini SDK For Android 02 On Device AI 03 Kotlin Multiplatform 04
- 71. A. Dependency on cloud-based services B. Increased latency due to network communication C. Large Model Size D. Local processing of sensitive data What is an advantage of executing a model on-device? Quiz
- 72. Thank You Hassan ABID GDE Android LinkedIn
- 73. Demo code : https://github.com/hassanabidpk/io-extended-2024 Demo Code (Multiplatform) : https://github.com/hassanabidpk/iox-multiplatform-2024 Generative AI Sample (Kotlin) : https://github.com/google-gemini/generative-ai-android/tree/main/generativeai-android-sample Generative AI Sample (VertexAI - Firebase Kotlin) https://github.com/firebase/quickstart-android/tree/master/vertexai Android 15 features : https://developer.android.com/about/versions/15/features Gemini Template : https://developer.android.com/studio/preview/gemini-template Gemini API Tutorial for Android : https://ai.google.dev/gemini-api/docs/get-started/tutorial Course + Quiz : https://developers.google.com/learn/pathways/solution-ai-gemini-getting-started-android References