![TensorFlow Inference
val inferenceInterface = TensorFlowInferenceInterface(getAssets(), MODEL_FILE)
val labels = loadLabels() // [ "reptilian", "reptile", "human" ]
fun doRecognize(image: Bitmap) {
val pixels = image.toPixelsArray()
inferenceInterface.feed(INPUT_NODE, pixels, NETWORK_STRUCTURE)
inferenceInterface.run(arrayOf(OUTPUT_NODE))
val outputs = FloatArray(labels.size)
inferenceInterface.fetch(OUTPUT_NODE, outputs)
}](https://image.slidesharecdn.com/android-things-reptilians-170714180248/85/Using-Android-Things-to-Detect-Exterminate-Reptilians-41-320.jpg)
![TensorFlow Inference
labels = [ "reptilian", "reptile", "human" ]
outputs = [ 0.0098f , 0.0039f , 0.9863f ]](https://image.slidesharecdn.com/android-things-reptilians-170714180248/85/Using-Android-Things-to-Detect-Exterminate-Reptilians-42-320.jpg)
Using Android Things to Detect & Exterminate Reptilians
- 1. Using Android Things to Detect & Exterminate Reptilians
- 2. Android is EVERYWHERE (01) (01) Reptilians too are everywhere, but they suck.
- 5. "If you can build an app, you can build a device!"
- 6. 8-bit Distributed Piano (02) (02) https://riggaroo.co.za/android-things-building-distributed-piano/
- 7. Play Music Like a Scientist
- 8. Beer Can Music Player (03) (03) http://nilhcem.com/android-things/discovering-the-I2C-api-creating-a-capacitive-sensor-driver
- 10. Google Assistant powered Robot (05) (05) http://nilhcem.com/android-things/create-your-google-assistant-robots
- 11. OK, BUT, WHY? I mean... I can use an Arduino too
- 12. Perfect for • Doing some computation locally • Using Google Services • A secure platform • Prototype and production
- 13. Perfect for • Doing some computation locally • Using Google Services • A secure platform • Prototype and production
- 14. Perfect for • Doing some computation locally • Using Google Services • A secure platform • Prototype and production
- 15. Perfect for • Doing some computation locally • Using Google Services • A secure platform • Prototype and production
- 16. Compatible boards (06) • Raspberry Pi 3 • NXP i.MX7D/i.MX6UL • Intel Edison/Joule (discontinued) (06) https://developer.android.com/things/hardware/developer-kits.html
- 17. Raspberry Pi 3 - NXP i.MX7D
- 19. SOM (System on module)
- 20. SOM (System on module)
- 22. SOMs are pre-certified for regulator approvals
- 23. Perfect for • Doing some computation locally • Using Google Services • A secure platform • Prototype and production
- 24. Facts • Humans don't use their brain to its full capacity • Humans are distracted by media, entertainment, and politics. • Borders, country lines lead to endless wars and conflicts
- 25. Reason?
- 26. REPTILIANS
- 27. Reptilian conspiracy theory (08) (08) https://www.theatlantic.com/national/archive/2013/04/12-million-americans-believe-lizard-people-run-our-country/ 316706/
- 28. Reptilian • Altered our DNA so we don't fully use our brain • Control humans with media / politics • Have, through the generations, ruled the world
- 29. Using Android Things to Detect & Exterminate Reptilians
- 30. Detect
- 32. TensorFlow codelabs • TensorFlow for Poets (10) • TensorFlow for Poets 2: Optimize for Mobile (11) • Android Things Image Classifier (12) (12) https://codelabs.developers.google.com/codelabs/androidthings-classifier/index.html (11) https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2/ (10) https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/
- 33. Inception v3 • Deep learning model for image recognition • Transfer learning
- 34. Gather lots of images
- 36. Transfer learning python tensorflow/examples/image_retraining/retrain.py --bottleneck_dir=~/tf_files/bottlenecks --how_many_training_steps 4000 --model_dir=~/tf_files/inception --output_graph=~/tf_files/retrained_graph.pb --output_labels=~/tf_files/retrained_labels.txt --image_dir ~/tf_files/reptilians
- 37. Generated files • retrained_graph.pb • retrained_labels.txt
- 38. Generated files $ cat retrained_labels.txt human reptile reptilian
- 39. Mobile model optimizations python tensorflow/python/tools/optimize_for_inference.py --input=~/tf_files/retrained_graph.pb --output=~/tf_files/optimized_graph.pb --input_names="Mul" --output_names="final_result"
- 40. TensorFlow Android Classify sample (13) (13) https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android
- 41. TensorFlow Inference val inferenceInterface = TensorFlowInferenceInterface(getAssets(), MODEL_FILE) val labels = loadLabels() // [ "reptilian", "reptile", "human" ] fun doRecognize(image: Bitmap) { val pixels = image.toPixelsArray() inferenceInterface.feed(INPUT_NODE, pixels, NETWORK_STRUCTURE) inferenceInterface.run(arrayOf(OUTPUT_NODE)) val outputs = FloatArray(labels.size) inferenceInterface.fetch(OUTPUT_NODE, outputs) }
- 42. TensorFlow Inference labels = [ "reptilian", "reptile", "human" ] outputs = [ 0.0098f , 0.0039f , 0.9863f ]
- 43. Test data
- 48. AndroidManifest.xml <application> <uses-library android:name="com.google.android.things" /> <activity android:name=".MainActivity"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.IOT_LAUNCHER" /> <category android:name="android.intent.category.DEFAULT" /> </intent-filter> </activity> </application>
- 49. MainActivity class MainActivity : Activity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) setContentView(R.layout.activity_main) } }
- 50. "Hi! I use RelativeLayouts, lol"
- 51. <?xml version="1.0" encoding="utf-8"?> <FrameLayout xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent" android:layout_height="match_parent"> <ImageView android:src="@drawable/background" android:layout_width="match_parent" android:layout_height="match_parent" /> <ImageView android:src="@drawable/reptilian" android:layout_width="280dp" android:layout_height="224dp" android:layout_gravity="bottom|center_horizontal" /> </FrameLayout>
- 54. build.gradle dependencies { compileOnly 'com.google.android.things:androidthings:0.5.1-devpreview' }
- 57. Peripheral I/O APIs (14) • GPIO • PWM • I²C • I²S • SPI • UART (14) https://developer.android.com/things/sdk/pio/index.html
- 59. Blinking LEDs
- 61. val pioService = PeripheralManagerService() val ledGpio = pioService.openGpio("BCM4")
- 62. Magic happens here ledGpio.value = true Thread.sleep(1000) ledGpio.value = false
- 64. PeripheralManagerService fun openGpio(…): Gpio fun openPwm(…): Pwm fun openSpiDevice(…): SpiDevice fun openUartDevice(…): UartDevice fun openI2cDevice(…): I2cDevice fun openI2sDevice(…): I2sDevice
- 65. fun writeBuffer(device: I2cDevice, data: ByteArray) { device.write(data, data.size) }
- 66. Drivers are standard Android libraries (16) (17) (17) https://github.com/amitshekhariitbhu/awesome-android-things (16) https://github.com/androidthings/contrib-drivers
- 71. MainActivity.kt val display = Ssd1306("I2C1") display.setPixel(x = 0, y = 0, on = true)
- 72. val display = Ssd1306("I2C1") val bmp = BitmapFactory.decodeResource(resources, R.drawable.background) BitmapHelper.setBmpData(display, 0, 0, bmp, false) display.show()
- 75. Camera
- 80. No Android Things compatible drivers
- 81. Arduino to the rescue
- 82. Arduino to the rescue
- 83. Arduino to the rescue void loop() { distance = getDistanceInCentimeters(); Serial.println(distance); delay(1000); }
- 84. Arduino to the rescue
- 85. Read data from USB (20) Hcsr04LiveData(this) .observe({lifecycle}) { distanceInCm -> Log.i(TAG, "Distance (cm): $distanceInCm") } (20) https://github.com/Nilhcem/usbfun-androidthings
- 87. HC-SR04 Hcsr04LiveData(this) .observe({lifecycle}) { distanceInCm -> if (distanceInCm < 25) { takePicture() } }
- 88. git commit -m "Kill reptilian when detected" if (detected.isReptilian()) { exterminate() }
- 89. Exterminate
- 90. Option #1: Gun
- 91. Servo motor
- 92. Servo motor
- 93. Servo motor
- 95. Moving the servo val servo = Servo("PWM0") servo.setAngleRange(0.0, 180.0) servo.setPulseDurationRange(1.0, 2.0) servo.enable = true servo.angle = servo.minimumAngle
- 96. Moving the servo servo.angle = 42.0
- 101. Remote control • Nearby Connections API • Bluetooth LE • Embedded http server • Whatever you want
- 102. Option #3: Chainsaw
- 103. Relay module (22) (22) http://nilhcem.com/android-things/google-assistant-smart-home
- 104. Magic happens here relay.value = true
- 108. val colorAnimator = ValueAnimator.ofFloat(0f, 360f).apply { duration = 8000 repeatCount = ValueAnimator.INFINITE repeatMode = ValueAnimator.REVERSE addUpdateListener { animator -> val leds = getLedsColors(animator.animatedValue) ledstrip.write(leds) } start() }
- 109. val path = Path().apply { moveTo(0f, 0f) lineTo(0.001f, 400f / 2000f) // 0 secs -> 2.5 fps lineTo(3f / 23f, 400f / 2000f) // 3 secs -> 2.5 fps lineTo(6f / 23f, 20f / 2000f) // 6 secs -> 50.0 fps lineTo(18f / 23f, 20f / 2000f) // 18 secs -> 50.0 fps lineTo(1f, 1f) // 23 secs -> 0.5 fps } val frameDelayAnimator = ValueAnimator.ofInt(0, 2000).apply { duration = 23000 repeatCount = 0 interpolator = PathInterpolator(path) start() }
- 110. Using Android Things to Detect & Exterminate Reptilians
- 112. Conclusion
- 113. Android Things is Fun
- 114. Easy to get started
- 115. Reptilians suck (23) (23) Unless Reptilians are just an analogy to tell you not to blindly trust media and get you think about reality in a different way
- 116. Be a maker Try Android Things, really
- 117. Using Android Things to Detect & Exterminate Reptilians • Twitter: @Nilhcem • Slides: bit.ly/reptilethings