Google Glass, the GDK, and HTML5

Google Glass, the GDK, and
HTML5-Based Mobile Apps
SVCC (10/11/2014)
Oswald Campesato
ocampesato@yahoo.com

Topics in This Session
Google Glass and Glassware
Glass Features (condensed summary)
What is the Mirror API
Some HTML5 Technologies
What is the GDK
PhoneGap and Google Glass
What about Android Wear

Some Glass Features (briefly)
Directions (“ok glass…get directions”)
Video (“ok glass…take a video”)
Camera (“ok glass…take a picture”)
Translate (scan text and translate)
Answer gmail (voice input)
Voice-activated Android apps
Pictures/videos are backed up on G+
Touch pad (gesture detection in GDK):
Swipe left/right/down; tap; scroll

How to Create Glassware
 Use the Glass Mirror API (lightweight/server-side)
 Use Eclipse (or Android Studio) with the GDK
 Use PhoneGap and add Glass-related plug-ins
 Use the GDK + Mirror API to create “hybrid” apps:
https://github.com/mimming/kitty-compass
 Note: Glass hybrid apps refer to GDK + Mirror API,
which is different from HTML5 hybrid apps

Glass Pattern Building Blocks
 UI elements:
+ Static cards (Card class is deprecated)
+ Live cards (Android service): low & high frequency
+ Immersions
 Voice commands:
+ Focus on the action, not the agent
+ Decrease time from intent to action
+ Is colloquial and easy to say
+ Is general enough to apply to multiple Glassware
 https://developers.google.com/glass/design/patterns

Internet Access for Glass
Go to https://glass.google.com/myglass
Enter the name of a WiFi and the password
Scan the QR code with Glass
“Okay Glass…get directions” (or whatever)
Note: GG remains tethered to your Android
device even when you switch it off

Screencasting GG via MyGlass
1) Launch MyGlass on your Android phone
2) Tap “MyGlass” (top of device screen)
3) Tap “Screencast” (bottom of drop-down list)
4) “Swipe” left or right on your Android device
+ Swipe gestures update GG (and vice versa)
+ swipe left/forward: displays the present/future
+ swipe right/backward: displays the past
NB: Screencasting Glass-to-iOS-device works as
well, but not from an iOS device to Glass

The Glass Mirror API (1)
“lightweight”: processes only on server-side
Reduces battery consumption (versus GDK)
Provides access to server-side Glass APIs
Easy to “install”
Supports static cards and timeline mgmt
Support for various languages:
Python/JavaScript/Node/Java
A ‘sandbox’ is available
Access requires a registered Glass device

Static cards can contain:
+ Text, HTML, media (images and videos)
+ Standard and custom menu items
+ multiple “pages” (articles)
+ auto or manual pagination
Timeline management:
+ Subscribe to timeline notifications
+ LBS (location-based services)

 Disadvantages of using the Mirror API:
1) Internet access is required:
+ apps only work when in WiFi range
2) limited number of API calls (1000/day)
3) no voice trigger
4) No real-time (no animation)
5) Time delay receiving server response(s)
6) No device access:
+ Camera/Compass/GPS
NB: infrequent location updates

Glass Mirror API: what happens?
Step #1: user authenticates
Step #2: Glass stores credentials (server-side)
Step #3: Glass inserts static card in timeline
Step #4: Glass synchronizes timeline
Step #5: user sees static card on Glass
Glass Mirror API Quick Start Samples:
https://developers.google.com/glass/develop/
mirror/quickstart/index

The GDK: Glass Development Kit
 An “add-on” to the Android SDK
 The GDK provides Android-based APIs
 provides Glass-specific APIs
 GDK-based Glassware runs on Glass itself
 Use standard Android development tools
 ICS (4.0.3) or higher
 Simple installation: adb install abc.apk
 Supports Timeline, cards, and immersions
 Support for sensors, Menu, and UI
 Support for touch pad and gestures
 Support for media (sound/camera/voice)
 https://developers.google.com/glass

The +/- of the Glass GDK
 Advantages:
+ GDK-based apps work offline (Internet not required)
+ Unlimited access to GDK-based apps
+ Support for voice trigger
+ Real-time support (smoother experience)
+ support for animation effects
+ No delay (no server access required)
+ Device access (camera/compass/etc)
 Disadvantages:
+ apps can drain the battery
+ more difficult to install GDK-based apps

Glass GDK Set-up
Option #1 software installation:
Eclipse (3.5 or higher?)
JDK 1.6 (or higher)
ADT installation
Option #2 software installation:
Android Studio (complete zip file)

Enable Debug Mode on Glass
+ swipe to "Settings" -> "Device info”
+ tap to open device menu
+ select "Turn on debug"

The Android ‘adb’ Utility
 Install apks on devices: adb install abc.apk
 List available devices: adb devices
 List apks on Glass: adb shell pm list packages –f
 Launch the Glass home card:
adb shell am start -n
com.google.glass.home/.timeline.MainTimelineActivity

Create/Deploy “Hello World” in Eclipse
 CREATE the application:
1) Launch Eclipse or Android Studio
2) File > New > Android Application Application
3) specify “HelloWorld” for “Application Name”
4) API 15 (or API 19) for all SDK-related options
5) Click “Next” 4 times and click “Finish” once
DEPLOY the app to a device:
 Run > Run > Android Application > OK
 ELAPSED TIME: 60 seconds

Android Project Structure
 The top-level project directory contains the following:
 ./src: Java code (usually in a package)
 ./res/values: XML files with text strings and layout styles
 ./res/layout: XML files for layouts (such as activity classes)
 ./res/drawable: static images (*)
 ./res/assets/www: HTML-related assets
 ./libs: JAR files
 ./gen: generated code
 ./bin: “apk” file (and other stuff)
 AndroidManifest.xml is the main “control” file
 NB: assets are “mapped” to properties in R.java
 (*) drawable-hdpi, drawable-mdpi, and so forth

Android App Components
 Activity: essentially a screen in an application
 Intent: launch Activity (provides other “stuff”)
 Service: background operations (no UI), such as
downloads
 Broadcast receiver (listens to system-wide events)
 Content provider: manages shared app data, such as
contacts
 Also defined in AndroidManifest.xml

The PhoneGap Way (HTML5)
 Install node and npm
 Install PhoneGap: npm install –g phonegap
 Create an app: cordova create test1
 Go to root directory: cd test1
 Add Android: cordova platform add android
 cordova plugin add https://github.com/aphex/cordova-glass-
core
touch
gesture
 Deploy Android apk: cordova run

What is CardBuilder?
 Replaces deprecated Card class with new layouts
 TEXT and TEXT_FIXED: text content and a footer
 COLUMNS and COLUMNS_FIXED: display a mosaic or
icon on the left side and text on the right side
 AUTHOR: an avatar, primary+secondary headings,
and body text
 TITLE: an image or mosaic with a name and icon
 ALERT and MENU: for dialogs with warning messages
and action confirmation flows
 improvements to GestureDetector’s scroll handling
 GDK samples fully support Android Studio

What is a Glass Card?
 A Card is “a unit of work” (one screen)
 A Card contains:
+ Main body text
+ Left-aligned footer
+ Right-aligned footer for a timestamp
+ One image for the background
 Creating a (deprecated) Card (native code):
+ set the properties of the card
+ invoke Card.toView() to convert to an Android view

Glass Live Cards
 Native Glassware “sits” on Timeline (left-side)
 display real-time info (weather/news/etc)
 frequent updates (running in background)
 Similar to Android Widgets (or Service)
 Display text and images (and 3D graphics)
 a 'stop' option is required for live cards

Live Card Project (1)
 Create Android Project in Eclipse (or Android
Studio)
 Specify version 15 or higher
 Rename MainActivity to MenuActivity
 Remove launcher/main intent filters from manifest
 Add Service as the main component, which
handles Live Card creation/management

Live Card Project (2)
 Implement Live Card Service (override 2 methods)
 onStartCommand() method:
 called when service is started
 Create a live card and publish it
 If live card already exists, then display it
 onDestroy() method:
 Called when service is destroyed
 Unpublish the live card

Glass Immersions
 They run outside the Timeline
 Typically consist of Android Activities
 fully-customizable screens
 you can design your own UI
 Access to all user input
 Better for interactive experiences

Touch Gestures on Glass
 GDK provides a gesture detector for the touchpad
 Gestures include: tapping/swiping/scrolling
Working with gestures in the GDK:
+ create a GestureDetector
+ implement one or more listeners
+ override onGenericMotionEvent()
+ pass motion events to onMotionEvent()

Glass and Voice Activation (1)
 OLD contents of <intent-filter> in AndroidManifest.xml:
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
 NEW contents of <intent-filter> in AndroidManifest.xml:
<intent-filter>
<action
android:name="com.google.android.glass.action.VOICE_TRIGGER" />
<meta-data
android:name="com.google.android.glass.VoiceTrigger"
android:resource="@xml/voice_trigger" />
</intent-filter>

Glass and Voice Activation (2)
 Contents of $APP_TOP/res/xml/voice_trigger.xml:
<?xml version="1.0" encoding="utf-8"?>
<trigger keyword="@string/glass_voice_trigger">
<input prompt="@string/glass_voice_prompt" />
</trigger>
 Contents of $APP_TOP/res/values/strings.xml:
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="glass_voice_trigger">take note</string>
<string name="glass_voice_prompt">tell me something</string>
</resources>

Sensors on Google Glass
 Supported sensors:
 TYPE_ACCELEROMETER
 TYPE_GRAVITY
 TYPE_GYROSCOPE
 TYPE_LIGHT
 TYPE_LINEAR_ACCELERATION
 TYPE_MAGNETIC_FIELD
 TYPE_ROTATION_VECTOR
 Unsupported sensors:
 TYPE_AMBIENT_TEMPERATURE
 TYPE_PRESSURE
 TYPE_PROXIMITY
 TYPE_RELATIVE_HUMIDITY
 TYPE_TEMPERATURE

Glass Mirror API versus GDK
Use Mirror API for:
+ platform independence
+ common infrastructure
+ built-in functionality
Use GDK for:
+ real-time interaction
+ access to hardware
+ offline functionality
Use the GDK and the Mirror API

What is HTML5?
 “Anything that you’re working on right now that’s
cool…that’s HTML5” (PPK)
 HTML5 means different things to different people
 Multiple Technologies (see W3C)
W3C REC status: MathML, SVG, and 2 others

Technologies “in” HTML5
 Canvas 2D
 CSS3
 Drag-and-Drop (DnD)
 File API
 Geolocation
 Microdata
 Offline Applications
 Server Sent Events (SSE)
 SVG
 Web Intents
 Web Messaging
 Web Storage
 Web Sockets
 Web Workers

HTML5 WAI ARIA
Web Accessibility Initiative: Accessible Rich
Internet Applications
W3C specification for increasing the accessibility
of web pages and user interface components
developed with Ajax, HTML, JavaScript and
related technologies.
WAI-ARIA predates HTML5
WAI ARIA specification defines support for
accessible web applications, which involves
markup extensions that are often attributes of
HTML5 elements

CSS3 Features (condensed)
(Perf) Gradients (linear and radial)
(Perf) Rounded corners/drop shadows
CSS3 filters (merged with SVG filters)
Simple shapes (circle/ellipse/rectangle)
(Perf) 2D and 3D animation
Multi-column support
Exclusions and regions
Flexbox (like “grid bag” in Java AWT)

CSS3 2D/3D Transforms
CSS3 2D transforms:
rotate, scale, translate, matrix, and
perspective, and skew (deprecated)
CSS3 3D transforms (invoke the GPU):
1) similar to 2D versions (but no ‘skew’)
2) also three axis-specific methods:
rotateX, rotateY, rotateZ (and so forth)
• CSS3 animation via a keyframes rule

What “works” with CSS3?
Combine these with CSS3 in a Web Page:
+ SVG and D3.js (=JavaScript+SVG)
+ HTML5 Canvas (bitmap and no DOM)
For Web applications:
jQuery (cross-browser + cross-platform)
BackboneJS (reduces spaghetti code)
AngularJS (open source from Google)
Other JavaScript-based toolkits

What is HTML5 Canvas?
HTML5 <canvas> element
a rectangular bitmap in a web page
Direct mode (SVG is “deferred” mode)
Draw-and-forget (“Etch-a-Sketch”)
JavaScript APIs for rendering shapes
Support for 2D but not for 3D
Available on iOS5+ and Android ICS+
Hardware accelerated (RIM Playbook)

Basic Shapes in HTML5 Canvas
Line segments
Circles and arcs
Ellipses (with the scale transform)
Bezier curves (quadratic/cubic)
Polygons
Paths (for arbitrary shapes)

What You can Do with 2D Shapes
Linear and radial gradients
Rotate, scale, skew, translate methods
Perspective and matrix() method
2D Animation effects

“Template” for HTML5 Canvas (Part 1)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Graphics with HTML5 Canvas</title>
<script type="text/javascript"><!--
window.addEventListener('load', function () {
// Step #1: Get the canvas element
var elem = document.getElementById('myCanvas');
if (!elem || !elem.getContext) {
return;
}

“Template” for HTML5 Canvas (Part 2)
// Step #2: Get the canvas 2d context
var context = elem.getContext('2d');
if (!context) {
return;
}
// Step #3: INSERT YOUR CANVAS CODE HERE
});
// --></script>
</head>
<body>
<canvas id="myCanvas" width=“800" height=“500">
</canvas>
</body>
</html>

What is D3.js?
 D3.js: an open source project (circa 2010)
 Created by Mike Bostock
 based on JavaScript
 a layer of "abstraction" over SVG
 also support for HTML5 Canvas
 github.com/mbostock/d3
 https://github.com/mbostock/d3/wiki/Gallery

Why/When use D3.js?
For data visualization
If you need “extreme versatility”
leverage JavaScript skills
leverage SVG
HTML5 Web pages with D3.js and:
HTML5 Canvas, CSS3, SVG, jQuery, …

What Can D3.js Do?
All the stuff you can do in SVG
graphics/animation (2D only)
filters/gradients
mouse/keyboard events
custom charts/graphs
Support for Ajax, JSON, XML, CSV files

Simple D3.js Example
<html>
<head>
<script src="d3.min.js"></script>
<script>
d3.select("body")
.append("p")
.text("Hello From D3");
</script>
</head>
<body>
<p>Hello From D3</p>
</body>
</html>

D3 and Other Technologies
You can create HTML5 web pages with:
+ D3 & Bootstrap + Firebase (or Parse)
+ D3 & BackboneJS
+ D3 & AngularJS
+ D3 & HTML5 Web Sockets
+ D3 & NodeJS (or Meteor)

What about SVG?
XML-based vocabulary for 2D shapes
Gradients/Bezier curves/polygons/animation
DOM-based and declarative syntax
 Supported in iOS5+ and Android ICS+
Partial hardware acceleration
 No support for 3D effects
SVG versus Canvas:
http://msdn.microsoft.com/en-us/hh552482.aspx

Useful Features of SVG
render circles/ellipses/elliptic arcs
squares/rectangles/parallelograms
cubic/quadratic Bezier curves
arbitrary polygons
linear/radial gradients and filters
mouse events and animation support (*)
good for charts/graphs
Works with CSS3 (reference SVG via “url()”
(*) consider using D3.js

Other Aspects of SVG
 SVG elements are inserted in the DOM so you can
track/manage groups of SVG elements
 no blurred/jagged edges when zooming in
 Convenient format for import/export between tools
 Can apply XSL stylesheets to SVG documents
Blog by Patrick Dengler: SVG versus Canvas

Disadvantages of SVG
• Verbose (what do you expect? It’s XML)
• Can be difficult/incomprehensible (SVG tiger)
Animation code can be cumbersome/tedious
 Consider using D3 instead of “pure” SVG

Strengths of CSS3/SVG/Canvas
 CSS3: a) 2D/3D graphics/animation, b) GPU support,
c) embeddable in SVG <defs> element, d) “easing
functions” for animation, e) animate HTML elements
 Canvas: a) 2D graphics/animation, b) GPU support,
c) good for updating many small objects (games),
d) works with video (use ThreeJS/WebGL for 3D
animation)
 SVG: a) 2D graphics/animation, b) some GPU
support, c) create arbitrary 2D shapes, d) custom
<pattern>s, and e) grouping via the <g> element,
and “easing functions in D3 (but SVG cannot
animate HTML)

Combining CSS3 and SVG
CSS3 can leverage the power of SVG:
+ reference SVG documents via “url()”
• SVG can leverage CSS3 by:
embedding CSS selectors in an SVG doc
use jQuery methods:
+ the css() method for updating properties
• Use jQuery Mobile features:
use bindings to handle mouse/touch events

CSS3+SVG or CSS3+D3.js?
CSS3+SVG: useful when 1) you already have
SVG-based data and 2) you must support IE6
(can do with Raphael toolkit but not D3)
• CSS3+D3: for modern browsers, and also easier
for defining event handlers and animation
(probably also easier to maintain/enhance)
• Notes:
• CSS3+SVG might not have GPU support
• Consider D3 with BackboneJS/AngularJS/etc

What about CSS3 and HTML5 Canvas?
Insert a <canvas> element in a Web page
• JavaScript APIs for 2D shapes (similar to Skia)
All shapes are rendered in a <canvas> element
Modern browsers support <canvas> element
“write-and-forget” (no DOM-based “history”)
• CSS3 selectors can match <canvas> element
• Canvas often renders polygons faster than SVG
• Tracing Canvas calls:
http://www.html5rocks.com/en/tutorials/canvas/i
nspection/

Some Useful CSS3 Resources
• caniuse.com
online forums (Yahoo group)
Various meetup groups in BA
stackoverflow.com

Impressive HTML5 Canvas Demos
• Various HTML5 Canvas demos:
http://www.craftymind.com/factory/html5video/C
anvasVideo.html
http://codepen.io/stuffit/pen/KrAwx
http://davidwalsh.name/canvas-demos

PhoneGap and Glass
Perform the following steps:
+ Install PhoneGap: npm install –g phonegap
+ Create a PhoneGap mobile app (CLI)
+ Add HTML/CSS/JavaScript assets
+ Deploy to Google Glass via an IDE
+ command line installation:
adb install –r myapp.apk

CSS3 and Glass Demo
CSS3 2D skew (for the three cube faces)
CSS3 3D linear/radial gradients
CSS3 2D and 3D transforms
CSS3 3D animation effects (keyframes)

Demos: Canvas and Glass
 Demo #1 uses Android Canvas:
 touch-enabled bar chart
 uses the GDK with Android Canvas
 Demo #2 uses HTML5 Canvas:
 PhoneGap-based application

What about Android Wear
Android Wear provides a separate SDK
Not compatible with the GDK(?)
Different paradigm from Glass

Google Glass Official Sites
 Currently available Glassware:
 https://glass.google.com/glassware
 Purchasing Google Glass:
https://glass.google.com/getglass/shop/glass
Google Glass community:
https://www.glass-community.
com/?auth_token=y8WXiHyf_TbV-
_0aDamI9Q1UFE5tTkEUyWCHoPPAy10&authuser=0&hl=
en_US&ts=1405039468196

Bonus: NodeJS and Quadcopters
 var arDrone = require('ar-drone');
 var client = arDrone.createClient();
 client.takeoff();
 client
 .after(5000, function() {
 this.clockwise(0.5);
 })
 this.animate('flipLeft', 15);
 })
 this.stop();
 this.land();
 });

Open Source Projects (graphics)
• Graphics Projects on http://code.google.com/p:
+ css3-graphics and html5-canvas-graphics
+ css3-jquery-graphics and d3-graphics
+ svg-graphics and svg-filters-graphics
+ easel-graphics, fabric-graphics, paper-graphics
+ ThreeJS, jQuery, Raphael, Google Go, Android
+ Dart, Dojo, JSF, Two.js, JavaFX 2.0
+ Lua, PHP, Perl, Python, Ruby, SWT graphics
Graphics Projects on github (ocampesato):
+ Famo.us, Polymer, AngularJS

NodeJS Course (10/19-10/21)
Adam CrabTree
Enerspace
2225 East Bayshore Road
Suite 100
Palo Alto, CA 94303
http://www.eventbrite.com/e/noderietytm-nodejs-
bootcamp-october-tickets-12539314427

Recent/Upcoming Books
1) HTML5 Canvas and CSS3 Graphics (2012)
2) jQuery, CSS3, and HTML5 for Mobile (2013)
3) HTML5 Pocket Primer (2013)
4) jQuery Pocket Primer (2013)
5) HTML5 Mobile Pocket Primer (2014)
6) D3 Pocket Primer (2014)
7) Google Glass Pocket Primer (2015)
8) AngularJS Pocket Primer (2015)
• “WebGL: Up and Running” (Tony Parisi)
Co-creator of VRML and X3D

Google Glass, the GDK, and HTML5

More Related Content

What's hot (20)

Similar to Google Glass, the GDK, and HTML5 (20)

More from Oswald Campesato (20)

Recently uploaded (20)

Google Glass, the GDK, and HTML5