PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs
1 like535 views
The document discusses the integration of Firebase with cloud functions and machine learning APIs, specifically utilizing Google's platform for application development. It provides an example of a 'hot dog or not' app that uses Firebase's cloud functions, storage, and the Google Cloud Vision API for image analysis. Key features include real-time database capabilities, user authentication, and the use of machine learning for image classification through label detection.
![@carlosazaustre
1. Take a Picture
const file = event.target.files[0]
const task = firebase.storage().ref(`/uploads/${file.name}`)
task.on('state_changed', snapshot => {
// Update progress indicator
}, error => {
// Show error details
}, () => {
// Upload completed!
this.imageUrl = task.snapshot.downloadURL
})
1
2](https://image.slidesharecdn.com/pixelscamp-cloudfunctionsmachinelearningapis-170929201958/85/PixelsCamp-2017-Cloud-Functions-for-Firebase-and-Machine-Learning-APIs-11-320.jpg){kind=tracking}
![@carlosazaustre
4. Use LabelDetection from GC Vision API
return vision.labelDetection({ source: { imageUri: file } })
.then(response => {
const labels = response[0].labelAnnotations
let isHotdog = false
labels.map(label => {
if(label.description === `hot dog`) {
isHotdog = true
}
})
return admin.database().ref(`/uploads/photo`)
.update({ isHotdog })
})
const file = `gs://${object.bucket}/${object.name}`](https://image.slidesharecdn.com/pixelscamp-cloudfunctionsmachinelearningapis-170929201958/85/PixelsCamp-2017-Cloud-Functions-for-Firebase-and-Machine-Learning-APIs-17-320.jpg){kind=tracking}
![@carlosazaustre
4. Use LabelDetection from GC Vision API
return vision.labelDetection({ source: { imageUri: file } })
.then(response => {
const labels = response[0].labelAnnotations
let isHotdog = false
labels.map(label => {
if(label.description === `hot dog`) {
isHotdog = true
}
})
return admin.database().ref(`/uploads/photo`)
.update({ isHotdog })
})
const file = `gs://${object.bucket}/${object.name}`](https://image.slidesharecdn.com/pixelscamp-cloudfunctionsmachinelearningapis-170929201958/85/PixelsCamp-2017-Cloud-Functions-for-Firebase-and-Machine-Learning-APIs-18-320.jpg){kind=tracking}
PixelsCamp 2017 - Cloud Functions for Firebase and Machine Learning APIs
- 1. Cloud Functions & Machine Learning APIs @carlosazaustre carlosazaustre.es
- 2. @carlosazaustre Carlos Azaustre Telecommunications Engineer Front-End Developer CTO & CoFounder Chefly @carlosazaustre https://carlosazaustre.es youtube.com/carlosazaustre
- 3. @carlosazaustre What is Firebase? Google Platform to build apps fast without managing infrastructure
- 4. @carlosazaustre Firebase Services ● Realtime NoSQL Database ● User Authentication ● Storage ● Hosting ● Push Notifications ● And Cloud Functions! (and much more)
- 5. @carlosazaustre Cloud Functions Develop a Backend without managing servers. Create functions that are triggered by Firebase products such as changes to data in Database, new user sign ups via Auth, file uploads to Storage, etc...
- 6. @carlosazaustre How works Cloud Functions? Firebase Product Trigger ● Database ● Storage ● Auth ● HTTPS ● Hosting ● PubSub Function Handler ● Code write in Node.js (v6) ● You can use GCP easily ● You can use Google APIs ● You can use NPM libraries ● You can use 3rd party APIs
- 7. @carlosazaustre What are Machine Learning APIs? Google Cloud Platform provides several API’s to use in opur projects such as ● Vision API ● Speech API ● Translation API ● Natural Language API ● Video Intelligence API Or you can train your models with TensorFlow and Machine Learning Engine
- 9. @carlosazaustre What is Hot Dog or Not App?
- 10. @carlosazaustre Frontend Cloud Storage Firebase Cloud Functions Firebase Cloud Vision API LabelDetection Database Firebase App Architecture
- 11. @carlosazaustre 1. Take a Picture const file = event.target.files[0] const task = firebase.storage().ref(`/uploads/${file.name}`) task.on('state_changed', snapshot => { // Update progress indicator }, error => { // Show error details }, () => { // Upload completed! this.imageUrl = task.snapshot.downloadURL }) 1 2
- 12. @carlosazaustre 2. Trigger a Cloud Function exports.isHotdog = functions.storage.object() .onChange(event => { ... }) /uploads/hotdog.jpg functions.storage.object().onChange isHotdog <Function>
- 13. @carlosazaustre 3. Take a File reference on Google Cloud Storage exports.isHotdog = functions.storage.object() .onChange(event => { const object = event.data const file = `gs://${object.bucket}/${object.name}` }) /uploads/hotdog.jpg gs://bucketName/hotdog.jpg
- 14. @carlosazaustre Cloud Vision API ● Label Detection ● Explicit Content Detection ● Logo Detection ● Landmark Detection ● Optical Character Recognition ● Face Detection ● Image Attributes ● Web Detection
- 15. @carlosazaustre Cloud Vision API "labelAnnotations": [ { "mid": "/m/0hhrcvd", "description": "frankfurter würstchen", "score": 0.96459 { "mid": "/m/01b9xk", "description": "hot dog", "score": 0.9400299 }, { "mid": "/m/0kdzn", "description": "sausage", "score": 0.93496704 }, { "mid": "/m/09370d", "description": "bockwurst", "score": 0.9349343 }, { "mid": "/m/01sn4c", "description": "kielbasa", "score": 0.9139112 }, { "mid": "/m/079yjd", "description": "knackwurst", "score": 0.8636358 }, { "mid": "/m/03f7_0", "description": "bologna sausage", "score": 0.8028193 }, { "mid": "/m/038rz", "description": "german food", "score": 0.76827973 }, { "mid": "/m/04pfxj", "description": "cervelat", "score": 0.7612521 }, { "mid": "/m/06l9l9", "description": "vienna sausage", "score": 0.7339414 },
- 16. @carlosazaustre Cloud Vision API "labelAnnotations": [ { "mid": "/m/0hhrcvd", "description": "frankfurter würstchen", "score": 0.96459 { "mid": "/m/01b9xk", "description": "hot dog", "score": 0.9400299 }, { "mid": "/m/0kdzn", "description": "sausage", "score": 0.93496704 }, { "mid": "/m/09370d", "description": "bockwurst", "score": 0.9349343 }, { "mid": "/m/01sn4c", "description": "kielbasa", "score": 0.9139112 }, { "mid": "/m/079yjd", "description": "knackwurst", "score": 0.8636358 }, { "mid": "/m/03f7_0", "description": "bologna sausage", "score": 0.8028193 }, { "mid": "/m/038rz", "description": "german food", "score": 0.76827973 }, { "mid": "/m/04pfxj", "description": "cervelat", "score": 0.7612521 }, { "mid": "/m/06l9l9", "description": "vienna sausage", "score": 0.7339414 },
- 17. @carlosazaustre 4. Use LabelDetection from GC Vision API return vision.labelDetection({ source: { imageUri: file } }) .then(response => { const labels = response[0].labelAnnotations let isHotdog = false labels.map(label => { if(label.description === `hot dog`) { isHotdog = true } }) return admin.database().ref(`/uploads/photo`) .update({ isHotdog }) }) const file = `gs://${object.bucket}/${object.name}`
- 18. @carlosazaustre 4. Use LabelDetection from GC Vision API return vision.labelDetection({ source: { imageUri: file } }) .then(response => { const labels = response[0].labelAnnotations let isHotdog = false labels.map(label => { if(label.description === `hot dog`) { isHotdog = true } }) return admin.database().ref(`/uploads/photo`) .update({ isHotdog }) }) const file = `gs://${object.bucket}/${object.name}`
- 19. @carlosazaustre 5. Frontend App reacts to changes //AppImage.vue created() { firebase.database() .ref(`/upload`) .on(value, snapshot => { let isHotdog = snapshot.val().photo.isHotdog this.$emit(`upload`, { isHotdog }) }) App.vue AppHeader.vue AppImage.vue emit (‘upload’, isHotdog)props: isHotdog