Biq query devfest2017_slides

BIGQUERY
Code Labs
DevFest DC - 2017

SELECT tweet.@dineshMD,email.dinesh@acarin.net from Contacts
Dinesh Ganesan

Why do we need Big Data tools?
BigQuery - Show the power
Getting Started
Let’s Play - Analyzing USA Census Data within few seconds.
Loading data into BigQuery
Talking to BigQuery
Q&A
4
Agenda?

Why Big Data?
5
3. Why “Big Data” Is a Big Deal - Harvard Magazine
Enabled Biologist to compute the genome sequence within 24 hours rather waiting for a full year.
- Reducing Cost & Time
Enabled NASA to provide Climate Modeling as Service(MERRA) using data since 1979 ( few hundred peta bytes )
- Better Living and New business opportunities.
2012 – 100 Billion Google Searches every month. Leverages 30 trillion URLs for data collection
- Optimal Knowledge from huge data size.
2. MERRA: MODERN ERA-RETROSPECTIVE ANALYSIS FOR RESEARCH AND APPLICATIONS
1. Google Search - Wikipedia

Getting Started - GCloud Registration
1. https://console.cloud.google.com
1
2 3
6

Getting Started - Project Creation
1
3
7
2
4
Refresh the page if you don’t see the project name

Getting Started - Enable API
8
5

Query Editor
11
Query Editor
UDF Editor

Query Execution
select year, SUM(record_weight) as Births from
publicdata:samples.natality where (mother_race = 18 and
father_race = 18) group by year order by year;
Asian Indian Kids born
The race of the child. One of the following numbers:
1 - White
2 - Black
3 - American Indian
4 - Chinese
5 - Japanese
6 - Hawaiian
7 - Filipino
9 - Unknown/Other
18 - Asian Indian
28 - Korean
39 - Samoan
48 - Vietnamese

SOURCE DATA Formats
13
CSV
JSON
Avro
Cloud datastore backups

SOURCE DATA Formats
14
http://www.ssa.gov/OACT/babynames/names.zip
https://goo.gl/qZW9sx

Loading data - CSV Direct Upload
15
1
2
3

16
4

17
4
name:string,gender:string,count:integer

Loading Data - GCP Storage
18
gs://ssa-babynames/names/yob2013.txt
How about loading multiple ﬁles into a single
table?

Running Queries
19
Standard SQL

Simple Aggregates
20
Select COUNT(GoogleDevs),MAX(GoogleDevs),
STDDEV(GoogleDevs) FROM …

Complex Processing
21
Select … FROM … where
REGEXP_MATCH(url,”Acarin$”) AND user
CONTAINS ‘Mr.Foo’

Nested Select
22
Select count(*) from ( Select devGurus ….) Group
BY Name

Small JOIN
23
Select tbl_school.student_id from tbl_school JOIN
tbl_classroom ON tbl_classroom.student_id =
tbl_shool.student_id

Accessing data through API
25
https://developers.google.com/api-client-library/

Validating API
26
https://developers.google.com/apis-explorer/#p/bigquery/v2/

Integrating with Google Sheet
28
https://developers.google.com/apps-script/advanced/bigquery
https://goo.gl/evaRRC
1 2

Q&A
T H A N K Y O U
SELECT @dineshMD from Contact.tweet

Biq query devfest2017_slides

More Related Content

What's hot (20)

Similar to Biq query devfest2017_slides (20)

Recently uploaded (20)

Biq query devfest2017_slides