Ask bigger questions
1 like142 views
The document discusses a shift from traditional data processing methods to a more integrated, data-centric approach in enterprise data management. It emphasizes the importance of analyzing diverse data types and provides perspectives on leveraging big data across various industries, including credit card processing, retail, and healthcare. The new paradigm focuses on bringing compute to data, facilitating faster analytics, and reducing costs through flexible and scalable solutions.
Ask bigger questions
- 1. 1  Becoming  Informa/on-ÂâDriven  Introduc/on  to  the  Enterprise  Data  Hub  Mike  Olson  Cloudera,  Inc.  Co-ÂâFounder  &  Chief  Strategy  OďŹcer Â
- 2. 2  Expanding  Data  Requires  A  New  Approach  Š2014  Cloudera,  Inc.  All  rights  reserved.  2  1980s  Bring  Data  to  Compute  Now  Bring  Compute  to  Data  RelaEve  size  &  complexity  Data  InformaEon-Ââcentric  businesses  use  all  data:    Mul/-Ââstructured,   internal  &  external  data   of  all  types  Compute  Compute  Compute  Process-Ââcentric   businesses  use:   â˘âŻStructured  data  mainly  â˘âŻInternal  data  only  â˘âŻâImportantâ  data  only    Compute  Compute  Compute  Data  Data  Data  Data Â
- 3. 3  The  Old  Way:  Bringing  Data  to  Compute  Š2014  Cloudera,  Inc.  All  rights  reserved.  3  Complex  Architecture  â˘âŻ Many  special-Ââpurpose  systems  â˘âŻ Moving  data  around  â˘âŻ No  complete  views  Visibility  â˘âŻ Leaving  data  behind  â˘âŻ Risk  and  compliance  â˘âŻ High  cost  of  storage  Time  to  Data  â˘âŻ Up-Ââfront  modeling  â˘âŻ Transforms  slow  â˘âŻ Transforms  lose  data  Cost  of  AnalyEcs  â˘âŻ Exis/ng  systems  strained  â˘âŻ No  agility  â˘âŻ BI  backlog  4  1  2  3  SERVERS  MARTS  EDWS  DOCUMENTS  STORAGE  SEARCH  ARCHIVE  ERP,  CRM,  RDBMS,  MACHINES  FILES,  IMAGES,  VIDEOS,  LOGS,  CLICKSTREAMS  EXTERNAL  DATA  SOURCES Â
- 4. 4  SERVERS  MARTS  EDWS  DOCUMENTS  STORAGE  SEARCH  ARCHIVE  ERP,  CRM,  RDBMS,  MACHINES  FILES,  IMAGES,  VIDEOS,  LOGS,  CLICKSTREAMS  ESTERNAL  DATA  SOURCES  Š2014  Cloudera,  Inc.  All  rights  reserved.  MulE-Ââworkload  analyEc  plaRorm  â˘âŻ Bring  applica/ons  to  data  â˘âŻ Combine  diďŹerent  workloads  on   common  data  (i.e.  SQL  +  Search)  â˘âŻ True  BI  agility  4  1  2  3  4  The  New  Way:  Bringing  Compute  to  Data  4  AcEve  archive  â˘âŻ Full  ďŹdelity  original  data  â˘âŻ IndeďŹnite  /me,  any  source  â˘âŻ Lowest  cost  storage  1  Data  management,  transforms  â˘âŻ One  source  of  data  for  all  analy/cs  â˘âŻ Persist  state  of  transformed  data  â˘âŻ SigniďŹcantly  faster  &  cheaper  2  Self-Ââservice  exploratory  BI  â˘âŻ Simple  search  +  BI  tools  â˘âŻ âSchema  on  readâ  agility  â˘âŻ Reduce  BI  user  backlog  requests  3 Â
- 5. 5  Beeer,  faster,  cheaper  and  mul/-Ââframework  BATCH  PROCESSING  MR  /  PIG/  Hive  /  Cascading  SQL  IMPALA  SEARCH  SOLR  MACHINE  LEARNING  SAS,  R,  H20,  MLlib  STREAM  PROCESSING  SPARK  STREAMING  NOSQL  HBASE  Process  Data  IN-ÂâMEMORY  SPARK  Train  &  Test  Models  Respond  to  Events  in  RT  Explore  &  Analyze  Data  â˘âŻHighly  mature  â˘âŻWide  range  of  clients  â˘âŻSigniďŹcant  advances  in  speed  &  usability  â˘âŻIntegra/on  with  the  SAS  &  Revolu/on  product  porgolio  â˘âŻPython  /  0xdata  /  ML  lib  for  advanced  users  â˘âŻVery  low  (~10ms)  latency  â˘âŻHigh  volumes  of  single  events  â˘âŻHigh  speed  â˘âŻHigh  concurrency  â˘âŻWorkload  mgt  â˘âŻBroad  BI  support  â˘âŻFor  unstructured  &  semi-Ââstructured  data  â˘âŻFor  business  users  â˘âŻLow  (1  second)  latency  â˘âŻWindows  (collec/ons)  of  events  Š2014  Cloudera,  Inc.  All  rights  reserved. Â
- 6. 6  Opera/onal  Data  Store  â˘âŻ Consolidate,  cleanse  &  stage  data  â˘âŻ Promote  to  other  opera/onal  systems  or  EDWâs  Data  Warehouse  â˘âŻ ELT  â˘âŻ Archive  Ra/onalizing  exis/ng  infrastructure  Migra/ng  data  sets,  workloads  or  en/re  systems  from  more  expensive  or  less  ďŹexible  systems  Š2014  Cloudera,  Inc.  All  rights  reserved. Â
- 7. 7  Combine  &  explore  new   data  sets  â˘âŻScrip/ng  â˘âŻData  blending  â˘âŻTradi/onal  ETL  Support  ad-Ââhoc  marts  and  self-Ââ serve  BI  users  â˘âŻTableau,  Qlik  et  al  Enable  data  scien/sts  to  train  &  test  models  â˘âŻML  libraries  â˘âŻSAS,  Revolu/on  What  do  we  mean  by  data  discovery?  Providing  a  ďŹexible  analy/c  sandbox  where  users  can  apply  mul/ple  tools  &  techniques  to  derive  insights  from  new  &  tradi/onal  data  Š2014  Cloudera,  Inc.  All  rights  reserved. Â
- 8. 8  Analyze  paeerns  over  deep  histories  â˘âŻRecommenda/ons  â˘âŻOutliers  Automate  responses  to  new  data  /  observa/ons  â˘âŻClassifying  or  scoring  new  data  User  explora/on  /  judgment  applica/on  â˘âŻReviewing  outliers  â˘âŻOverriding  sugges/ons  What  do  we  mean  by  pervasive  analy/cs?  Using  predic/ve  analy/cs  to  improve  business  processes  or  augment  professional  judgment  in  an  automated  way  across  the  organiza/on  Š2014  Cloudera,  Inc.  All  rights  reserved. Â
- 9. 9  Big  Data  in  Credit  Card  Processing  âCustomer  privacy  is  paramount,  but  we  need  to  keep  vast  amounts  of  informaFon  online  to  run  our  business.  Can  we  achieve  both  goals?â  âModern  credit  card  fraud  rings  operate  globally  over  long  Fme  scales  â  how  can  we  collect,  store  &  analyze  the  petabytes  of  data  it  takes  to  detect  them?â  âWe  obviously  have  vast  and  detailed  informaFon  about  customer  purchases.  Can  we  combine  it  with  GPS  &  mobile  data,  combined  with  browsing  behavior  to  oďŹer  new  products?â  âHow  can  we  deliver  what  the  business  team  wants,  and  faster,  without  spending  tens  of  millions  of  dollars  to  expand  our  data  warehouse?â  Fraud  DetecEon  Regulatory   Compliance  Product  &  Service   InnovaEon  OperaEonal   EďŹciency  CFO  &  CRO  CIO  &  CRO  R&D,  CMO  CIO Â
- 10. 10  Big  Data  in  Retail  360°  Customer  View  Fraud  PrevenEon  LogisEcs  &   Supply  Chain  OperaEonal  EďŹciency  CMO  CMO  &   Customer  Service  CEO,  VP  OperaEons  CIO  âWe  want  to  know  what  our  customer  do  on-Ââline  and  in  our  stored.  How  can  we  combine  data  from  separate  analyFcs  silos  to  understand  &  serve  them  beSer?â  âTheT,  or  âshrinkageâ  in  our  stores  is  on  the  increase  â  can  we  combine  POS  data  with  video  surveillance  to  reduce  it  without  impacFng  customer  service  negaFvely?â  âHow  can  we  reduce  stock-Ââ outs  &  ensure  products  are  in  the  right  stores  at  the  right  Fme?  Can  we  combine  data  from  our  carriers  with  in-Ââ store  historical  data  from  thousands  of  stores?  âOur  EDW  infrastructure  is  being  overwhelmed  with  data  and  workloads;  we  are  running  into  capacity  limits,  and  the  annual  costs  of  expansion  are  in  the  tens  of  millions.  What  can  we  do?â Â
- 11. 11  Big  Data  in  Health  Care  360°  PaEent  View  Regulatory  Compliance  Maximize  Medical  EďŹcacy  OperaEonal  EďŹciency  VP  OperaEons,   Chief  of  Compliance  VP  OperaEons  Chief  Medical  OďŹcer  CFO  Chief  Medical  OďŹcer  CIO  âPaFent  data  ends  up  scaSered  across  many  diďŹerent  systems  â  is  there  a  way  to  get  a  complete  picture  by  combining  it  while  ensuring  HIPAA  compliance?â  âThe  move  to  EMR  combined  with  the  strict  regulaFons  means  we  need  to  keep  at  least  7  years  of  data  online  â  how  can  we  aďŹord  to  do  that  and  make  it  searchable  and  available  for  analysis?â  âWe  invest  hundreds  of  millions  in  new  equipment  every  year.  How  can  we  judge  the  long  term  eďŹcacy  for  paFent  outcomes,  and  make  smarter  investment  decisions?â  âOur  EDW  infrastructure  is  being  overwhelmed  with  data  and  workloads;  we  are  running  into  capacity  limits,  and  the  annual  costs  of  expansion  are  in  the  tens  of  millions.  What  can  we  do?â Â
- 12. 12
- 13. 13
- 14. 14  Š2014  Cloudera,  Inc.  All  rights  reserved.  Mike  Olson  @mikeolson  mike.olson@cloudera.com Â