Big data workshop october 18
4 likes480 views
IoT related revenue is expected to grow significantly from $130 billion in 2015 to $883 billion by 2022, representing a compound annual growth rate of 32.4% according to a market analysis. Future trends in big data and analytics include generating data from machines and sensors, processing vast amounts of structured and unstructured data from various sources using techniques like machine learning, and applying insights to improve business outcomes. Open source platforms are commonly used to ingest, store, analyze and act on big data in solutions that address tasks such as predictive maintenance, fraud detection, and customer experience optimization.
Big data workshop october 18
- 1. ANALYTICS, IOT & MOBILE @mohammadzaman
- 2. US$883.55B IoT related revenue is expected to grow from USD 130.33 Billion in 2015 to USD 883.55 Billion by 2022, at a CAGR of 32.4% between 2016 and 2022 Source: http://www.postscapes.com/internet-of-things-market-size/
- 4. LIFECYCLE OF IT.. Problem Statement Problem Statement Design Thinking Design Thinking Business Model Workshop Business Model Workshop Eco-System Development Eco-System Development Scalability of Innovation Scalability of Innovation Define the problem in a single sentence using everyday words for absolute general audience Problem Statement Put yourself in the shoes of your users.. All users. Design Thinking How would you sustain the Innovation? What is the revenue model? Business Model Workshop We can not do any BIG thing alone! So, a very collaborative Eco-System is a must! Eco-System Development Once we Innovate, then we need to scale the innovation with sustainable progress Progressive Innovation
- 5. It is a structured process of Innovation! Adding the touch of Practicality! Viability to be a BUSINESS Desirability By PEOPLE Feasibility with TECHNOLOGY Process Innovation Functional Innovation Emotional Innovation
- 6. EDUCATION... Institutional Education • Intellectual Readiness Institutional Education • Intellectual Readiness Professional Education • Technical Readiness Professional Education • Technical Readiness Forward Education • Trend Readiness Forward Education • Trend Readiness Practical Education • Leadership Readiness Practical Education • Leadership Readiness This is scoring for Institutional Knowledge (course curriculum, instructors & student peers); score yourself by comparing the rank of the institution & rank of the program Intellectual Readiness Have you learned something beyond your curriculum? Internship, competition? Local, Global? Did you take some certification training from the vendors? Contributed? Technical Readiness Forward education is fundamental for Innovation.The question is, who’s done it already? How would you know? How often you read a new book? A Journal? A blog/slideshare/vlog? How many and how often you blog/tweet/post/respond? Trend Readiness Technical competencies are sometime barricade for leadership, as it tends on micro-controls.What is your influential power? Is this progressing? Managing on eXperience of Customers, Employees, Industry, Leaders and beyond?Your Style?? Leadership Readiness
- 7. APTITUDE... AttitudeAttitude AdaptableAdaptable Self Driven Self Driven DisciplineDiscipline DependableDependable CollaborativeCollaborative What is your attitude – Always positive? Is everyone extremely excited to work with you? Are you the person everyone wants you in their team? Attitude How often you adapt yourself to the environment? Change technology? Learn something new and apply? Adaptable Does someone has to tell you what you need to do? If something looks odd, do you take action? (Pick up the banana peel from the pathway?) Self Driven How disciplined are you? How do you manage your time? How many things are you engaged in? Can you get things done ontime? Do you always meet expectations? Discipline How dependable are you? Can I give you something and forget all about it? Does is only apply for your superior’s orders? Do you do this for everyone? By yourself? Dependable What is your communication style? Work Alone? Work with team? Delegate? Influence? Collaborative
- 8. Think out-of-the-box Share the idea // Move Fast & Break Things // // Think Different // // Don’t be Evil // My Z Score Score Sheet - Aptitute Score Sheet - Education
- 9. Yesterday:Visualization Today: Executable Action Tomorrow: Prediction 40 ZB by 2020 85% Usage Data LBS Sentiment Click Stream Social Data 2.8 ZB by 2012 15x Machine Data Geo, Logs, Sensor Other New Sources..
- 10. Collect or Generate the Data – Machine to Machine, User Generated, System Generated Ingest the Data – Stream, Batch,Variation with Velocity.. (Scribe, Kinesis, Kafka, Flume) Store the Data – Structured (SQL/NoSQL, DW, Search/Cache) or Unstructured (Hadoop/HDFS or QueryLess Storage) Analyze the Data – Process, Classify, Predict, etc. Batch processing (Hadoop, Spark, Redshift) or Stream processing (Spark Stream, Storm) Act on the Data – Execute,Visualize, Simulate.. NB – Need to have a stable and updated Data Set (Specially for Bangladesh) INGEST STORE ANALYZEGenerate ACT
- 12. OPEN SOURCE DISTRIBUTION Ingest Logs Receive Events Train Models Score Models Sync Results Search Data Big Data Components • Kafka • Storm • Flume to HBase, ES • Cassandra • Kafka • Storm • Flume to ES • HBase read • Spark • Kafka • HBase write • Spark • Elasticsearch write (index) • Spark • HBase read • Elasticsearch read (query)
- 13. Hypervisor/ CMP BIG DATA SOLUTION COMPONENTS Advanced Analytics / ML Reporting/Dashboard/ Visualization Analytics Data Integration and Processing SQL / DW Cluster Manager / Admin Storage Service Compute/ Storage HW No SQL / Time Series PivotalPivotal ECS, Isilon,xTremIO Open Stack YARN/ HDFS Spring XD Gemfire Pivotal HDB / Greenplum HAWQ Graphlab MadLib HortonworksHortonworks Super-Micro DAS Open stack YARN/ HDFS Spark, Flume, Sqoop, Kafka HBASE Hive Hive, Spark Zeppelin Spark ML, Spark R SAP HANASAP HANA Cisco/ VMAX/VNX VMware AppDirector / HANA SAP Data Services HANA BW HANA Business Objects, Lumira PAL SparkSpark Super-Micro DAS OpenStack/ xStream YARN Spark Streaming HBASE Spark SQL Hive Storm Zeppelin Spark ML SAP HANA VoraSAP HANA Vora Cisco/SM, DAS/ VMAX VMware / OpenStack YARN/ HDFS Spark Streaming, Sqoop, DS HANA Spark SQL/BW Vora/Spark Business Objects, Lumira PAL, Spark
- 15. Video Surveillance Activity Monitoring Traffic Jam & Diversion Facial Recognition Terrorist Recognition Criminal Monitoring Preventive Maintenance Automobile Proactive Maintenance Customer Experience 360 Degree Customer View Collaborative Fraud Detection Transaction Analysis Data Anomaly – Cyber Security
- 16. Can you see the end of the road? Your imagination is the JOURNEY will take you THERE