Big Data Cloud June 3rd Meetup - Presentation by Mark Davis
Download as PPTX, PDF2 likes543 views
The document discusses the challenges and solutions in managing big data, specifically focusing on the need for advanced analytics and search tools due to the overwhelming amount of unstructured data. It highlights the use of emerging technologies like Hadoop, machine learning, and various database systems to bridge the gap between structured and unstructured data. The presentation also emphasizes the importance of integrating search and analytics to improve efficiency in data processing and decision-making.
Big Data Cloud June 3rd Meetup - Presentation by Mark Davis
- 1. Big Data Cloud MeetupBig Data & Cloud Computing - Help, Educate & Demystify.June 3rd 2011
- 2. Kitenga, Mark Davis CTOJune 3rd 2011 MeetupUnlocking Big Data through Analytics and Search
- 3. Big DataEnormous transactional dataEnormous unstructured informationToo big for databasesNew tools are needed
- 4. Unstructured data explosionMultimedia ContentTextImageryAudioVideoSensor StreamsBiometric data3DTextEmailDocumentsWeb pagesTweetsPosts<5%Structured Enterprise DataDatawarehouseCDRsFinancial recordsAccess logs4
- 5. Big Data Trillions of user interactions/transactions == Big Data>100M<10M<1MOpen sourceMySQLPHPData warehousingParallel SQLBig hardwareNoSQLHadoop/MapReduceHbase/HIVEEmerging technologies Traditional (DBMS-based) solutions 5
- 6. The Structured/Unstructured ChasmSQLRDBMSTransactional DataBI ToolsSearchDocumentsText ClassificationTaxonomiesOntologies
- 7. Unstructured Analytics: Surfacing Metadata
- 8. Information ExtractionMachine-LearningFinite State TransducerFinite State TransducerFinite State TransducerParts-of-Speech TaggingLemmatizationTokenization
- 9. Search + AnalyticsResource IntegrationFacet BrowsingFacet ChartingAutosuggestSpellcheckQuery LanguageIndexingMetadata Extraction
- 10. Defense IntelligenceAnalyst support staff needs to convert raw data into actionable intelligence10Named Entity ExtractionImage taggingVideo analyticsLinkage AnalysisNetwork VisualizationSearchImprove Force EffectivenessHadoop/MapReduce, GPUs, HDFS, Hbase, SOLRSituation ReportsGeo-tagged ImageryUS Army NavyDHSNSA
- 11. CASE STUDY: US ARMY11The Solution>200 data feeds<0.5s queriesFast analysis cyclesMachine LearningAnalyticsBiometricsLinkage AnalysisFace recognitionVideo taggingCollaborative systemsAnalysis Bottlenecks200 data feedsUnacceptable response timeAnalysts avoid complete searchesBasic entity extractionSlow analysis cyclesDistribution by PowerPointEnabling techonolgies: GPU clouds, Hadoop/MapReduce, Katta, Lucene, NoSQL, HbaseEnabling Technologies: Oracle and custom thick clients
- 12. Pharma BioinformaticsIncrease speed of drug discovery12Biological Named Entity ExtractionAuthor Name Extraction and NormalizationLinkage AnalysisTimelinesFacetted SearchZettaVoxFaster DiscoveryHadoop/MapReduce, HDFS, Hbase, GPUs, SOLRPatentsGenetic Sequence DataJournal Articles
- 13. PharmaTreemap13
- 14. 14
- 16. Demo
- 18. SummaryBig Data spans unstructured and structured dataEffective tools for managing both involve understanding the differences and similarities of bothBridging the chasm between them means merging search and analytics together
- 19. Questions?
- 20. Contact Info20mark@kitenga.comhttp://www.kitenga.comKitenga, Inc.2953 Bunker Hill Lane, Suite 400Santa Clara, CA 950541-(408)-462-KITE1-(253)-541-6799 (FAX)