ccna course 2
- 1. Cloud Computing & NW Traffic Dr.S.Sridhar, Ph.D.(JNUD), RACI(Paris, NICE), RMR(USA), RZFM(Germany) Dean – Cognitive & Central Computing Facility R.V. COLLEGE OF ENGINEERING BANGALORE
- 3. The iSchool University of Maryland What is Cloud Computing? 1. Web-scale problems 2. Large data centers 3. Different models of computing 4. Highly-interactive Web applications
- 4. The iSchool University of Maryland 1. Web-Scale Problems Characteristics: Definitely data-intensive May also be processing intensive Examples: Crawling, indexing, searching, mining the Web “Post-genomics” life sciences research Other scientific data (physics, astronomers, etc.) Sensor networks Web 2.0 applications …
- 5. The iSchool University of Maryland How much data traffic? Wayback Machine has 20 TB/month (2006) Google processes 60TB /Week (2015) 640K ought to be enough for anybody.
- 8. The iSchool University of Maryland There’s nothing like more data! s/inspiration/data/g;
- 9. The iSchool University of Maryland What to do with more data? Answering factoid questions Pattern matching on the Web Works amazingly well Learning relations Start with seed instances Search for patterns on the Web Using patterns to find more instances Who shot Abraham Lincoln? → X shot Abraham Lincoln Birthday-of(Mozart, 1756) Birthday-of(Einstein, 1879) Wolfgang Amadeus Mozart (1756 - 1791) Einstein was born in 1879 PERSON (DATE – PERSON was born in DATE
- 10. The iSchool University of Maryland 2. Large Data Centers Web-scale problems? Throw more machines at it! Clear trend: centralization of computing resources in large data centers Necessary ingredients: fiber, juice, and space What do Oregon, Iceland, and abandoned mines have in common? Important Issues: Redundancy Efficiency Utilization Management
- 13. The iSchool University of Maryland Key Technology: Virtualization Hardware Operating System App App App Traditional Stack Hardware OS App App App Hypervisor OS OS Virtualized Stack
- 14. The iSchool University of Maryland 3. Different Computing Models Utility computing Why buy machines when you can rent cycles? Examples: Amazon’s EC2, GoGrid, AppNexus Platform as a Service (PaaS) Give me nice API and take care of the implementation Example: Google App Engine Software as a Service (SaaS) Just run it for me! Example: Gmail “Why do it yourself if you can pay someone to do it for you?”
- 15. The iSchool University of Maryland 4. Web Applications A mistake on top of a hack built on sand held together by duct tape? What is the nature of software applications? From the desktop to the browser SaaS == Web-based applications Examples: Google Maps, Facebook How do we deliver highly-interactive Web-based applications? AJAX (asynchronous JavaScript and XML) For better, or for worse…
- 21. The iSchool University of Maryland Web-Scale Problems? Don’t hold your breath: Biocomputing Nanocomputing Quantum computing … It all boils down to… Divide-and-conquer Throwing more hardware at the problem Simple to understand… a lifetime to master…
- 22. The iSchool University of Maryland Divide and Conquer “Work” w1 w2 w3 r1 r2 r3 “Result” “worker” “worker” “worker” Partition Combine
- 23. The iSchool University of Maryland Different Workers Different threads in the same core Different cores in the same CPU Different CPUs in a multi-processor system Different machines in a distributed system
- 24. The iSchool University of Maryland Choices, Choices, Choices Commodity vs. “exotic” hardware Number of machines vs. processor vs. cores Bandwidth of memory vs. disk vs. network Different programming models
- 25. The iSchool University of Maryland Flynn’s Taxonomy Instructions Single (SI) Multiple (MI) Data Multiple(M SISD Single-threaded process MISD Pipeline architecture SIMD Vector Processing MIMD Multi-threaded Programming Single(SD)
- 26. The iSchool University of Maryland SISD D D D D D D D Processor Instructions
- 27. The iSchool University of Maryland SIMD D0 Processor Instructions D0D0 D0 D0 D0 D1 D2 D3 D4 … Dn D1 D2 D3 D4 … Dn D1 D2 D3 D4 … Dn D1 D2 D3 D4 … Dn D1 D2 D3 D4 … Dn D1 D2 D3 D4 … Dn D1 D2 D3 D4 … Dn D0
- 28. The iSchool University of Maryland MIMD D D D D D D D Processor Instructions D D D D D D D Processor Instructions
- 29. The iSchool University of Maryland Memory Typology: Shared Memory Processor Processor Processor Processor
- 30. The iSchool University of Maryland Memory Typology: Distributed MemoryProcessor MemoryProcessor MemoryProcessor MemoryProcessor Network
- 31. The iSchool University of Maryland Memory Typology: Hybrid Memory Processor Network Processor Memory Processor Processor Memory Processor Processor Memory Processor Processor
- 32. The iSchool University of Maryland Parallelization Problems How do we assign work units to workers? What if we have more work units than workers? What if workers need to share partial results? How do we aggregate partial results? How do we know all the workers have finished? What if workers die? What is the common theme of all of these problems?
- 33. The iSchool University of Maryland General Theme? Parallelization problems arise from: Communication between workers Access to shared resources (e.g., data) Thus, we need a synchronization system! This is tricky: Finding bugs is hard Solving bugs is even harder
- 34. The iSchool University of Maryland Managing Multiple Workers Difficult because (Often) don’t know the order in which workers run (Often) don’t know where the workers are running (Often) don’t know when workers interrupt each other Thus, we need: Semaphores (lock, unlock) Conditional variables (wait, notify, broadcast) Barriers Still, lots of problems: Deadlock, livelock, race conditions, ... Moral of the story: be careful! Even trickier if the workers are on different machines
- 35. The iSchool University of Maryland Patterns for Parallelism Parallel computing has been around for decades Here are some “design patterns” …
- 36. The iSchool University of Maryland Master/Slaves slaves master
- 37. The iSchool University of Maryland Producer/Consumer Flow CP P P C C CP P P C C
- 38. The iSchool University of Maryland Work Queues CP P P C C shared queue W W W W W
- 39. The iSchool University of Maryland Conclusions For all these traffic , Network access has to be smart NW traffic has to be controlled Effective security is needed A lot of research scope is there in the domains of NW access, security algorithms This area is everlasting !