Emerging trends in software development: The next generation of storage
Download as PPTX, PDF4 likes2,720 views
The document discusses emerging trends in software development, particularly focusing on the shift to cloud computing and its implications for storage solutions. Key topics include the adoption of Docker and microservices, the influence of DevOps on software delivery, and the evolving landscape of data storage options. It emphasizes the demand for agility in storage and the transformative impact of technologies such as Spark and continuous delivery on development practices.
Emerging trends in software development: The next generation of storage
- 1. Emerging trends in software development: The next generation of storage Donnie Berkholz, Ph.D. Research Director — Development, DevOps & IT Ops SNIA Storage Developer Conference, Sept 2015
- 3. App-dev workloads are quickly moving to the cloud 3 Current Future 59.0% 7.9% 14.2% 5.2% 6.6% 7.1% On-Premises, Non-Cloud Off-Premises, Non-Cloud On-premises Private Cloud Hosted Private Cloud Infrastructure-as-a-Service (IaaS)/Public Cloud Software-as-a-Service (SaaS) n = 146 37.3% 6.1% 19.1% 9.9% 15.2% 12.4% n = 134 Source: 451 VotE Cloud, Q2 2015
- 4. CLOUD COMPUTING Q1 2015 Q. Estimate the percentage of your organization’s data currently stored in off premises cloud environments? Q. How many terabytes of data does your company currently have under management across all environments? *Terabytes of Data Stored in Off-Premises Cloud" is calculated from an organization's total data and the percent they store in off-premises cloud environments. Data Storage - Cloud Respondents 4 20.0% 7.5% 7.3% 6.3% 42.8% 16.2% > 1000TB 750TB-1000TB 500TB-749TB 250TB-499TB 10TB-249TB <10TB Percent of Sample n =877 Percent of Organization’s Data Stored in Off-Premises Cloud Total Data at Organization Source: 451 Research, Voice of the Enterprise: Cloud Computing Q1 2015 3.8% 1.4% 2.5% 3.1% 42.0% 47.1% > 1000TB 750TB-1000TB 500TB-749TB 250TB-499TB 10TB-249TB <10TB Percent of Sample n =766 Terabytes of Data Stored in Off-Premises Cloud * 7.8% 6.4% 6.9% 14.6% 64.3% > 80% 60%-80% 40%-59% 20%-39% < 20% Percent of Sample n =896
- 5. The new stack? 5 An infinite array of possible stacks.
- 7. What do enterprises think of storage vendors? 7 Source: 451 Storage Wave 19, June 2015; n=250ç
- 8. Development trends that will transform storage use 8
- 9. 9 Docker Docker Docker Source: 451 Research, “Now Shipping: The Docker and containers ecosystem rapidly takes shape”
- 10. Developers love Docker Donnie Berkholz Source: http://www.slideshare.net/dberkholz/is-your-community-healthy-metrics-on-the-top-cm-software 10 *
- 11. But it’s not just a toy 11 21%} Source: 451 VotE Cloud, Q1 2015; n=991 3.1% 19.8% 56.1% 10.7% 3.9% 4.2% 2.1% Unfamiliar No Plans Discovery and Evaluation Running Trials/Pilot Projects Used for Test and Development Environment Initial Implementation of Production Applications Broad Implementation of Production Applications
- 13. Loosely coupled teams “ One of the biggest changes is that we no longer have an official ‘architecture’ team. Instead, we have made ‘architecture’ an ‘ingredient’ on each of our teams.” 13 http://tech.gilt.com/post/102628539834/making-architecture-work-in-microservice – Lauri Apple, Gilt Groupe, 14 Nov 2014
- 14. The foundation of microservices 14
- 15. 15 Disruption in data processing
- 16. 16 Spark came out of nowhere Donnie Berkholz Source: http://redmonk.com/dberkholz/2015/03/13/the-emergence-of-spark/
- 17. 17 On Hacker News, too Donnie Berkholz Source: http://redmonk.com/dberkholz/2015/03/13/the-emergence-of-spark/
- 18. The market is coming 18 Source: 451 Research Total Data: Platforms & Analytics Market Monitor, May 2015
- 20. Concurrency for developers 20 Donnie Berkholz Source: http://redmonk.com/dberkholz/2014/03/18/go-the-emerging-language-of-cloud-infrastructure/
- 21. Development trends that will transform storage vendors 21
- 22. 22 What in the world is DevOps, anyhow?
- 23. 23 Agile, truly tip to tail Business to customer
- 24. 3 pillars of DevOps 24 Culture Automation Measurement
- 25. 25 Culture: Tear down all the silos Flickr: kalandrakas
- 26. 26 Flickr: respresFlickr: hartvig, snapeverything, roymaloon Automation: Pets vs Cattle
- 27. Automation: Infrastructure as code 27
- 28. Knight Capital and the $460 million bug 28 Wikipedia: Jericho Release management – quarterly to daily Gary Gruber, HP
- 31. DevOps is reaching DBAs 451 DevOps study, Q3 2014 31
- 32. 32
- 33. 33 Adopting open-source technologies Blogspot: ThatsABigIf
- 34. The changing role of standards 34
- 35. The changing role of benchmarks 35
- 36. Wrapping up • Docker and microservices will exacerbate hypervisor-style use & increase demands on the network • Spark will drive requirements for memory and memory-like storage • DevOps & continuous delivery will transform how you deliver storage software • The interplay between open source, standards and benchmarks continues to tilt toward agility 36
- 37. Global Digital Infrastructure Alliance — 20,000 strong Share your thoughts, get our data 451alliance.com 37
- 38. Storage needs to cope with agility. Donnie Berkholz @dberkholz donnie.berkholz@451research.com
- 39. Some images from this presentation are Creative-Commons licensed. https://creativecommons.org/licenses/by-sa/ https://creativecommons.org/licenses/by/ 39
Editor's Notes
- #3: Cathedral, indulgences to bazaar Open source, cloud, DigitalOcean
- #4: Future is two years out App dev is 7.8% of workloads, ranked #5 (highest is email/collab at 15.9%) … What does this mean for the data behind the apps?
- #5: Data gravity. Still mostly on-prem. … that’s what we’re building, now how are we building it?
- #6: Languages, databases, frameworks … What’s driving the way we build technology?
- #7: The new mode for access & transfer of data has become APIs Most storage file/block. This is more likely object-driven … why should you and your companies care?
- #8: N not large enough for everything Exciting vendors: Pure and EMC
- #10: Hypervisors already had randomized I/O, now exacerbated. Flash optimal, tough for SMR Flocker BoF
- #11: Explain GitHub, Stars/PRs/Forks
- #13: Active-active, cross-DC/cross-region (DynamoDB outage) Distributed datastores Most storage backup/replication is array-/hypervisor-level, not app-level Loosely coupled Business-defined separations. Bounded context based on cross-organizational empathy. Steve Yegge memo — Amazon must be SOA, or you’re fired.
- #14: DevOps + microservices Bounded contexts, empathy defined DevOps is how you build and run microservices.
- #15: Also note PaaS providers moving to containers
- #16: Spark. What is Spark? In-memory compute, can cache to disk. IoT In-memory compute Real-time analytics (streaming), graph, ML (repeated computations in memory) NoSQL at scale – distributed databases (Cassandra)
- #17: Spark. What is Spark? In-memory compute, can cache to disk. IoT In-memory compute Real-time analytics (streaming) NoSQL at scale – distributed databases (Cassandra)
- #18: Spark, IoT In-memory compute Real-time analytics (streaming) NoSQL at scale – distributed databases (Cassandra)
- #19: Hadoop market 26 vendors included 42% CAGR above $100m rev 50% below $100m rev … A couple of quick hits before we get into how you change
- #20: Do app devs need to care? Storage round trips much cheaper, how does this change things? Suitable data structures – parallels to Redis/Memcached? IoT (equivalent of moving compute to data as SCM reaches CPU)
- #21: Their view: onload/offload and RDMA Mine: functional programming, JVMs, Go, distributed systems, Spark, etc.
- #24: OODA loops
- #26: How can SP do this? Marketing, PS, dev-focused services, SDKs etc
- #28: Where storage and networking fit in. APIs, vanilla (SDS, Cumulus, etc)
- #30: Release management – quarters to weeks to days to hours Gary Gruver, HP
- #31: Community, UX Keep using cattle metaphor Storage – metrics output, APIs
- #34: Seen a lot of open source on the agenda here What are your criteria?
- #35: “s3 api” vs “Storage api” Agile. Ad hoc. Agile foundations
- #36: Jepsen Session from Mai Zheng from NMSU Open-source benchmarks