2020 Big Data & Analytics Maturity Survey Results

Editor's Notes

• #5: We collected data from over 150 data practitioners for this survey. The respondents are working across industries and are located around the world. 49% identified as data & analytics professionals while the other 51% were in business intelligence (BI) or IT teams or were data consumers.
• #6: Most survey respondents have data on multiple on-premise platforms. The most common sources are Open Source (Hadoop), Oracle and SQL Server. A recent CRN article asserts that “With the bulk of information technology spending still on-premises, enterprises’ modernization of their core applications will continue to transform business technology and the industry this year.” Our respondents have 33% of their data on Open Source (Hadoop), 28% on SQL Server, 18% on Oracle and 8% on Teradata.
• #7: 53% of survey respondents indicated that they plan to keep their investment in Open Source (Hadoop) the same. While 30% plan to invest more.
• #8: Respondents were more evenly split on their plans to invest in their existing on-premise data platforms other than Open Source (Hadoop) in 2020. 42% plan to say the same. 33% plan to invest more. And 25% plan to invest less.
• #9: 61% of respondents are operating data platforms in the public cloud. This is in line with Forrester’s research that shows that currently, 65% of North American enterprises rely on public cloud platforms.
• #10: When asked to check off which data platforms they were working with, respondents showed that they are using multiple data platforms in the public cloud from AWS to Open Source to Teradata Cloud.
• #11: Of the 38% of respondents who indicated that they are not on the cloud at all, 48% plan to deploy a public cloud data warehouse/platform in the future, 16% said that thy plan to stay off the cloud, and 36% are still evaluating their options.
• #12: When asked to rank how your organization’s maturity with running a data platform in the cloud, 23% are deployed in the cloud and say that it’s working well. 29% are running a data platform in the cloud and are still working out wrinkles while 3% say that it’s not working well. The remainder of respondents said they were either not on the cloud or want to get there but haven’t started planning yet.
• #13: 41% of survey respondents are either deployed or in the process of deploying on the public cloud. 21% will be deployed on the public cloud in less than a year.
• #14: We found it surprising that only 24% of those surveyed are all in with a single cloud vendor. The majority of respondents are working with multiple public cloud vendors and have a hybrid cloud strategy in place.
• #15: When respondents were asked what they were trying to achieve in the public cloud, the top responses were: flexibility to scale up and down, get better data and analytics, and lower costs. Close behind that was the ability to access new technologies and deploy new applications faster.
• #16: Our respondents told us that they are looking to deploy data science, data warehousing, and business intelligence in the cloud. And how are they going to do it? By streaming real-time data and using ETL.
• #17: Most of our respondents are storing data on a data lake in the public cloud or in a cloud data warehouse.
• #18: Our survey shows that 49 respondents are using data virtualization now and 57 respondents plan to use data virtualization in the future. Data virtualization technology has been around for quite some time. However, the explosion in data size and variety, coupled with the increased focus on analytical use cases, has created new challenges for legacy data virtualization technologies. The need for ad hoc access to both live and historical data has steadily increased among business users as they leverage artificial intelligence (AI) and machine learning (ML) platforms for analytics.
• #19: When asked about the challenges that respondents are experiencing with the public cloud, 32 people indicated that security was a challenge. Given that 79% of respondents indicated that having consistent, integrated security and governance for your data in public cloud, private cloud and/or hybrid cloud is very important, this is not a surprise. It’s also not as surprising to see that 29 people said that the cost of the public cloud is higher than expected.
• #20: When asked to select all areas that apply to challenges that you’re experiencing with your analytics infrastructure, governance was the number one challenge followed by skills, performance, costs and security.
• #21: Microsoft applications - Excel and PowerBI - are some of the most popular business intelligence tools today along with Tableau. Looking towards the future, Excel will become less of a “go to” business intelligence tool as enterprises look to using more PowerBI and Tableau. When it comes to artificial intelligence (AI) and machine learning (ML), Spark (Open Source) is the most used technology with respondents.
• #22: In the future, Spark remains the technology of choice, along with Databricks and Cloudera Data Science Workbench. When respondents chose “other” the most frequent responses were: Pecan, Azure, and “don’t know yet.”
• #23: So what is the preferred data platform? 39% said that they’d like analytics and machine learning from multiple vendors that are tied together.
• #24: Most respondents 79% indicated that having consistent, integrated security and governance for your data in public cloud, private cloud and/or hybrid cloud is very important.
• #25: 73% indicated that having consistent, integrated security and governance across all of your analytics and machine learning functions is very important.
• #26: Security and data governance are extremely important as enterprise customers store data on data lakes and in the public cloud and as they perform analytics and artificial intelligence on their data. And, more and more data users are looking to data virtualization in order to get live connections to data from multiple sources so that they can make better data-driven decisions for their business. Thank you again for joining our webinar today. For more information, download the 2020 Big Data & Analytics Maturity Survey Results at www.atscale.com