Managing Information Storage: Trends, Challenges, and Options (2013-2014) (Whitepaper)
- 1. MANAGING STORAGE: TRENDS, CHALLENGES, AND OPTIONS (2013-2014) Includes impact of virtualization and cloud computing How are IT and storage managers coping with the organizational challenges posed by the explosion of data, increasing criticality of digitized information, and rapid introduction of new information infrastructure technologies, virtualization, and cloud? This updated paper contains the findings of a study based on input from over 1,000 storage professionals and managers across 800+ organizations worldwide. This research will assist IT/storage managers in comparing and correlating their environment and plans with the overall trends in the industry and the impact of emerging technologies such as storage virtualization and cloud computing. EMC PERSPECTIVE
- 2. TABLE OF CONTENTS EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 CHALLENGES FACED BY IT/STORAGE MANAGERS . . . . . . . . . . . . . 9 IT MEGATREND: DATA MOVEMENT TO VIRTUALIZED AND CLOUD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 RATE OF IT TRANSFORMATION . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 DATA MOVEMENT BY COMPANY SIZE . . . . . . . . . . . . . . . . . . . . . . 12 COMPLEX STORAGE ENVIRONMENTS . . . . . . . . . . . . . . . . . . . . . . 12 STORAGE TECHNOLOGY SEGMENTS . . . . . . . . . . . . . . . . . . . . . . . 13 ORGANIZATION IMPLICATIONS OF DATA MOVEMENT TO VIRTUALIZED AND CLOUD ENVIRONMENTS. . . . . . . . . . . . . . . . . 15 FORMALIZED STORAGE GROUPS . . . . . . . . . . . . . . . . . . . . . . . . . . 16 RESPONSIBILITIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 STORAGE GROUP SKILLS AND PERFORMANCE . . . . . . . . . . . . . . . . . . . . . 18 MANAGERS’ SKILLS FOR CLOUD COMPUTING AND BIG DATA ANALYTICS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 SOURCES FOR HIRING AND DEVELOPMENT . . . . . . . . . . . . . . . . . . . . . . . . 20 RECOMMENDATIONS AND CONCLUSIONS . . . . . . . . . . . . . . . . . . 22 EMC RESPONSE AND INITIATIVES . . . . . . . . . . . . . . . . . . . . . . . . 23
- 3. EXECUTIVE SUMMARY The explosion of data, its criticality, and the increasing dependency of businesses on digital information are leading to larger and more complex information storage environments that are increasingly challenging to manage. Poorly designed or managed storage infrastructures put the entire business at risk in case of a catastrophic failure. At the same time, the data available to businesses represents an enormous asset that can contribute to new, valuable business opportunities via predictive analytics and other Big Data techniques. A robust storage infrastructure requires highly reliable equipment as well as a strong team of experts to manage it efficiently. This paper focuses on storage infrastructure and presents findings from a global survey of more than 1,000 IT professionals across 800+ organizations. In addition, the study tries to capture indicators and data points to understand the trend and rate of transformation with: • New and emerging storage technologies • Virtualization and cloud • Big Data and Big Data analytics This study will help IT managers benchmark their plans to align with the overall trends in the industry. The first such study was published in 2006 and has since been updated annually. A large number of IT/storage managers have used the information to refine their planning and decision-making. KEY CHALLENGES IT/storage managers and storage professionals across companies of all sizes face the following mission-critical challenges: • Managing storage growth • Designing, deploying and managing backup, recovery, and archive solutions • Making informed strategic/big-picture decisions • Designing, deploying, and managing disaster recovery solutions • Designing, deploying, and managing storage in a virtualized server environment • Addressing the lack of skilled storage professionals • Designing, deploying, and managing storage in cloud computing environments • Addressing the lack of skilled cloud technology professionals • Convincing higher management to adopt cloud • Providing infrastructure for Big Data analytics • Managing external cloud service providers Explosive growth in storage requirements and a widening storage technology knowledge gap across the industry are making all of the above mission-critical tasks even more challenging. 3
- 4. IMPACT OF CLOUD COMPUTING AND IT AS A SERVICE The findings of this study clearly indicate a rapid adoption of virtualization and cloud computing technologies. Server/storage consolidation and virtualization has been a focus for IT organizations for a number of years. The trends show that storage deployed in virtualized environments has exceeded storage in traditional/classic (not-virtualized) in late 2012. Projecting forward, we can expect to see more data and storage in private cloud than classic by mid-2016. Currently, 42 percent of the storage capacities are in traditional/classic IT environments. A large percentage of this allocation (31 percent) is likely to move to either virtualized server or cloud environments in the next 24 months. Only a small percentage of the managers (organizations) believe that they have at least half of their teams capable of working with virtualization (35 percent) and cloud (14 percent), respectively. Migrating to a highly virtualized cloud environment is a significant transformation. It requires a considerable amount of technology and business planning. Over 84 percent of the organizations recognize the need for having an in-house team of professionals to lead, plan, design, and implement cloud and related technologies. Since cloud computing requires cross-technology domain skilled expertise, IT professionals need to have knowledge across technologies that will be used in cloud infrastructure and services. Thirty-seven percent of the professionals express that they have the prerequisite knowledge and skills (O/S, storage, networking, and virtualization) to get trained and become cloud-ready. IT transformation typically is an organization-wide initiative which requires leaders, managers, architects, and all other technical and non-technical professionals to collaborate to define and successfully execute the strategy. At a very high level, cloud and IT as a Service transformation requires detailed financial, governance, organizational, and technical considerations and planning. Only 18.4 percent of the managers (or organizations) are rated as strong across these dimensions, with financial skills being the strength of only 13.5 percent of the managers. Similarly, management skills are important for introducing the use of Big Data analytics in the organization to drive use of Big Data for various business objectives such as optimizing business operations, reducing risks, predicting new opportunities, compliance, marketing, and so on. Only 10-15 percent of the managers/ organizations seem to have a strong understanding of Big Data analytics. COMPLEX STORAGE ENVIRONMENTS Despite the differences in industry segments and data center size, there is a strong consistency across companies in terms of the technology deployed, storage management practices, and challenges. Nearly all critical data is now stored on external disk storage subsystems. The average usable capacity is approximately 2.18 petabytes (up 12.8 percent year over year). Factors such as growth in storage requirements, larger capacity disks and subsystems, and affordable pricing have led to larger storage configurations. Over 52 percent of responding companies now have 100 terabytes or more usable storage to manage. Seven percent of the companies now manage 10 petabytes or more usable storage out of which, 4 percent now have more than 25 petabytes of usable storage deployed. 4
- 5. Virtualization is implemented at 79 percent and cloud at 39 percent (up from 29 percent in 2012-13) of the represented companies. Big Data (infrastructure and/or analytics) is deployed by 20 percent (up from 16.5 percent in 2012-2013) of the organizations. Replication, storage area networks (SANs), and backup/recovery technologies are most commonly implemented, followed by network-attached storage (NAS). Technologies such as virtualization and cloud (private and public) have started to emerge strongly in the organizations. Each of these storage technology segments is unique, offering its own specific business and operational value. Each requires a different set of skills for effective design and management. Lack of knowledge and expertise in a specific segment can lead to under-deployment of one or more of these technologies. CRITICALITY OF STORAGE GROUPS Regardless of the environment—classic, virtualized, or cloud—the data, storage, and storage infrastructure are mission-critical. Losing storage in a catastrophic situation can severely damage an organization’s business. When a disaster does occur, information on storage subsystems can be lost permanently unless a well-designed recovery mechanism is planned and implemented. In addition to reliable equipment, a well-structured storage group of highly skilled professionals is critical to build and maintain a high-performance, high-availability storage infrastructure. An increasing number of companies have recognized this need and have created formal storage management groups. Storage groups are responsible for overall planning, design, implementation, monitoring, administering, managing, and operating the storage infrastructure. While the structure of the group, titles, and roles may not be standardized, responsibilities and tasks are common across companies. Based on IT/storage manager feedback, approximately two-thirds of existing storage teams can manage SANs, backup and recovery, and storage subsystems. However, only 40 percent or fewer of the existing storage teams are adequately prepared for emerging storage technologies such as Automated Storage Tiering, Data Mobility, Scale-out NAS, and so on. The emergence of virtualization, cloud/IT as a Service and Big Data technologies has placed new training and development requirements on storage managers and IT professionals. Only 10 percent are considered strong on cloud and 5 percent on Big Data. STORAGE TECHNOLOGY KNOWLEDGE GAP Although managers prefer to hire experienced or certified storage professionals, a severe shortage of such skills in the marketplace is causing managers to resort frequently to internal recruitment. The skills gap continues to widen with the adoption by organizations of virtualization and cloud computing. The shortage of experienced professionals in storage and emerging technologies (cloud, Big Data analytics) and the lack of education in the marketplace and in academia have restricted the growth of information storage and management functions and may be impacting the adoption of emerging IT and business transformation technologies. 5
- 6. RECOMMENDATIONS AND CONCLUSIONS Based on the study’s findings, IT managers and storage managers must ensure that: • Regardless of the current or future IT environment, a strong team of storage professionals is critical to store, manage, and protect valuable information. Continued focus and investment in this function will ensure an optimized, secure information infrastructure. • They evaluate the impact of virtualization and cloud computing on the skills of their storage and other technology teams and plan for them to acquire new, relevant skills in virtualization, cloud, and related domains. • Skills assessments and development of IT/storage professionals are top priorities to address growth and transformation. • Managers and business leaders invest in developing a stronger understanding of the impact and value IT as a Service and Big Data have on their businesses. The serious shortage of skilled storage professionals also creates attractive opportunities for the next generation of IT professionals and for those looking for a different career in a challenging, high-growth, dynamic industry. EMC’S RESPONSE AND INITIATIVES EMC® Education Services conducted this study to identify and address challenges faced by managers in the IT/storage industry. The following key initiatives offer options for IT/storage managers and professionals to acquire or improve their skills to benefit their organizations: EDUCATION SOLUTIONS • EMC Proven Professional Learning framework and Certification Program Quality assurance, formal validation and recognition Option of “Open” and EMC technology specialties The concept of “open” curriculum is a unique offering in the industry with a focus on concepts and principles rather than any specific vendor’s products • Cloud Infrastructure and Services “Open” curriculum and certification For all IT professionals to develop knowledge and skills on virtualized data center and cloud to become part their organization’s cloud transformation • Cloud Architect “Open” curriculum and certifications Cross-domain architecting and designing expertise for highly virtualized cloud environments • Data Center Architect “Open” curriculum and certification Storage domain architecting and designing expertise for highly virtualized data centers • “Open” courses for business leaders and managers Cloud and IT as a Service for Business Transformation Data Science and Big Data Analytics for Business Transformation 6
- 7. • Data Science, Big Data Analytics “Open” curriculum and certification For all professionals who want to learn advanced analytics, techniques, and tools and develop Data Science skills • Information Storage Technology “Open” Curriculum and Certification Develop the skills and knowledge to store, manage, and protect digital information in classic, virtualized, and cloud environments • EMC Technology-Specific Learning Paths Help leverage extensive capabilities of EMC technology and solutions Comprehensive coverage for all segments of EMC technology All of the above education solutions are available globally via EMC Education Services to EMC’s customers, partners, and employees. “Open” courses are available to all industry professionals via EMC, its business partners, and leading training providers and various websites (in popular Video-ILT self learning modes). To address the widening knowledge gap in the industry, the following exclusive programs were introduced to enable non-EMC users, as well as university students, to take advantage of the storage technology “open” curriculum to build a successful career in this high-growth industry: EMC ACADEMIC ALLIANCE PROGRAM Storage, cloud, and Big Data analytics “open” curriculum for students in hundreds of colleges and universities, targeted to help build a highly skilled pool of future storage, cloud, and Data Science professionals. INFORMATION STORAGE AND MANAGEMENT BOOK The most comprehensive book on information storage and management in the industry, recently re-issued in an expanded second edition, the Information Storage and Management Book is a “must have” addition to any IT reference library. Written by EMC technical experts, the book takes an “open” approach to teaching information storage and management, focusing on concepts and principles—rather than product specifics—applicable to all IT environments including classic, virtualized and cloud. 7
- 8. INTRODUCTION The unprecedented explosion of data, its increasing criticality, and business’ dependency on digital information are leading to larger and more complex information storage environments that are increasingly challenging to manage. From the perspective of data availability and protection, information storage infrastructure is the most critical component of an overall IT infrastructure. It plays a vital role in making applications work efficiently, both locally and across multiple sites. With the increasing complexity and criticality of storage, highly skilled and focused storage groups are as mission-critical as the technology being deployed. This paper summarizes a global research study that was conducted to learn how companies are meeting these challenging requirements. These findings will assist IT/Storage Managers to compare and correlate their plans with the overall trends in the industry. Even though each company has unique requirements, this information will be helpful in building stronger and more efficient storage management teams. Stronger storage management teams will, in turn, lead to more robust IT/storage infrastructures. The first such study was carried out by EMC in 2005-2006 and has been updated once a year since then. The updates and revisions for 2013-2014 include: • Most current information via global survey of managers (25 percent) and IT professionals (75 percent) from 800+ organizations • Trends related to established and latest storage technologies such as flash and scale-out-NAS • Adoption rates and impact of virtualization and cloud computing on current and required expertise • Skills and knowledge required by managers in IT as a Service and Big Data analytics for business transformation Global trends are identified in areas related to: • Technical environments and emerging technologies • Management challenges • Practices for building storage management groups • Need for acquiring more or better storage domain, cross-domain cloud, and emerging Data Science skills • Migration of data to virtualized and cloud environments The study was conducted over January and February 2013. We used comprehensive surveys and reached out to thousands of IT/storage professionals to assemble and compile this information. The study included: • All major geographies and industry segments • EMC users as well as those using storage solutions from other vendors • Large, medium, and small enterprises Figure 1. Geographic distribution of participants and annual revenue of companies 8
- 9. CHALLENGES FACED BY IT/STORAGE MANAGERS IT Managers/storage managers and storage professionals identified the following activities and constraints as their key challenges. These challenges are common to both large enterprises as well as small and medium enterprise (SME) sectors of the industry. Table 1. Most important activities/constraints identified as challenges by managers and professionals It is not surprising that managing storage growth continues to be the foremost challenge. A logical extension to meeting the capacity, performance, allocation, and management challenges is the ability to protect and secure information via comprehensive backup, recovery, archive, and disaster recovery strategy and execution. Being able to make big picture/informed decisions also continues to be an important challenge. In fact, this is becoming even more critical due to rapid IT transformation and the need to have trusted, highly competent leaders, advisors, architects, and designers to guide the direction of the IT organizations. The leaders and managers are challenged to understand and lead transformation that expands to financial, organizational, and technical considerations. Lack of skilled professionals for storage technologies and cloud computing is also becoming a major challenge. Newer information infrastructure technologies such as automated tiering, deduplication, flash, scale-out NAS, etc. provide some answers to managing unprecedented storage growth but also challenge the storage professionals to constantly invest in maintaining their knowledge, skills, and expertise. Table 2 provides a comparison of the challenges with the stacked list of challenges from the 2012-2013 study. Areas that saw significant increase in importance compared with last year’s survey include: • Managing external cloud service providers (100% increase in respondents from 4% to 8%) • Infrastructure for Big Data analytics (43% increase in respondents from 7% to 10%) • Designing, deploying, and managing storage in virtualized server environment (19% increase) 9
- 10. • Designing, deploying, and managing storage in cloud environment (12.5% increase) • Convincing higher management to adopt cloud (10% increase) • Making informed strategic/big picture decisions (8% increase) • Lack of skilled storage professionals (7% increase) Please note that since each respondent represents one company or organization, these increases represent the number of companies/organization that have rated these challenges higher than they did 12 months ago. Table 2. What keeps them awake? Pain points in order of priority identified by managers and professionals Emerging technologies and the transformation trends to cloud, IT as a Service, and Big Data appears to be a consideration in an increased number of companies/ organizations. Though the absolute percentages are relatively small, the increase in percentage year over year is significant. The stack rank and percentage is very likely to be higher if a similar study were done outside of IT/storage teams. IT MEGATREND: DATA MOVEMENT TO VIRTUALIZED AND CLOUD ENVIRONMENTS IT transformation projects are driving the movement of data to virtualized server and cloud environments from traditional or classic computing environments. Currently about 41 percent (down from 46.3 percent last year) of data is housed in the classic environment. During the next 24 months, about a third of the data (31 percent) from classic environments is expected to be moved to virtualized or cloud environments. Data volume in internal private cloud is expected to grow by 56 percent, whereas a growth of 100 percent of data is expected in the external cloud, even though the overall allocation still remains relatively smaller (6 percent). 10
- 11. Figure 3. Data movement to virtualized and cloud environments in next 24 months Combining the data from the 2011-12 study with this study creates a well-defined picture of data/storage movement in the short term—toward virtualized—and in the long-term—toward cloud. This trend is represented in Figure 4. It appears that by end-2012 there is more data in consolidated virtualized environments than in traditional/classic/non-virtualized environments. RATE OF IT TRANSFORMATION In order to predict the trend for the next four to five years, an extrapolated trend line is plotted against the actual data from the 2011-12, 2012-13, and 2013-14 studies. The 2013-14 study provides 2013 and 2015 data points (current and expected allocations 24 months out) similar to previous studies. Based on the identified trends: 1. More storage is now allocated in virtualized environments than in classic environments. This inflection point was projected in the 2012-2013 study and is confirmed by the data analyzed this year. 2. The industry will have more storage in internal/private cloud environments than in classic environments by end-2016. There seems to be a slower transformation than expected last year, when the survey analysis had shown this transition to occur by end-2015. 11
- 12. Figure 4. Rate of IT Transformation: data/storage migration from classic to virtualized and cloud environments DATA MOVEMENT BY COMPANY SIZE Is there any impact when it comes to adoption of virtualization and cloud by the size of the business or company? Figure 5 reflects that the general trend is common whether the company is relatively small (less than $100M revenue) or large (over $1B in revenue). Regardless of company size, data from traditional/classic environments is rapidly moving to virtualized and to private and external cloud environments. Figure 5. Current and 24 months out storage allocation by computing environments across studied companies COMPLEX STORAGE ENVIRONMENTS This study sought to determine the similarities and differences among storage infrastructures. The responses reveal that data centers across the Americas, Europe, and Asia deploy very similar storage solutions, including hardware and software. The sizes vary based upon business requirements, and in some cases a particular vendor may have a stronger presence in a given environment, but on the whole, the deployed technology and challenges are very similar. • 52 percent of the companies have 100 TB or more usable storage to manage • Nearly 7 percent of the companies have crossed the 10 PB mark in their usable storage capacity, including the 4 percent that have crossed 25 PB of usable storage • One out of every five (22 percent) of data centers has at least 1 PB of usable storage capacity to manage 12
- 13. Figure 6. Distribution of storage capacity across represented companies Average installed usable capacity is 2.18 PB. A similar calculation with last year’s (2012-13) data shows an average capacity of 1.9 PB, reflecting an increase of 12.8 percent year over year in the (average) installed usable capacities per company. This rate is generally in the same range (13.5 percent) as was seen from 2011-12 to 2012-13. This high rate of growth in storage demand causes the number one challenge for the IT/storage managers—managing storage growth. STORAGE TECHNOLOGY SEGMENTS Storage technology deployment and its importance to the data center aligned with general market trends for each of the storage technology segments. • Backup/Recovery/Archive, Storage Area Networks (SANs), and server virtualization were identified by the participants as the top three important technologies for their organization. • The reported importance of SAN technologies declined by another 12.5 points in the last year which makes this drop close to 22 percent over the last two years. This seems to be a significant shift as perceived by IT/storage managers and professionals. • With growing demand for storage comes a corresponding requirement for more optimized and efficient use of storage assets. Thus, this year was the first time the survey asked about automated storage tiering, a technology that reduces manual data administration/movement and improves mission-critical application performance. It was rated as the fourth most important and seventh most commonly deployed storage technology overall. Viewed together with other technologies aimed at resource, energy, and performance optimization—such as flash drives and storage virtualization—this strong technology presence paints a clear picture of IT storage environments increasingly emphasizing improved efficiencies. • Several new technology segments have been added (as below) which will help track trends in coming years. 13
- 14. Figure 7. Storage technology segments and their relative importance Each of the technology segments is unique, bringing its own specific business or operational values. For example, SAN and NAS provide connectivity options with unique functionality, while BR and replication technologies provide options for information protection against planned and unplanned outages. Figure 7 highlights the relative importance of technology segments associated by the respondents whereas Figure 8 highlights the technologies as they are currently implemented across the surveyed organizations. Figure 8. Most common storage technology segments implemented It is noticeable that a large percentage of organizations have deployed newer and emerging technologies such as Flash (56.5%), Automated Storage Tiering, Big Data, Mobility, scale-out NAS, and object-based storage (14.1%). These are fairly high percentages as typically the past studies have shown single-digit percentages at the early stages of new technology introduction. 14
- 15. ORGANIZATIONAL IMPLICATIONS OF MEGATREND TOWARD VIRTUALIZED AND CLOUD ENVIRONMENTS As the industry moves an increasing amount of storage/data to virtualized and cloud environments, leaders need to consider the multifold increase in complexity of designing, deploying, and managing storage infrastructure. Considering that at least 50 percent of the team needs to have the desired level of virtualization and cloud skills for a successful IT transformation, only 13.9 percent of managers (companies) feel confident that at least 50 percent of their storage teams have the necessary skills to support a private/hybrid cloud environment and only 34.7 percent indicate that they have at least half their team capable of working with virtualized environments as shown in Figure 9. Figure 9. Managers’ view of their storage teams’ skills for virtualized and cloud environments Migrating to a highly virtualized cloud environment is a significant transformation. It requires a considerable amount of technology and business planning. Companies recognize the need for having an in-house team of professionals to lead the planning, design, and implementation of cloud and related technologies. As illustrated by Figure 10, more than 80 percent of managers have indicated that they need in-house skills to design and build virtualized environments. The reported level of need for cloud environment in-house skills is currently at ~57 percent. An additional 27 percent of the managers believe that they will need these skills in 12- 24 months. Figure 10. Organizations requiring skills for architecting and designing virtualized and cloud environments 15
- 16. Designing, implementing, and operating cloud infrastructure and services requires IT professionals to have expertise across all of the foundational technologies. Seventeen percent of the professionals indicated that they possess the required level of skills across all core infrastructure technologies (see Figure 11). At the very least, it can be said that 37 percent of the professionals believe that they have the prerequisite knowledge and skills (e.g., O/S, storage, networking, and virtualization) to be trained and become “cloud-ready.” Figure 11. Current cross-domain skills of professionals for cloud infrastructure Only 28 percent of storage professionals assert that they have a level of skill that will enable them to perform security-related tasks independently. Table 3. Storage professional self-assessment of skill by domain FORMALIZED STORAGE GROUPS Storage infrastructure is mission-critical, and a very significant part of infrastructure budgets is allocated to storage-related products and services. A well-structured storage group of highly skilled professionals is critical to building and maintaining high-performance, highly available storage infrastructures. With the advent of storage virtualization and cloud computing, the industry is expecting an expansion of storage professionals’ core skills to include systems and networking skills, as well as increasing conversance with security considerations. 16
- 17. RESPONSIBILITIES Storage groups are responsible for the overall planning, design, implementation, monitoring, managing, testing, and operating of all components of the infrastructure. Interactions with IT/storage managers and professionals resulted in the list of activities/tasks for which they are responsible (see Table 3). The list includes various job functions, including storage administration, architecting, DR Administration, and BR Administration. The percentage of time captured for each of the activities highlights the effort involved and the possible importance of the tasks. This list could be used as a tool to define responsibilities of the storage group and individuals. When compared with results from the 2011-2012 study, though there are minor variations in the percentages, in general the priorities and time distributions remain very similar. Storage groups are spending significantly more of their time on monitoring storage infrastructure and storage capacity planning than previously reported. Nearly a third (31% percent) of storage team time is reported to be spent on planning/designing activities (14+6.8+6.4+4.3) as opposed to operational activities. Table 4. Typical tasks and responsibilities of storage teams and percentage of time spent (last 12 months) Another equally important dimension of these responsibilities is their alignment to different storage technology segments. Skills and processes are required to manage these tasks against expected expertise in one or more assigned ‘specialty’ or storage technology segment. Figure 12 illustrates the percentage of time spent in the last 12 months by key technology segments implemented. 17
- 18. Figure 12. Percentage of time spent by storage teams by storage technology segment (last 12 months) STORAGE GROUP SKILLS AND PERFORMANCE Managers were asked to rate the skills and capabilities of their teams both from a task/responsibility as well as focused technology perspective. Figure 13 illustrates the overall rating of the storage teams against the identified tasks and responsibilities (from Table 3). Strong, moderate, and weak bars indicate the level of expertise within the overall storage team in the represented companies. Figure 13. Ability to execute tasks—skill levels of storage professionals to carry out their tasks and activities Figure 14 maps skill sets of the storage teams against the important technology segments. Team skill levels in server virtualization, SAN, backup/recovery, and NAS are rated higher while skills in remote replication, local replication, and others are rated low. Most companies report very limited skill sets in emerging (and value-add service or strategically important) technologies such as converged infrastructure, Big Data, and cloud technologies. And yet, as noted in Table 5 below, more than four out of 10 responding organizations indicated that they plan to provide cloud-based IT services to external clients. This ratio held across different company sizes and points to an evolving IT as a Service strategy on the part of many organizations. A correlation between ability to execute tasks (as shown in Figure 13) and competence in relevant technology segment (from Figure 14) will paint a clear picture of the effective competence of the storage group. Detailed assessment of each individual within the group is required to ascertain strengths and weaknesses for each task and related technology segment. 18
- 19. Figure 14. Competence by storage technology segments Table 5. Organizations planning to provide cloud IT Services to external clients Yes 43.3% No 56.7% As has been well documented in industry literature, such a strategy requires storage and IT teams to adopt and support an increasingly service-oriented and value-add posture compared with the traditional infrastructure operation/maintenance role. The skill sets needed to accomplish this are more oriented to business planning and infrastructure design and architecture than the traditional operational skills found in storage and IT teams. MANAGERS’ SKILLS FOR CLOUD COMPUTING AND BIG DATA ANALYTICS IT Transformation typically is an organization-wide initiative which requires leaders, managers, architects, and all other technical and non-technical professionals to collaborate to define and successfully execute the strategy. At a very high level, cloud and IT as a Service transformation requires detailed financial, governance, organizational, and technical considerations and planning. Figure 15 shows the rating of skills that the management teams have today as perceived by the respondents to this study. Only 18.4 percent of the managers (or organizations) are rated as strong across the dimensions, with financial skills being the strength of only 13.5 percent of the managers. It is not surprising that over 42 percent of the IT managers are technically strong. 19
- 20. Figure 15. Managers’ skills for cloud computing Similarly, Figure 16 highlights the rating of skills of the managers for Big Data analytics (Data Science). These skills are important to introduce and foster use of Big Data analytics in the organization for business objectives ranging from optimization of business operations, to risk reduction, to predicting new opportunities, and so forth. Figure 16. Managers’ skills for Big Data analytics SOURCES FOR HIRING AND DEVELOPMENT The most significant challenge faced by IT/storage managers is the shortage of skilled storage professionals in the marketplace. In fact, lack of skilled storage professionals is the most serious industry challenge. Figure 17 shows the preferred hiring options. The majority of managers prefer to hire experienced professionals to reduce the learning period and risks associated with hiring new employees. The next-best alternative, indicated by 63.1 percent of the participants, is to hire certified individuals. 20
- 21. Figure 17. Managers’ preference for hiring or acquiring storage skills There is a shortage of skilled manpower in the industry. Capable, experienced, and skilled individuals are usually not available for hiring. Major factors for this skills shortage include lack of storage technology education in the marketplace and in academia. EMC has taken the lead and successfully introduced storage technology curriculum (and certification) in hundreds of universities as well as in the open market via public classes. A lot more needs to be done to convert this industry-wide bottleneck into lucrative employment opportunities for aspiring professionals. Given the scarcity of certified or well-skilled storage professionals in the market, managers frequently resort to internal recruitment. Often the internal recruitment involves moving an existing valuable employee who has a different expertise (such as operating systems, databases, etc.) but has very limited storage technology knowledge, creating a gap in both the technologies. Figure 18. Most trusted training sources for development of storage skills Technology vendor training, on-the-job training, and self-development by reading manuals, etc. are preferred sources of storage training and development, followed by training provided by authorized/independent IT training companies and training for certifications. On-the-job training, technology vendor training, and self-development typically cover usage and management of products and technology that are either already implemented or are in the process of being implemented. In addition, there is a need for wider and deeper training focusing on underlying technology concepts, planning, design, and management. 21
- 22. This will enable storage professionals to independently and more efficiently design and deploy storage infrastructures fully leveraging the capabilities of all applicable storage technology segments. RECOMMENDATIONS AND CONCLUSIONS From the perspective of data availability and protection, information storage infrastructure is the most critical component of an overall IT infrastructure. It plays a vital role in making applications work efficiently, both locally and across multiple sites. With the increasing complexity and criticality of storage, highly skilled and focused storage groups are as mission-critical as the technology being deployed. This study underscores a widening knowledge and skill gap in this fast-growing industry. A high-quality hiring and placement decision is possibly the most important challenge faced by IT/storage managers today, exacerbated by the shortage of experienced or certified professionals. Due to a lack of comprehensive storage technology education in the industry, most storage professionals have relied on on-the-job training, vendor product training, and self-development. Though this approach is probably adequate for day-to-day administration, the ability to make informed strategic decisions, and to proactively plan, design, and manage storage infrastructure is hampered by a lack of broad and deep knowledge. Well-constructed, comprehensive, and strategic plans must be efficiently implemented to meet the challenges presented by IT skills transformation, including: • Organizations must carefully analyze future skills requirements and current skill levels of their teams in the areas of storage technology (and other IT domains), virtualization, and cloud computing. • While maintaining and enhancing expertise in storage technology segments, storage managers/professionals must consider developing cross-domain skills combined with virtualization and cloud computing. • Organizations must evaluate the need for in-house expertise in designing highly virtualized, IT as a Service environment. A team of highly competent cloud and domain architects will ensure successful transformation to the cloud environment. • Business leaders and managers must invest time and effort to develop skills to drive business models, strategy planning, and execution for both IT Transformation and use of Big Data—the two mega trends in the industry. • IT vendors should develop core knowledge and skills in the industry when they introduce new technologies (in addition to focusing on their products’ features and functions). • Leading universities, colleges, and training providers must include storage technology, virtualization, cloud, and Data Science in their curricula to offer their graduates career opportunities in this high-growth industry. The next generation of IT professionals, or anyone looking for a different career path, have a great opportunity to learn the skills and meet the demands in this high- growth, dynamic environment. 22
- 23. EMC’S RESPONSE AND INITIATIVES EMC Education Services conducted this study to identify and address challenges faced by managers in the storage industry. The following key initiatives offer options for IT/storage managers and professionals to acquire or improve their skills to benefit their organizations: EDUCATION SOLUTIONS • EMC Proven Professional Learning framework and Certification Program Quality assurance, formal validation and recognition Option of “Open” and EMC technology specialties The concept of “open” curriculum is a unique offering in the industry with a focus on concepts and principles rather than any specific vendor’s products. • Cloud Infrastructure and Services “Open” curriculum and certification For all IT professionals to develop knowledge and skills on virtualized data center and cloud to become part their organization’s cloud transformation • Cloud Architect ‘Open’ curriculum and certifications Cross-domain architecting and designing expertise for highly virtualized cloud environments • Data Center Architect “Open” curriculum and certification Storage domain architecting and designing expertise for highly virtualized data centers • “Open” courses for business leaders and managers Cloud and IT as a Service for Business Transformation Data Science and Big Data Analytics for Business Transformation • Data Science, Big Data Analytics ‘Open’ curriculum and certification For all professionals who want to learn advanced analytics, techniques, and tools and develop Data Science skills • Information Storage Technology “Open” Curriculum and Certification Develop the skills and knowledge to store, manage, and protect digital information in classic, virtualized, and cloud environments • EMC Technology-Specific Learning Paths Help leverage extensive capabilities of EMC technology and solutions Comprehensive coverage for all segments of EMC technology 23
- 24. All of the above education solutions are available globally via EMC Education Services to EMC’s customers, partners and employees. “Open” courses are available to all industry professionals via EMC, its business partners, as well as leading training providers and various websites (in popular Video-ILT self learning modes). To address the widening knowledge gap in the industry, the following exclusive programs were introduced to enable non-EMC users, as well as university students, to take advantage of the storage technology “open” curriculum to build a successful career in this high-growth industry: EMC ACADEMIC ALLIANCE PROGRAM Storage, cloud, and Big Data analytics “open” curriculum for students in hundreds of colleges and universities, targeted to help build a highly skilled pool of future storage, cloud, and Data Science professionals. INFORMATION STORAGE AND MANAGEMENT BOOK The most comprehensive book on information storage and management in the industry, recently re-issued in an expanded second edition, the Information Storage and Management Book is a “must have” addition to any IT reference library. Written by EMC technical experts, the book takes an “open” approach to teaching information storage and management, focusing on concepts and principles—rather than product specifics—applicable to all IT environments including classic, virtualized, and cloud. For more information on the programs, offerings, alliances, and partnerships, visit EMC.com/training 24
- 25. Alok Shrivastava is senior director, EMC Education Services. Alok is the architect of several of EMC’s successful education initiatives including the industry-leading EMC Proven Professional program, industry readiness programs such as EMC’s Academic Alliance, and this unique and valuable paper on information storage technology. Alok provides vision and leadership to a team of highly talented experts, practitioners, and professionals that develops world-class technical education for EMC’s employees, partners, customers, students, and other industry professionals covering technologies such as storage, virtualization, cloud, and Big Data. Prior to his success in education, Alok built and led a highly successful team of EMC presales engineers in Asia-Pacific and Japan. Earlier in his career, Alok was a systems manager, storage manager, and a backup/restore/disaster recovery consultant working with some of the world’s largest data centers and IT installations. He holds dual Master’s degrees from the Indian Institute of Technology in Mumbai, India, and the University of Sagar in India. Alok has worked in information storage technology and has held a unique passion for this field for most of his 30-year career in IT. Ira Schild is a Senior Technical Education Consultant in EMC Education Services where he contributes to a diverse range of strategic, cross-functional projects. Since joining EMC in 1998 as a corporate systems engineer, Ira has led software product and API development groups as well as engineering initiatives in the areas of knowledge management and electronic license management. In the earlier part of his 30+ years in the technology industry, Ira was an applications programmer and a manager of technical support for several applications, networking products, and an operating system at Digital Equipment Corporation. Prior to that, he was a secondary education classroom teacher in English and mathematics. He holds an undergraduate degree from Cornell University, a Master’s in education from Hofstra University, and an MBA from the University of Massachusetts. CONTACT US Online http://education.EMC.com/contact Phone 888-EMC-TRNG (888-362-8764) International Asia-Pacific +61 2 9463 0000 (ANZ) +65 6333 6200 (South Asia) Latin America +55 11 5185 7138 EMEA_Education@EMC.com +44 208 758 6080 (UK) +49 6196 4728 666 (Germany) Greater China +86 10 8438 6593 India +91 80 6737 5064 Japan +81 44 520 9830 South Korea +82 22125 7750 www.EMC.com EMC2 , EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. © Copyright 2013 EMC Corporation. All rights reserved. Published in the USA. 4/13 EMC Perspective H2159.10 EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.