The Impact of AI on Storage IT Careers

Welcome to the latest edition of the Komprise Intelligent Data Management newsletter! We cover new ways for IT leaders to be more productive managing enterprise data and storage to dealing with ever-changing compliance issues, working with departments on data strategies and understanding the new requirements for data management and delivering AI-ready data. Learn more about Komprise, a SaaS solution for unstructured data management and mobility here and follow us on LinkedIn.

In 2020, the World Economic Forum predicted that by 2025, 50% of all employees would need to reskill because of technology change. Well, it’s now 2025, and my how this prediction has come true. Employees across most disciplines are expected to understand AI and use it in their jobs. Job ads increasingly call for a data-driven approach to leading teams and projects. Reskilling today is a continual process of experimentation, short courses and more experimentation.

So where does this leave the people who work in the data center, those who are responsible for managing and protecting all the data that AI needs?

Let’s take a closer look.

This month’s newsletter covers how storage IT roles and responsibilities are changing with AI. Krishna Subramanian, our cofounder and COO, wrote on the topic for BuiltIn. Here are the high notes below:

Expanding Job Roles

Storage IT professionals are increasingly managing unstructured data movement and access across complex hybrid cloud and multi-vendor environments. Top of mind today is:

• Addressing security threats from AI and cyberattacks;

• Helping prepare unstructured data for AI;

• Managing IT infrastructure and automating data workflows for AI.

Storage architects and engineers should consider how they can energize their career by focusing on data services not just storage procurement. This entails reducing security and compliance risks, using cost modeling and FinOps tools, delivering self-service tools for departments such as to search and tag their own data for classification, and preparing data and infrastructure for AI.

8 Ways Storage Pros Can Evolve for AI

1.  Stop. Collaborate and Listen. Thank you, Vanilla Ice, for elevating the words that matter. The job of managing technical configurations and resolving issues now requires a broader understanding of the full IT infrastructure. Storage professionals will thrive as trusted advisors to other IT and business roles to help set the requirements for data management vis-à-vis business objectives. This requires a collaborative mindset and business-savvy approach to meet departmental needs while satisfying IT objectives.

2. Get Data on your Data. In the AI age, storage professionals will need to work with data scientists, data analysts, project teams, department leaders—bringing more knowledge about data in storage. This entails identifying, segmenting, and defining data types and managing that data granularly, according to business and user needs. 

Identify and track:

• Data volumes and growth rates across the organization and by department;

• Cold versus hot data;

• Sensitive data;

• Common file types and sizes;

• Costs over time and modeled with new storage;

• Data tagging opportunities for improved classification.

3.    Cost modeling and FinOps. Large organizations are spending millions of dollars annually on data storage, backups and disaster recovery. On balance, there’s nothing wrong with that since data is the center of everything today – but all data should not be treated the same. Using cost modeling tools, the storage manager can enter actual storage costs to determine upfront new projected storage costs and actual usable capacity, based on data growth rates.  These costs must factor in backups and disaster recovery, which can be 3X of storage spending, and should compare on-premises versus cloud models. This eBook goes into more details on the tactics.

4.  Improve SLAs and data metrics for business.  Service level agreements for storage typically include data availability and uptime [a.k.a. the Three Nines or Four Nines], latency for hot and cold storage, backup frequency and windows, RTO and RPO and response times for issues such as critical data loss. There may be new metrics, however, that storage professionals need to start tracking as they deliver more services to business. These could include departmental chargeback or Showback, percent of non-compliant data in storage (such as personal videos or legacy system files that must be deleted), top data owners by department and individual and the amount of duplicated and orphaned data reduced.

5.   Reduce risks from ransomware.  Storage teams must mitigate ransomware risks associated with file data.  One way to do this is by implementing hybrid tiering strategies that offload infrequently accessed (cold) files to immutable cloud storage, which reduces the active attack surface by as much as 70 or 80 percent. Immutable storage ensures that once data is written, it cannot be altered or deleted, providing a robust defense against ransomware attempts to encrypt or corrupt files.

Read the blog for more insight on unstructured data and ransomware.

6.    Prepare infrastructure for AI.   Storage teams play a crucial role in AI by ensuring the reliable, scalable, and efficient storage and management of the massive datasets required for AI model training and deployment.  Procuring the right infrastructure to run AI workloads will vary significantly depending upon the size and budget of the organization, the level of AI customization needed, and other considerations such as security. Launching an AI initiative in your enterprise may require model development and training if you need to build your own generative AI model.  This typically begins with acquiring adequate high-performing computational resources—the pricey CPUs, GPUs, and TPUs that are required to host machine learning models and process data at warp speed. 

While pre-baked infrastructure, public models, and cloud services offer cost and ease-of-use benefits, IT organizations must also weigh the benefits of keeping AI in-house or developing a hybrid cloud AI model for better controls.

7.    Prepare data for AI.  Big data analytics and AI projects often involve processing large and diverse unstructured data sets, such as images, videos, documents, messages and sensor data. Storage teams are responsible for classifying and preparing this data for model training.  The first step is gathering basic insights about the data sets – such as its size, location, file type, access and usage patterns and if data needs enrichment (via additional metadata tagging) for better context and classification. Storage leaders also need to organize and manage data so that end users such as data scientists and researchers can easily search for and leverage the specific data required for their projects.  Using technology for automated data workflows streamlines this process of discovering, enriching, copying and/or moving data to the optimal location for analysis.

8.   Protect data for AI.   Storage professionals need to prevent sensitive data leakage to AI, as employees use GenAI and other tools daily.  This begins with identifying and segregating sensitive and proprietary data into a private, secure domain where it can’t be shared with commercial applications. It’s also vital to maintain an audit trail of your corporate data that has fed AI applications. A healthcare organization, for instance, would need to verify that no patient PII data has been leaked to an AI solution per HIPAA rules. Storage IT managers will need to help institute an AI data governance framework that covers privacy, data protection, ethics and more.

The age of AI and data is not just transforming technology and business models—it’s redefining careers. For IT storage professionals, the shift from managing infrastructure to orchestrating data presents both challenges and incredible opportunities.

Last Words

Now is the time to ask yourself: Are you simply maintaining storage and backups, or are you preparing data to drive business success? By expanding your skills in data intelligence, automation, FinOps, security, and AI governance, you can position yourself as an indispensable leader in the modern enterprise.

You can subscribe to our blog to receive new posts in your inbox and check out what's new by visiting our Resource Center.

Comment on the post or send a note to: ks@komprise.com.