8000 guide

Symmetrix 8000 Plus Storage Systems
Enterprise
Product Description Guide

8530 8230

8830

EMC SYMMETRIX 8000 ENTERPRISE PLUS STORAGE SYSTEMS PRODUCT DESCRIPTION GUIDE

Symmetrix 8000 Enterprise Plus Storage Systems

Table of Contents Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
EMC Enterprise Plus Differentiated Platform Capabilities . . . . . . . . . . . . . . . . . . . .4
Optimized Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Hyper-Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Ensure Information Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Provide System Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
The Challenge of Differentiated Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
The Solution: EMC Enterprise Plus Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Symmetrix 8000-Series Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Chapter 2: Symmetrix 8000 Enterprise Plus Storage Product Overview . . . . . . . . . . 7
EMC’s Architecture for Enterprise Storage: MOSAIC . . . . . . . . . . . . . . . . . . . . . . . . 7
Symmetrix System Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Channel Connectivity and Host Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Host Channel Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Open Systems Channel Directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Mainframe Channel Directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Remote Link Directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Disk Directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Disk Drives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Disk Scrubbing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Hyper-Volume Extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Meta Volume Addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Global Cache Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Parallel Cache Memory Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
CacheStorm ASICs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Proactive Cache Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Cache Chip Level Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Longitude Redundancy Code Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Cache Access Path Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Byte-Level Parity Checking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
System-Wide Error Checking and Correction . . . . . . . . . . . . . . . . . . . . . . . . .14
Efﬁcient Use of Available Cache Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Online Maintenance and Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Cached Data Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Enginuity: EMC’s Storage Operating Environment . . . . . . . . . . . . . . . . . . . . . . . . . .15
Optimized Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Optimizing Response Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
Symmetrix Read and Write Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Read Hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Read Miss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

1


Fast Write . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Delayed Fast Write . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Destaging Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
Enginuity Performance Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Intelligent Prefetch Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Least Recently Used Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Write Pending Indicator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
Back-End Layout Optimization or SymmOptimizer . . . . . . . . . . . . . . . . . . . .23
Quality of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Multiple ACCess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Disk Drive Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Disk Rotational Position Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Fast Write Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Write Destage Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Back-End Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Multiple Priority Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Disk Permacache Option . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
Disk Prefetch Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27
Chapter 3: Symmetrix 8000 Data Protection Options . . . . . . . . . . . . . . . . . . . . . . . .28
Symmetrix Data Protection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Mirroring (RAID 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Write Operations with Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Read Operations with Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
Mirroring Error Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Symmetrix Mirroring Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Parity RAID (RAID S) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
Write Operations with Parity RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Read Operations with Parity RAID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31
Parity RAID Error Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Symmetrix Parity RAID Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . .32
Symmetrix Remote Data Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
SRDF Campus Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
SRDF Extended Distance Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
SRDF Adaptive Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
SRDF Error Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
SRDF Multi-hop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35
SRDF Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Symmetrix Dynamic Sparing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36
Symmetrix Dynamic Sparing Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Chapter 4: Symmetrix Reliability, Availability and Serviceability Features . . . . . . . .38
EMC Design and Maintenance Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
EMC Remote Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Secure Network (SymmIP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38
Redundant Power Subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
Enhanced Battery Testing Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
Dual Initiator Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

2


Non-disruptive Component Repair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
Non-disruptive Microcode Upgrades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40
Symmetrix Non-disruptive Enginuity Upgrade Procedure . . . . . . . . . . . . . . . . . . . . .41
Chapter 5: Additional Symmetrix 8000 Mainframe-Class Features . . . . . . . . . . . . .43
Enterprise Storage Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Parallel Access Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44
Multiple Allegiance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Dynamic Parallel Access Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
IBM ESS 2105 Channel Command Emulation . . . . . . . . . . . . . . . . . . . . .45
Multi-System Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Sequential Data Striping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Mainframe Systems Hyper-Volumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Peer-to-Peer Remote Copy Emulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
FICON Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Symmetrix RAID 10 (Mirrored Striped Mainframe Volumes) . . . . . . . . . . . . . . . . . .47
Intelligent Resource Director Dynamic Channel Path Management . . . . . . . . . . . . .47
Dynamic Path Reconnection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Host Data Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Partitioned Data Set Search Assist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Multi-Path Lock Facility/Concurrent Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48
Chapter 6: Symmetrix 8000 Family Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Automated Information Storage (AutoIS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
WideSky Storage Management Middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Information Management Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49
Information Protection Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51
Information Sharing Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52
Chapter 7: EMC Global Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
EMC Powerlink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
Professional Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
Operations Management Consulting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
Information Storage Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Information Storage Consolidation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Business Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
Customer Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Pro-active and Pre-emptive Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Remote Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Software Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Change Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Installation Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Post-sale Warranty and Product Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Worldwide Organization, Local Support . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
Global Technical Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
Educational Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
The EMC Proven Professional Certiﬁcation Program . . . . . . . . . . . . . . . . . . .56
E-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56

3


Chapter 1
Introduction

Overview This technical overview provides information on the EMC Symmetrix® 8000 Enterprise Plus
Storage systems, including product descriptions and details of key features and operations.
This overview also describes EMC’s Symmetrix underlying storage system architectural
philosophy. The objective is to provide IT management and staff with a thorough technical
understanding of Symmetrix Enterprise Plus Storage systems.

EMC Enterprise Plus The Symmetrix architecture is designed to deliver industry-leading capabilities for customers who
Differentiated Platform have requirements beyond what industry standard storage delivers. Symmetrix goes beyond
Capabilities delivering just high performance to delivering optimized performance across hundreds of
applications with various workload characteristics. Symmetrix is also designed for customers who
require not just server or storage consolidation but hyper-consolidation of everything in the data
center from open systems, to mainframe and AS/400, to everything else.

Hyper-consolidation also dictates that the architecture be able to scale to terabytes of
information and support petabytes of information as a single managed infrastructure. And as
the number of applications grows and the amount of information increases, the need to
automate common management tasks becomes critical. But the most critical component of an
Enterprise Plus storage system is the ability to deliver true fault tolerance and non-disruptive
business continuity. All this and more is capable with the Symmetrix 8000 Enterprise Plus
storage systems.

Optimized Performance Symmetrix systems use a global memory and one hundred percent cache fast writes to ensure the
highest possible performance when writing data. EMC proprietary caching algorithms
dramatically increase the probability for “cache hits” when reading data. Symmetrix systems can
determine data access patterns in real time and intelligently optimize themselves for the best
performance, independent of the host processor, operating system, and application. Symmetrix
8000 series systems incorporate evolutionary improvements of Symmetrix cache with multiple
memory regions for increased concurrency of memory operations and provide the highest system-
level performance in the industry.

Also, with the introduction of Symmetrix 8000, EMC has incorporated more powerful
microprocessors, introduced faster memory, and doubled the number of internal data buses.
The result of these evolutionary enhancements is an enterprise storage system that operates at
peak efficiency, adapts to a constantly changing business climate, and easily accommodates
Internet-driven growth.

Hyper-Consolidation The Symmetrix 8000 series supports every major connectivity interface in the industry,
including mainframe connections through ESCON and FICON, as well as connections to open
UNIX, Windows, and AS/400 systems with connectivity to SCSI and Fibre Channels. Adding
Symmetrix Enterprise Storage Platform (ESP) software to Symmetrix 8000 systems enables
simultaneous support of mainframe and open systems connections, a capability unmatched in
the industry. This level of Symmetrix connectivity enables simultaneous support of multiple
hosts and multiple host types for greater configuration flexibility and the fulfillment of EMC’s
differentiated platforms philosophy.

4


Beyond just connectivity, Symmetrix also delivers infinite scalability. Symmetrix 8000 systems
enable consolidated storage strategies by providing scalable storage in a common family.
System capacities scale from 72GB to tens of terabytes of fully protected storage. Symmetrix
offers new ways to manage change and growth in applications, databases, servers, and overall
business requirements.

Ensure Information Protection Symmetrix provides a variety of hardware information protection features as well as optional
software applications. The Symmetrix 8000 architecture offers a choice of data protection at
the disk level: Mirroring, the optimal Redundant Array of Independent Disks (RAID) level for
both performance and availability; EMC’s enhanced parity protection; Symmetrix Remote
Data Facility (SRDF™); and Dynamic Sparing.

These basic data protection schemes are supported by full redundancy of data paths, Disk and
Channel Directors, and redundant power supplies with full battery backup to provide
protection against loss of data access due to component failure or power loss. All Symmetrix
8000 components are capable of non-disruptive replacement in case of a failure, enabling
Symmetrix 8000 systems to remain online and operational during component repair, with full
data availability.

Provide System Intelligence Traditional systems have placed the bulk of storage management decisions and overhead on
the operating system and host processor. Through its operating system-independent
technology, Symmetrix 8000 enables customers to consolidate storage from multiple
heterogeneous hosts. And since Symmetrix does not require specialized host device drivers,
customers can add new versions of operating systems and platforms while minimizing
operational impact. Since these capabilities are not tied to specific operating systems or
versions of operating systems, they can be exploited and do not require time-consuming and
costly software upgrades. These capabilities are used for virtually all major mainframe, UNIX,
Windows, PC LAN, and AS/400 systems without incurring host processor overhead.

The Challenge of Businesses today run at the speed of their information. Access to timely, robust information is
Differentiated Platforms a powerful asset that can fuel new ideas, boost revenues, build competitive advantage, and
enhance customer service. Yet in order to derive maximum business value from information,
companies must first unlock it from behind specific applications and processors across the
enterprise. No one can take full advantage of information that is isolated by different operating
systems and platform-specific data formats.

To drive better business results with technology, many companies are now consolidating their
information. Servers are being moved into the data center. Mainframes are being blended into
client/server environments. IT managers are acknowledging the wasted resources, expense, and
negative business impact of managing information across multiple operating environments
without a common management framework for the enterprise.

The Solution: EMC Symmetrix To realize an organizational vision of enterprise information, more and more IT departments
Enterprise Plus Storage are rejecting the notion of storage as an isolated CPU add-on or peripheral and searching for a
higher category of storage. They want storage that acts as a strategic element of an IT structure,
bridging the gaps between disparate platforms, so they can use their information in powerful
new ways. Beyond simply holding information, this storage must allow companies to manage,
protect, provide access to, and efficiently plan the growth of enormous amounts of information
previously dispersed on multiple servers and mainframes.

5


EMC Enterprise Plus Storage is answering the demand for enterprise information. Organized
through a suite of intelligent software capabilities, EMC Enterprise Storage™ is becoming a
fundamental technology enabler-as fundamental as networks, servers, and databases.

Symmetrix 8000 Series The Symmetrix 8000-series Enterprise Plus Storage systems provide a shared repository for a
Systems company’s most valuable resource—its information. Symmetrix 8000 systems provide the
industry’s highest performance, availability, and scalable capacity with unique information
protection, sharing, and management capabilities for all major open systems, mainframe, and
other environments.

There are currently three models in the Symmetrix 8000 family—the Symmetrix 8230, 8530,
and 8830. They form scalable families with leadership performance and capabilities in each of
their respective capacity classes. Additionally, Symmetrix Enterprise Plus systems deliver a
ﬂexible and continuously upgradeable information infrastructure. Symmetrix Enterprise
Storage systems deliver the performance, capacity, and availability required to compete in
today’s information-centric marketplace.

Symmetrix 8830 Symmetrix 8530 Symmetrix 8230

* Up to 69.5TB of storage with the * Up to 17.4TB of storage with Up to 4.3TB of storage with full
throughput, capacity, and con- increased capacity and perfor- Symmetrix functionality in the
nectivity to support the largest mance for multiple applications smallest footprint ever
data center consolidations and
* 8-96 disk drives * 4-48 disk drives
information infrastructures
* Up to 64GB of cache * Up to 32GB of cache
* 32-384 disk drives

* Up to 64GB of cache

As a result companies can:
• Connect to heterogeneous environments, facilitating the storage and retrieval from all major
computing platforms, including mainframe and open systems environments

• Create a competitive advantage by leveraging large amounts of information

• Provide high-level performance, capacity, and availability

• Ensure business continuity in the event of a disaster

• Deliver rapid and non-disruptive data migration from one system to another

6


Chapter 2
Symmetrix 8000 Enterprise Plus Storage Product Overview
EMC’s Architecture for EMC revolutionized storage in the mainframe environment with the introduction of the first
Enterprise Storage: Symmetrix in 1990. EMC became the first company to provide intelligent storage systems based
MOSAIC on redundant arrays of small, independent hard disk drives for the mainframe market. As a result,
businesses were able to access information more rapidly and reliably than ever before, and they
quickly began to view the strategic use of information as a competitive advantage. Today,
redundant array of independent disks (RAID) technology is widely accepted as the industry
standard for storage systems. In 1994, EMC extended Symmetrix technology to create the first-
ever platform-independent storage system, capable of simultaneously supporting all major
computer operating systems. Since the introduction of Symmetrix, more than 60,000 systems have
been shipped to customers around the world. In October 1999, Fortune magazine named EMC
one of the top-three “World’s Most Admired Companies” in its annual executive survey of product
quality and services.

Symmetrix is based on MOSAIC architecture, which is the field-proven time-tested foundation
for Symmetrix Enterprise Storage Plus functionality. The modular hardware architecture,
developed by EMC in the early 1990s, has enabled EMC to rapidly deploy the most advanced
technology, features and functionalities on high-performance Symmetrix platforms for a decade.

When advances in hardware, software, connectivity, or disk technology offer enhanced
capabilities, they are easily and economically integrated into Symmetrix family systems. The
basic system architecture can be continually enhanced as individual elements are added or
replaced. Designed-in investment protection is a hallmark of all EMC storage systems. As a
direct result of MOSAIC, EMC continues to introduce advanced technology and features into
the Symmetrix family, maintaining EMC’s lead in performance, data availability and
protection, mainframe and client/server integration, and many other customer requirements.

Customer Cache
Support Disk Scrubbing
Cache Center Scrubbing
Management
Configuration
Management Cache
Continuous
Traffic
Power
Management

Disk

Channel SCSI Interface
Adapters

Disk
Adapters PC Interface

Remote
Interface
Service
Expert Processor
Application Systems
Module

7


Symmetrix System Operation Basic operations in the Symmetrix 8000 systems involve Channel Directors, Global
Memory Directors, Disk Directors, Disks, and the ﬂow of data among these components, as
illustrated in the following architectural diagrams.

Symmetrix 8230 Architecture

8




Channel Connectivity Symmetrix systems can be integrated easily and quickly with all major enterprise servers and
and Host Integration mainframes systems. Symmetrix 8230, 8530, and 8830 systems support connectivity to
mainframe and/or open systems hosts. Open systems platforms connect through SCSI and Fibre
Channel interfaces. Mainframe connectivity is supported through ESCON and FICON channels.

All Symmetrix systems are operating-system independent. The Enginuity™ Storage Operating
Environment is self-managed, and Symmetrix 8000 systems do not depend on host cache
commands to receive the beneﬁts of read and write caching. This means that the Enginuity
Storage Operating Environment provides simultaneous connections for mainframes (IBM
OS/390 and zSeries), UNIX, Linux, Windows, and AS/400 (IBM iSeries) systems.

9


This specialized Storage Operating Environment enables combinations of ESCON Channel
Directors, FICON Channel Directors, Ultra SCSI Channel Directors, and Fibre Channel
Directors on the same Symmetrix system. For configuration flexibility, these Directors can be
installed in combination in the Symmetrix systems, facilitating the concurrent storage of
mainframe and open systems data in the same system.

EMC Symmetrix systems support connectivity options to a vast majority of host environments
that include all major open systems and mainframes hosts. For details of specific server models
and supported operating system versions and interface technologies, see the EMC Support
Matrix at www.emc.com/horizontal/interoperability/interop_support_matrices.jsp, or contact
your EMC sales representative.

Host Channel Connection All Symmetrix 8000 systems provide exceptional host channel connectivity through combinations
of Channel Directors. Each Channel Director supplies multiple independent data paths to global
memory, then to disk, from the host system. Channel Directors are installed in pairs, providing
redundancy and continuous availability in the event of repair or replacement to any one Channel
Director. These include ESCON channels, FICON channels, SCSI and Fibre Channels, and
Remote Link Directors.

Open Systems The Symmetrix 8000 systems support open UNIX systems, Linux, Windows NT systems,
Channel Directors TRU64, and AS/400 connectivity through Symmetrix Fibre Channel and SCSI Channel
Directors. Each SCSI Channel Director is a single board with four host connections. Fibre
Channel Directors have two to twelve connections per Director, and depending upon the
Symmetrix 8000 model, there are from two to eight Channel Directors per system.

Mainframe Channel Directors The Symmetrix 8000 systems support mainframe connectivity through ESCON Channel
Directors and FICON Channel directors. Each ESCON Channel Director supports four
ESCON channel connections, and each FICON Channel Director supports two FICON
channels.

Remote Link Directors The EMC Remote Link Director (RLD) facilitates the direct connection between two
Symmetrix systems in a Symmetrix Remote Data Facility (SRDF) or Symmetrix Data
Migration Services (SDMS) configuration. SRDF and SDMS mainframe implementations
require a minimum of two, and support a maximum of four RLDs in each connected system.
SRDF implementations can be either ESCON or Fibre Channel. SDMS implementations are
ESCON only. For open systems, SRDF over Fibre Channel implementations use Remote Fibre
Directors (RFD) for connecting Symmetrix systems using high-speed Fibre Channel links.

Disk Directors The Disk Directors manage the interface to the physical disks and are responsible for data
movement between the disks and global memory over the Symmetrix 8000’s four-bus memory
architecture. Symmetrix 8000 models have up to eight Disk Directors per system, each with
two advanced microprocessors. Each Disk Director is connected to two memory buses to
maximize data throughput and performance. Each logical data volume is connected to two of
the Symmetrix 8000’s Disk Directors to provide a redundant, or alternate, data path. Disks are
connected to Disk Directors through industry-standard SCSI interfaces. This allows rapid
introduction of the latest disk drive technology into Symmetrix systems.

Disk Drives Symmetrix systems use industry-standard SCSI disk drives for physical disks, allowing EMC to
keep pace with customer needs as technology enables increased capacities and improved
performance. Each hard disk drive is configured with its own controller consisting of control
logic, a microprocessor, and a device-level cache, designed to enable high-speed transfer
between the buffer on the hard disk drive and the Disk Director.

10


Every disk drive contains its own microprocessor that has the capability of self-management.
This gives Symmetrix the ability to perform parallel tasks such as diagnosis and simultaneous
transfers, and further enhances performance.

Symmetrix 8000-series systems support mixed configurations of 36GB and 73GB (10K rpm),
and 181GB (7200 rpm) disks drives. This breadth of scalable capacity and configuration
choices allows Symmetrix systems to adapt to virtually any enterprise storage requirement.
Any combination of disk drives can be deployed in Symmetrix 8000 systems to provide the
exact combination of performance and capacity required.

Disk Scrubbing During idle time, the disks are read (“disk scrubbing”), looking for any type of error. Disk
scrubbing is accomplished in a manner similar to cache scrubbing, as described later.

Upon sensing a correctable error, the error is corrected and then rewritten. The block of data is
read again to verify that it was a permanent correction. If it is correctable, the pertinent
information is logged and scrubbing continues. If the error is not permanently corrected, the
process is repeated until it is either corrected or the error recovery routines determine that a
skip defect must be executed. If the skip defect must be executed, it is done via Symmetrix
Enginuity. When the skip defect is complete, notification is given, and the scrubbing process
continues. Should a threshold number of skip defects occur on a track that would make an
alternate track assignment necessary, that too is accomplished through Symmetrix Enginuity
and is transparent to the user.

Hyper-Volume Extension Symmetrix enhances disk system functionality by supporting up to 128 logical volumes on one
physical device. Logical volumes are the actual volumes with which a host communicates. The
hyper-volumes are configured upon initial Symmetrix setup. Additional hyper-volumes can be
dynamically added as the customer requires more capacity. Up to a maximum of 8,000 logical
volumes are supported on a Symmetrix system.

For mainframe customers, the standard IBM device types are supported, including all 3380 D,
E, and K’s and 3390 models 1, 2, 3, 9, and 27. Non-standard hyper-volumes can also be
defined for customers who desire them.

For the customer using Symmetrix in an open systems, UNIX, NT, or Linux environment,
hyper-volumes can be created as large as 15GB in size. For those customers needing larger
volume sizes than 15GB, EMC offers meta volume addressing.

Meta Volume Addressing Symmetrix also enhances disk system functionality in Windows NT and open systems UNIX
and Linux environments through meta volume addressing capability. A meta volume is a group
of logically connected hyper-volumes that creates a single logical view to a host. Symmetrix
supports up to 255 logically connected logical volumes. These logically connected hyper-
volumes are not required to be contiguous. This facility can be used to overcome the addressing
limitations imposed in Windows NT environments, where currently allowable volume size is
15GB. With Symmetrix system’s 255 logical volumes, meta volumes of up to 3.8TB are
possible.

Global Cache Director At the heart of EMC Symmetrix is the Global Cache Director with CacheStorm™ technology,
a multi-functional, high-performance, parallel-designed, solid-state subsystem that delivers
unmatched high-end performance and data integrity. CacheStorm technology enhances system
performance, improves responsiveness, and manages peak I/O requests through a series of
techniques that reduces contention for shared cache and optimizes utilization of system
resources. The underlying principles are fairly simple:

11


• Cache memory is partitioned into 16 separately addressable regions

• Requests for cache are expedited to reduce locking

• Requests are intelligently arbitrated to optimize available resource usage

CacheStorm consists of two major functional components, described as follows.

Parallel Cache The Symmetrix Global Cache Director with CacheStorm technology accommodates four
Memory Regions separately addressable, simultaneously accessible regions. So, in a Symmetrix system with four
cache directors, there are 16 separately addressable and accessible cache regions. Compared to
single region cashing, this greatly reduces the probability of contention for cache access that
results in cache queuing and lower performance.

CacheStorm ASICs The Global Cache Director expedites transactions between process requests and cache.
CacheStorm technology Application Speciﬁc Integrated Circuits (ASICs) on the Global Cache
Director act as intelligent ofﬂoad engines to perform repetitive system critical functions.

One function ASICs performs is buffering service requests for cache. These buffers have a
region to store reads, a place to store writes, and an area to store address and
command/instructions. As soon as a process gets access to the cache region it needs to access,
the intelligent ASIC buffers the incoming request and frees up the cache region. Then, within
the ASIC, it performs the instructed operation e.g., read/write to cache. Buffering incoming
requests locally on ASICs and freeing up blocked cache regions as soon as possible results in a
truly non-blocking architecture that is capable of massive performance scaling.

CacheStorm ASICs also arbitrate incoming requests for cache resources in a way that optimally
allocates cache regions to incoming requests by appropriately timing and intelligently pre-

12


fetching required information from cache into the ASIC buffer. This results in optimal
utilization of available resources.

Instruction Set Logic, routine cache-related activities being requested from processors mounted
on channel (front-end) and disk (back-end) directors are built into the logic in the ASIC. This
expedites the process of cache-related transactions and reduces the time for which cache is to
be blocked for servicing a process request. The result is that cache requests are processed
through ASIC hardware instantaneously without waiting for PowerPC processors on the
channel and disk directors.

Proactive Cache Maintenance EMC makes every effort to provide the most highly reliable hardware in the industry. However,
all hardware is subject to the effects of aging and occasional failures. The unique methods used
by Symmetrix for detecting and preventing these hard failures in a proactive way set it apart
from all others in providing continuous data integrity and high availability.

Symmetrix 8000 actively monitors I/O operations for temporary errors. By tracking these soft,
or temporary, errors during normal operation, Symmetrix can recognize patterns of error
activity and predict a potential hard failure before it occurs. This proactive error tracking can
usually prevent an error in global memory by fencing off, or removing from service, a failing
memory segment before data errors occur.

Constant cache scrubbing to detect and correct single- and double-bit errors dramatically
reduces the potential for multi-bit or hard errors. In addition to monitoring recoverable
conditions during normal access, all locations in global memory are periodically read and
rewritten to detect, and correct, single- and double-bit errors. A Symmetrix system’s global
memory scrubbing technique maintains a record of errors for each memory segment.

If the predetermined error threshold is reached, the segment’s contents are moved to another
area in global memory, and the segment is ‘fenced’ and removed from service. A service
processor call-home function alerts EMC to the unacceptable level of errors, and a non-
disruptive memory replacement is ordered. A Customer Service engineer is dispatched with the
appropriate parts for a speedy repair.

Should a multi-bit error be detected during the scrubbing process, it is considered a permanent
error, and the segment is immediately fenced. Data affected by the error is recovered from disk
or ﬂagged as invalid in the case of write-pending data. A service processor call home is placed
as previously noted.

Cache Chip-Level Redundancy Traditional cache memory systems usually provide for 8 bits of parity information to support
bit error correction and detection in a 64-bit long word. EMC’s Global Cache Director
incorporates Single Nibble Correction Double Nibble Detection. (A nibble is four consecutive
bits of information.) This is achieved by internally generating 16 bits of ECC parity
information and replacing existing 8 bits of incoming ECC information. This enables the
system to correct up to four bit errors associated with a 64-bit long word.

Symmetrix Global Cache Directors can also detect up to eight bit errors. Another beneﬁt is that it
interleaves 64 bits of information plus 16 CacheStorm parity information (total 80 bits) across 20

13


memory chips on the cache board. This results in each memory chip storing only a nibble of
information corresponding to a word. So, a chip-level error will disable access only to the nibble
stored on that faulty chip. However, CacheStorm enables regeneration of data from the faulty chip.
This leads to chip-level redundancy making every chip on the cache memory board redundant.

Longitude Redundancy Symmetrix Global Cache Directors also incorporate Sector Level longitudinal redundancy
Code (LRC) Checks checks, which further assure data integrity. The check bytes are the XOR (exclusive OR) value
of the accumulated bytes in a 4KB sector. LRC checking can detect both data errors and
incorrect block access problems.

Cache Access Path Protection Before Symmetrix cache can accept data from a host connection, it must ensure that the area to
which the data is to be written is without error. Symmetrix assures the highest level of data
integrity by checking data validity through the various levels of the data transfer in and out of
cache.

Byte-Level Parity Checking All data and control paths have parity generating and checking circuitry that verify data
integrity at the byte or word level. All data and command words passed on the system bus, and
within each director and global memory board, include parity bits used to check integrity at
each stage of the data transfer.

System-Wide Error Checking Both channel and disk directors correct single-bit errors and detect and report double-bit
and Correction (ECC) errors. Error detection and correction circuits on each director continuously check all transfers
within Symmetrix.

A service processor call-home function alerts EMC Global Service Call Centers whenever an
unacceptable level of errors has been detected and a non-disruptive replacement is ordered.
Customer Service is immediately notified of all call-home alerts, and a customer engineer can
be dispatched with the appropriate parts for speedy repair. Even in cases where errors are
occurring and are easily corrected, if they exceed a preset level, the call home is executed. This
represents the EMC philosophy of not accepting any errors.

Efficient Use of Available In early design testing, EMC discovered that cache mirroring is an inefficient way of creating
Cache Memory redundancy for failsafe operations. Cache mirroring results in two cache operations in the case
of system read events and five cache operations in the case of system writes. In addition to this,
mirroring wastes 50 percent of useful memory on the mirror. EMC analysis revealed that
memory boards themselves do not fail, however, memory chips on memory boards start
misbehaving over time. This leads to a design to ensure that each and every chip on the memory
board is redundant - eliminating any single point of failure on cache boards. This also results in
higher utilization of available memory resources resulting in higher system throughput.

To achieve the goal of making each and every memory chip redundant on the memory board, 8
bits of extra parity information are stored in addition to usual 8-bit parity information that
goes with a 64-bit long word. The result is 10 percent of extra memory capacity to create chip-
level redundancy as compared to 50 percent waste in the case of mirrored cache boards.

14


Online Maintenance Every Symmetrix is configured with a minimum of two global memory directors to allow for
and Replacement online hot replacement of a failing board. If a hard error is detected, or the temporary errors
reach a predetermined threshold, the Symmetrix service processor calls home to request an
immediate maintenance action. When board replacement is required, global memory usage is
redirected to the remaining good boards in the system, and the suspect board is removed and
replaced non-disruptively while the system remains online.

Cached Data Protection Symmetrix Enterprise Storage systems provide 100 percent system non-volatility. If there is
any power interruption, EMC’s fully redundant battery backup system fully powers the entire
system, flushes the cache, completes all pending writes, parks the drives, and gracefully powers
the system down into a known good state. Symmetrix batteries are “N+1” and are not only
voltage tested but also continuously “load tested” as part of the normal internal preventive
monitoring performed by the Symmetrix to ensure the highest level of data protection.

Enginuity: EMC’s Storage The Symmetrix Enginuity storage operating environment consists of over 1.6 million lines of
Operating Environment system software executing on over 61,760 MIPS of processing power (EMC Symmetrix 8830).
Enginuity orchestrates all hardware, onboard functionality (such as SRDF, TimeFinder, Data
Mobility, etc.) and application workloads concurrently, while maintaining the highest levels of
end user responsiveness and system availability.

The combination of Symmetrix hardware architecture and Enginuity operating system software
has been continuously updated over time to deliver advancements across all aspects of storage
operations, including performance, functionality, connectivity, capacity, and availability.

Customers’ real-world workloads are very different than most benchmarks used to measure
the performance envelopes of many competing storage subsystems. Real-world workloads are
composed of many different types of I/O activity. They can be read or write requests, they have
different data block sizes, they can be skewed (some disks or host channels doing more work
than others), they can be highly random, sequential or mixed, and they are often “bursty”
(peak reads or writes can come at unexpected times). The workloads used for envelope
measurements are normally static, simple, and designed to always yield certain levels of hit
ratio (access of r/w data directly out of cache), regardless of the cache size and algorithms. In
real life, the actual application behavior is greatly influenced by the performance optimization
algorithms.

Enginuity contains extensive algorithmic intelligence that is designed to achieve the following goals:

• Maximize the read hit (read access from cache memory) ratio...leading to fast application
response time

• Minimize data de-stages to the disks...improving write hit (write access to cache memory) ratios,
optimizing use of internal resources and improving response time

• Avoid extreme situations...to not over consume and to optimize use of internal resources

• Allow end user definition (and future assignment) of priorities for Symmetrix operations...to set
service levels for specific workloads

• Be Efficient...to reuse valuable information for multiple purposes, balance the load evenly
among Symmetrix components, and save valuable resources

• Be proactive...to identify patterns or sequences as soon as possible to optimize operations

• Optimize data layout based on detection of long-term workload patterns

15


Mainframe Host

Directory

Open Systems Host
Cache
Memory

Channel Director Disk Director Disk

Symmetrix 8000 Systems

Optimized Data Flow Symmetrix 8000 models optimize the movement of data for the highest performance possible.
There are four internal buses-top high, top low, bottom high, and bottom low. Symmetrix 8000
systems greatly exceed the throughput and response time performance of conventional disk
storage systems, because the majority of data is transferred to and from global memory at
electronic memory speeds, not at the dramatically slower speeds of physical disk devices.
Director boards, both those connecting to a host and those connecting to the disks, are the
means by which data interfaces with global memory. Director boards are designed to work in
pairs, where each director is connected to two buses. This ensures access to data in the event of
an unlikely failure of any bus.

Symmetrix 8000 systems optimize the ﬂow of data between hosts and disks by:

• Minimizing the number of accesses to the disks

• Executing I/Os in an order that minimizes the time the disks spend for seek and latency, whenever
disk access is unavoidable

Optimizing Response Times The data inside Symmetrix is logically organized in tracks. These tracks are organized into
logical volumes, which are presented to hosts. All data travels through the global memory
directors. The global memory is logically divided into slots. A slot in global memory is
associated with a track of data. A slot may contain an entire track of data, or just part of it.

The slots in the Symmetrix global memory are divided into three logical groups. This division
of data is very ﬂexible. A cache slot can move from one group to another by merely changing a
few pointers without having to move any data.

1. Least Recently Used (LRU) Chain

An LRU chain is a bi-directional linked list dynamically sorted by age of the linked slot. The LRU
chain is the main contributor to read hits. The Symmetrix supports multiple (up to sixteen) simul-
taneous LRU chains. The LRU in these chains are de-staged to the disk in order to create more
room in global memory.

16


2. Permacache

Permacache is a collection of cache slots that is “permanently” associated with tracks. These
tracks contain critical information that needs rapid response whenever it is needed. Users can spec-
ify which tracks need an association with Permacache. In addition, whenever Enginuity storage
operating environment running on Symmetrix systems can predict that certain data is likely to be
accessed extensively in the near future, it creates a Permacache association for that piece of data.

3. Write Pending Slots and Write Pending Indicators (WPI)

Write pending slots contain data that was written to global memory but has not been destaged to
disks. These slots are removed from the LRU chain. The WPI indicates which slots are waiting for
a disk destage.

Depending on the I/O pattern at any moment, the portion of cache dedicated to the LRU or to
Write Pending varies signiﬁcantly. The tracks designated by the user to be Permacache remain
in Permacache until the user changes their designations. The other Permacache tracks, those
that were automatically selected by Symmetrix, will change their status automatically when
the likelihood of reusing them does not justify their Permacache status.

Symmetrix Read Four basic types of operations occur in a Symmetrix system: Read Hit, Read Miss, Fast Write, and
and Write Operations Delayed Fast Write. The following diagrams illustrate these operations.

Read Hit A Read Hit occurs on a read operation when all data necessary to satisfy the host I/O request
is in global memory. The Channel Director immediately transfers the requested data from
global memory to the host and updates the cache directory. Since the data is in global memory,
there are no mechanical delays, and data is transferred at electronic speeds. With the large
amounts of global memory offered on Symmetrix 8000 systems, it is common for applications
to attain a read hit ratio (requested data is in global memory) of 90 to 95 percent.

Read Hit

1] Directory Search- Hit
2] Transfer to
Host
3] Update
Directory

Host Channel

2 1

Directory

3
Channel Cache
Director

Disk Director Disk

17


Read Miss In a Read Miss, data necessary to satisfy the host I/O request is not in global memory, so it must
be retrieved from disk. The Disk Director reads the block(s) containing the data from disk,
transfers them to global memory, and updates the cache directory. Simultaneously, the Channel
Director reconnects to the host and transfers the requested data to the host.

Read Miss
1] Directory
Search- Miss
2] Position
Read/Write
Head, Stage
Data to Cache
3] Transfer to
Host
Host Channel 4] Update
Directory

3 1

Directory

4
Channel Cache
Director

2

Disk Director Disk

Fast Write A Fast Write occurs whenever there is global memory available to accept the data being written.
On a host write command, the Channel Director places the incoming block(s) directly in global
memory and immediately sends a ‘write complete’ message to the host. Since Symmetrix Fast
Writes are complete when the data is written to global memory, there are no mechanical delays.
The Disk Director will asynchronously write the data to disk.

Fast Write

1 Search-hit cache directory
2 Transfer to Cache
3 Update directory
Host Channel 4 Destage asynchronously

Directory

Channel
Director Cache

Disk Director Disk

18


Delayed Fast Write A Delayed Fast Write occurs only when the Fast Write threshold has been exceeded (that is, the
percentage of global memory containing modified data, unwritten to disk, is too high to
accommodate the Fast Write data). The Disk Directors immediately destage data to disk as a high-
priority task. When sufficient global memory space is available, the Channel Director processes the
host I/O request as a Fast Write. With sufficient global memory installed, this type of global
memory operation will rarely occur.

Delayed Fast Write

1 Search cache directory
(cache is full)
2 Destage page
3 Update cache directory
Host Channel 4 Transfer to cache
5 Update directory
6 Destage asynchronously

Directory

Channel
Director Cache

Disk Director Disk

Destaging Operation A background operation also occurs in Symmetrix systems. This background operation destages
blocks of data to disk. Frequently used data is maintained in two locations: global memory for high
performance in the occurrence of reuse of that data and on disk to maintain the highest levels of
data integrity. All pending writes are assured of arrival to the intended disk even in the event of
power failure. (See the Non-Volatile Power System section.) The following diagram illustrates this
destaging operation.

Destaging Operation

1] Destage
Block(s)
2] Update Directory

Host Channel

Directory
2

Channel Cache
Director

1
Disk Director Disk

19


Enginuity Performance Simply having these robust cache configurations is not enough. One of the fundamental differences
Optimization Algorithms between Symmetrix products and all other data storage systems is the advanced caching
algorithms that allow intelligent use of the installed global memory for high performance. A
potential problem with increasingly large global memory configurations is that search time
increases proportionally, since this search time is added to every I/O request, read hit, read miss, or
write. This is a considerable penalty for every I/O request, especially in performance-critical
applications. In some data storage systems, the controller may actually disconnect from the
channel during this process and must then reconnect if there is a cache hit.

Symmetrix systems perform the global memory search via advanced patented algorithms,
determining-in microseconds-if a record is in global memory. As well as searching quickly and
efficiently to determine whether the requested data is in global memory, they also understand how
the application is accessing the data and tune themselves accordingly in real time. These advanced
algorithms allow the search time to remain constant regardless of application workload.

With global memory searches performed at electronic speed, there is no reason to disconnect
from the channel during the search. In fact, it takes longer to disconnect and reconnect than it
does to perform the global memory search. In normal operation, the only time that a
Symmetrix system will disconnect from the channel is in the event of a read miss. This is a
complex series of tasks and requires the advanced global memory management algorithms of
Symmetrix to be accomplished effectively.

Symmetrix global memory management is based on the principle that the working-set of data
at any given time is relatively small when compared to the total system storage capacity. When
this working-set of data is in global memory, there is a significant improvement in I/O
performance. The performance improvement achieved is dependent on both:

• Locality of Reference-If a given piece of data is used, there is a high probability that a nearby
piece of data will be used shortly thereafter.

• Data Reuse-If a given piece of data is used, there is a high probability that it will be reused
shortly thereafter.

This cache principle has been in use for years on host processor systems. The following figure
illustrates this type of host cache use. The cache used in this manner is often a high-speed, high-
cost storage unit used as an intermediary between the CPU and main storage.

CPU Cache Memory

Intelligent Prefetch Algorithm This algorithm prefetches data from disks to the cache before the host issues a read command
to this data, in anticipation that the host will shortly want to read this data. It works by
identifying sequential reads. EMC’s prefetch algorithm will reduce response time and improve
the utilization of the disks. The prefetch algorithm maintains, per each logical volume, an array
of statistics and parameters based on the latest sequential patterns observed on the logical
volume. Prefetch dynamically adjusts based on workload demand across all resources in the
backend of the Symmetrix. This algorithm also ensures that cache resources are never overly
consumed in order to maintain optimal performance.

20


Enginuity algorithms continually monitor I/O activity and proactively look for access patterns.
When a second sequential I/O to a track read occurs, the sequential prefetch process is invoked, and
the next track of data is automatically read into global memory. The intent of this process is to avoid
a Read Miss by anticipating the data that will be requested. Once the first track is completely read
by the host processor, the third track is read and reuses the same global memory location as the first.

This process of using the cache track slots in a round-robin fashion prevents cache pollution
caused by conventional sequential caching algorithms. Should a Read Miss occur, the
Symmetrix global memory management will increase the number of track slots read from two
to five. If a Read Miss still occurs, the Symmetrix prefetch routines will continue to increase the
number of track slots read. The maximum number of track slots that will be allocated for a
sequential operation is 12. Should I/O activity reduce, the number of track slots will be reduced
accordingly. When the host processor returns to a random I/O pattern, the Symmetrix system
will discontinue the sequential prefetch process.

Whenever the workload presented to the storage system contains sequential read patterns, it is
very beneficial to prefetch data from the disks to the cache before this data is actually requested
by the host. This helps in two major ways:

• If the data resides in cache when the host is actually reading it, then the response time for this
operation is reduced by about 10 times. Reading from cache takes a few hundred microseconds,
while accessing the physical disk takes several milliseconds.

• The utilization of the physical disk drive is improved, since large portions of data are read from
the disk each time, seek and latency times are reduced to almost zero.

It’s no wonder that all storage vendors employ a prefetch algorithm to achieve these
improvements. However, a bad prefetch algorithm can have a devastating effect on the overall
performance of the system. For sequential I/O performance measurements, most benchmarks
use workloads with very long sequences. Even a simple prefetch algorithm can be made to look
good in these situations. But, in real-life cases where sequences are of various lengths,
customers want a sophisticated and self-adjusting algorithm that on one hand, does not
prefetch too much, and on the other hand, prefetches all the data that is needed and does it on
time without affecting the response times of the other I/Os.

Various storage vendors use different approaches to prefetching. Most vendors use a very
simple algorithm: they prefetch a very large (e.g., 1MB) amount of data from disks to cache
upon detecting a certain number of sequential read operations. Some of the simple algorithms
are very aggressive about prefetch. They prefetch after detecting a sequence of two I/Os. Others
are more conservative. They start to prefetch only after detecting a sequence of eight I/Os.

The Symmetrix adaptive intelligent algorithm automatically adjusts to the workload and
constantly monitors the success rate of its decisions. In real-life workloads, the Symmetrix
approach is significantly superior to the others. The conservative approach fails to detect 90
percent of the sequences, and thus fails to use the disks more efficiently and improve host
response times. The aggressive approach may prefetch significant amount of data that will
never be used by the host computers.

Least Recently Cache Least Recently Used (LRU) is a list of slots (a pre-defined piece of cache that relates to
Used Algorithm data areas on disk) with application data that was recently used. Numerous studies have
proven that data that was more recently accessed has a higher chance of being accessed again
shortly. The LRU algorithms in Symmetrix are designed to maximize hit ratio in the most
efficient manner. There are sixteen independent LRUs in a Symmetrix system.

21


Write Pending Indicator Cache Write Pending Indicator controls all the slots that have written data that has not been
destaged to the disks. Like the Read Hit case, numerous studies have proven that data that was
written recently has a higher chance of being written again shortly. Therefore, it is beneficial to
keep this data in cache before it is de-staged to the disk. The write destage algorithm constantly
adjusts itself to the existing workload. It is designed to improve the overall performance by
taking into account the effect of keeping written data in cache on the Read and Write Hit ratios
and by optimizing the order in which the tracks are being destaged.

In Symmetrix, the preferred mode of data protection is RAID 1. In RAID 1, each logical volume is duplicated
into at least two mirrors. Each mirror resides on a different hard drive or drives. In most cases, the different
mirrors reside on different disk directors that are serviced by different memory buses. This duplication of
pathing allows Symmetrix to decide from which mirror the data should be read. Symmetrix allows users to
manually set the mirror service policy for each logical volume. However, because workloads change over
time, and because the number of logical volumes in a system is permanently growing, setting one policy as
optimal, or close to being optimal, is practically impossible. When the user sets the Mirror Service Policy
(MSP), he or she determines which of the mirrors of a given logical volume should service a Read Miss
operation.

The two possible policies are:

• M1/M2: One of the mirrors should service all the reads from this logical volume.

• Interleave: The different mirrors alternate on each cylinder. Mirror 1 (M1) serves the odd num-
bered cylinders, while Mirror 2 (M2) serves the even numbered cylinders.

Generally speaking, the Interleave policy benefits sequential patterns, because under this
policy, all the physical drives transfer data. The M1/M2 policy benefits random patterns,
because it limits the distance the disk actuator needs to travel.

DMSP is a dynamic approach to setting the optimal mirror service policy. The DMSP
algorithm monitors the access patterns to the different logical volumes in the back-end, and
based on these access patterns, determines a policy for the next short time interval. As of
Enginuity 5x68, DMSP takes into account all the local mirrors of the logical volume, including
its Business Continuance Volumes (BCVs). The DMSP algorithm tries to achieve two goals:

• Balance the load among all the disks and other Symmetrix back-end components.

• Minimize the time the physical drives spend on seek and latency.

22


The challenge is to achieve the two goals simultaneously or to achieve the goal that is more
relevant to the current situation. Assume, for example, that a mirrored physical drive has two
logical volumes, one doing 10 I/Os per second, while the other is doing 40 I/Os per second.
Intuition will mislead us to use a policy that will balance the load between the drives. That
way, each physical drive will execute 25 reads per second, 20 from one logical volume, and 5
from the other. But deeper analysis, or simple disk simulation, proves that in this case, we will
be much better off if each physical drive serves one logical volume. This is so because whenever
I/Os are limited to a smaller portion of the disk, the disk performance is much improved, and
because executing 40 I/Os per second on a physical drive does not create any significant
queues. If the expected load on the logicals was doubled (80 and 20), then the considerations
may be different, based on the physical disk characteristics.

The DMSP algorithm has three distinct stages:

• The first stage is geared towards load balancing the different Symmetrix components. These
components include the Disk (DA) directors, the interfaces to the disk drives, and the disk
drives themselves.

• The second stage starts with the policy determined by the first stage, and derives from it several
other potential policies in which seek and latency times are improved.

• The third stage uses a simple simulation to evaluate all the policies produced at the previous
stages, taking into account the actual characteristics of the workload, like random versus sequen-
tial, write percentage, etc. The policy that scores the best is chosen for the next time interval.

Back-End Layout Optimization Similar to DMSP, SymmOptimizer is designed to improve disk utilization by balancing the
or SymmOptimizer load and minimizing the disk seek time. While DMSP is focusing on the short term (every few
minutes), Optimizer examines the workload patterns over extended periods of time and
optimizes disk performance for the long term. It achieves this by moving logical volumes to
different disks or to different locations on the same disk. Decision making data is collected at a
granularity of 5-15 minute intervals. The optimization algorithm module uses this data to
identify overloaded physical volumes, or hot spots. It then determines a series of logical
volume moves that would relieve these hot spots. The data-moving module is responsible to
control the actual moving of logical volumes on the physical drives.

Like the DMSP algorithm, SymmOptimizer is designed to improve disk utilization by
balancing the load among the hard drives, while minimizing the disks’ seek and latency times.
DMSP focuses on the near real time. It examines the workload patterns of the last few minutes
and sets the mirror service policy for the next few minutes. SymmOptimizer, on the other hand,
examines the workload patterns over extended periods of time, usually days or weeks, and
optimizes disk performance for the long term. It does this by moving logical volumes to
different disks or to different locations on the same disk.

SymmOptimizer has three modules:

• Data collection

• Optimization algorithms

• Data moving

23


The data collection module collects back-end activity statistics of each logical volume and of
each physical drive. The optimization algorithm module uses this data to identify overloaded
physical volumes or hot spots. It then determines a series of logical volume moves that would
relieve these hot spots. The data moving module is responsible for controlling the actual
moving of logical volumes on the physical drives.

The SymmOptimizer algorithm is based on a very interesting observation. The workloads that
run on a given Symmetrix vary over time. In general, the workload characteristics observed in
the last few minutes are a good predictor of the workload characteristics of the next few
minutes. This is the basis for DMSP. But beyond a few minutes, most of the workload
characteristics may change considerably. The one characteristic that is most stable in the
workloads running on the same Symmetrix is the activity correlation between the logical
volumes. If logical volumes X and Y are active at the same times today, they are very likely to
be active at the same times tomorrow. Similarly, if logical volume X is not active when Y is,
then there is a very good chance that this correlation will remain.

Given this observation, the SymmOptimizer goal puts highly correlated volumes on different
hard drives as much as possible. A second goal puts the busiest logical volumes in the most
optimal location on the drive, which is close to the outermost tracks. A third goal is that, if
positively correlated volumes need to reside on the same hard drive, then they should reside
close to one another. All these goals are translated to a cost function that the SymmOptimizer
algorithm tries to minimize.

The SymmOptimizer algorithm performs two functions. Based on the cost function described
above, the SymmOptimizer algorithm ﬁrst calculates an optimized layout of data on the
physical drives. Next, the SymmOptimizer algorithm calculates an optimal series of data
moving steps to achieve the desired layout. The focus of the second function is to execute the
moves in an order that yields better performance as soon as possible.

Quality of Service Quality of Service, or QoS, lets Symmetrix users control, to a great degree, the performance
level that selected applications receive from Symmetrix. The settings of Quality of Service can
be adjusted at any time to adapt to a system’s I/O requirements. For instance, by reducing the
“quality of service” for BCV or SRDF copy operations on selected devices, customers free
Symmetrix resources and increase the overall performance of the other Symmetrix devices.

One of these Quality of Service features, nLRU-QoS, enables users of Symmetrix systems to
allocate a portion of cache for a subset of the logical volumes. Being able to control how cache
is allocated guarantees that these logical volumes, and the applications they are used for,
achieve a high hit ratio, regardless of the other applications running at the same time. This
feature also lets customers specify when an application can lend portions of its cache to other
applications.

With the nLRU-QoS, customers can guarantee a certain level of performance for applications
or users that demand certain levels of performance, regardless of other applications running on
the system at the same time. The nLRU-QoS feature is implemented through the nLRU
mechanism. The cache slots can be divided among up to 16 independent LRU rings. Customers
can assign a different size for each LRU and map sets of logical volumes to sets of LRU rings.

Another QoS feature permits Symmetrix users to specify the time when a background activity,
such as a Copy, Backup, or BCV Establish, needs to complete. Customers set the time period

24


for the background copy operation, and Symmetrix executes the background activities with
minimal effect on the performance observed by the host applications and completes the
background activities on time.

Multiple ACCess (MACC) Multiple ACCesses (MACC) is available for both mainframe and open systems; this algorithm
constantly scans the incoming I/O requests queue and tries to execute as many of them as
possible concurrently. Up to four concurrent accesses to disks are supported per logical
volume. MACC benefits application performance in several ways. When the logical volume is
striped, it allows parallel use of several disks. Otherwise, it improves disk performance by
queuing the I/Os on the disk, thus allowing the RPO optimization to kick in. Another
advantage is that when there is a mix of Read Hits and Read Misses on the same logical
volumes, the hits do not have to wait.

Disk Drive Optimizations Modern hard disk drives have their own optimization algorithms. These optimizations have a
huge effect on the disk’s performance. For example, the Rotational Position Ordering (RPO)
optimization can more than double the number of random I/Os a disk can do. This section
concentrates on the RPO and on four new disk-level performance features that are unique to
EMC drive microcode.

Disk Rotational Position Whenever multiple I/O requests are queued on the disk, Enginuity optimizes the order in which
Ordering (RPO) the I/Os are executed. The RPO optimization reorders the I/Os based on their physical
locations on the drive. RPO optimization significantly reduces the effect of seeks and latency
times on the overall performance of the disk. To take full advantage of the RPO optimization,
Symmetrix needs to queue enough I/Os on the physical drives. The more I/O demands the
Symmetrix encounters, the better it will perform.

Whenever multiple I/O requests are queued on the disk, the disk microcode can optimize the
order in which the I/Os are executed. The Rotational Position Ordering optimization reorders
the I/Os based on their physical locations on the drive. It always schedules the I/O that can start
before all the others in the queue, following the completion of the current I/O. Simulations and
real-life benchmarks show that the RPO optimization significantly reduces the effect of seeks
and latency times on the overall performance of the disk.

With RPO, whenever a sufficient number of I/Os (about five or more) are directed to the disk,
the rate of random I/Os that a disk can perform more than doubles. Up to a certain limit, the
number of random I/Os a disk can do increases as the number of I/Os queued on the disk
increases, because with more I/Os queued, the RPO optimization has more candidates from
which to choose. RPO optimization especially benefits large capacity drives (such as the
Seagate Barracuda 181) for two reasons. Large capacity drives are more likely than smaller
drives to have several I/Os queued - just because they have more data to be accessed. This lets
the RPO optimization kick in more often.

Another reason why RPO benefits large capacity drives more than smaller drives is that RPO
optimizes seek and latency times. It does not optimize transfer times. Large capacity disks, by
their very nature, are denser than disks with smaller capacities, therefore, their transfer rates
are much higher. As a result, the large capacity disks spend a greater portion of their time doing
seek and latency as compared to smaller capacity drives.

Fast Write Algorithm Fast write I/O operations benefit the customer by lowering response time for write activity to the
sub-millisecond level. Keeping the written data in cache for a while saves destage operations, as
discussed in the section on cache algorithms. Fast write also allows Symmetrix to accommodate
bursts of writes at a speed above and beyond the speed that the hard drives allow. The negative
effects of “bursty” writes are minimized through this algorithm.

25


Write Destage Algorithm Write destage algorithm orders the cache write data destage activity to minimize disk seek
times, a major factor in optimizing application performance. For each logical volume,
Symmetrix maintains a special data structure that points to the data that needs to be destaged
in the cache Write Pending Indicators (WPI) already discussed. This dynamically adjusting
algorithm saves disk seek and latency time by destaging data in groups of up to four tracks
concurrently per logical volume.

Back-End Scheduler The back-end handles various types of activities. Some of them are of high priority, like
servicing Read Misses. A Read Miss has high priority because the host computer is waiting for
the data. Most of the other tasks, like write-data destaging and prefetching, have lower priority
because the host is not waiting. There are three priority levels: high, medium, and low. The
scheduler’s job is to make sure that all the low-priority tasks are executed in a timely manner,
with minimal effect on the performance of the high-priority tasks. It makes sure that no task is
starved for too long. Users can tune up performance of the scheduler by adjusting parameters.
These parameters specify what percentage of the time should be dedicated to tasks in each
priority level and to each type of task within the same priority level.

Multiple Priority Queues Disk Multiple Priority Queues enable Symmetrix to give better response times to I/Os that the
hosts are waiting for without sacrificing the disk RPO optimization. The Multiple Priority Queues
algorithm handles starvation situations, so that even low-priority I/Os are serviced within a certain
period of time. Definition of the starvation time may have a huge effect on disk performance. If we
only cared about average response time, then we would not worry about starvation at all. In a real-
life situation, it is important to respond to an I/O within some reasonable amount of time. To
guarantee this, we want to shorten the definition of starvation time.

On the other hand, in order to get the full benefit of the RPO algorithm, we need to queue
many I/Os on the drives. Whenever many I/Os are queued on the drive and the starvation time
is too low, it is likely that many of the queued I/Os will starve, and therefore will be executed
out of the optimal order. This will cause more I/Os to starve. Eventually, the effect of the RPO
algorithm will totally vanish. Symmetrix can adjust the definition of starvation time based on
the queue length at any given moment. The starvation time grows with the length of the queue,
up to a certain limit, defined separately per each disk type. This lets Symmetrix queue many
I/Os on the disk, have the full benefit of the RPO algorithm, and still have very reasonable
starvation time when the disk is not very busy.

Disk Permacache Option Disk Permacache allows Symmetrix to control, to a very high degree, what data resides in the disk
cache. This can be viewed as an extension of Symmetrix Permacache, with the benefit that only
data that is actually requested by the host will travel on the Symmetrix buses.

26


Disk Prefetch Algorithms Controlling Disk Prefetch allows Symmetrix to prefetch more aggressively to the disk cache.
The regular Symmetrix prefetch algorithm prefetches data only when the probability that the
host will actually read the data is high. With Disk Prefetch, we benefit even when the
probability that the data will be read by the host is lower. This is so because prefetching to the
disk cache does not use Symmetrix resources, but still has the traditional benefits of prefetch:
reducing response times and improving the utilization of the disks. The probability that the
prefetch data will be read by the host is computed using the sequential pattern statistics
collected for the traditional prefetch.

Disk Prefetch allows Symmetrix to prefetch more aggressively to the disk cache. The Cache
Prefetch Algorithm of Symmetrix prefetches data only when the probability that the host will
actually read the data is high. With Disk Prefetch, Symmetrix benefits even when the
probability that the data will be read by the host is lower. Prefetching to the disk cache does not
use valuable Symmetrix global cache and bus resources but still has traditional benefits of
prefetch, including reducing response times and improving the utilization of the disks. The
probability that the disk prefetch data will be read by the host is computed using the sequential
patterns statistics that are collected for the cache prefetch.

27


Chapter 3
Symmetrix 8000 Data Protection Options

Symmetrix Data Protection EMC has chosen to enhance the basic RAID level definitions in each of the three implementations
of data protection that are offered for Symmetrix 8000 systems. The EMC Symmetrix
implementations of data protection are able to exploit Symmetrix intelligent functionality that
differentiates the EMC offerings from typical RAID offerings. Users can select the protection
schemes they desire-Mirroring, Parity RAID, SRDF, and/or dynamic sparing-to optimize the best
relationships of availability, performance, and cost for individual data sets. These options are
configurable at the physical volume level, so that different levels of protection can be applied to
different data sets within the same Symmetrix 8000 system. This unique flexibility allows the
customer to maintain the lowest possible costs in relation to the necessary levels of performance
and data availability.

• Mirroring (RAID 1)-provides the highest level of performance and availability for all mission-
critical and business-critical applications by maintaining a duplicate copy of volumes on two disk
devices.

• Parity RAID (RAID S)-offers more usable capacity than a mirrored system containing the same
number of disk drives through performance-enhanced parity-based data protection.

• Symmetrix Remote Data Facility (SRDF)-a system-based version of real-time mirroring between
multiple Symmetrix systems that can include remote and multiple sites.

• Symmetrix offers a Dynamic Sparing option, which reserves volumes as standby spares. This
option increases data availability without impacting performance and can be used in conjunction
with Mirroring, Parity RAID, or SRDF.

Mirroring (RAID 1) Mirroring provides the highest level of performance and availability for all mission-critical and
business-critical applications. Mirroring maintains a duplicate copy of each logical volume on
two physical disk devices. Symmetrix maintains these copies internally, transparent to the
host(s), by writing all modified data to both devices. Symmetrix designates two logical volumes
residing on different physical disks as a mirrored pair-one volume being mirror-1 (M1) and the
other volume mirror-2 (M2). The host(s) view the M1 and M2 volumes as the same logical
volume because each has the same address. To ensure the highest availability, each volume is
attached to separate Disk Directors, which are attached to different memory buses.

The EMC implementation of RAID 1 Mirroring on Symmetrix systems includes performance
enhancements such as DMSP, beyond the high-performance capabilities normally associated
with RAID 1.

Write Operations with Mirroring A write operation to any mirrored volume is executed identically to a non-mirrored write. The
Channel Director presents Channel End/Device End to the host after data is written and
verified in global memory. The Disk Directors then asynchronously destage the data to each
drive of the mirrored pair of drives. As such, Mirroring on Symmetrix exploits the 100 percent
fast write capability, and the application does not incur additional time associated with having
to physically perform two disk write I/Os (one to each drive of the mirrored pair) as is normally
associated with RAID 1.

Read Operations with Mirroring The Symmetrix performance algorithms for read operations in mirrored pairs offer three service
policies to best balance the use of the Symmetrix architecture: interleave; split; and dynamic.

28


The interleave service policy has an objective of maximizing throughput. It uses both the M1 and
the M2 disk for each read operation in a flip-flop method, a number of tracks from M1, and a
number of tracks from M2. Sequential workloads make the best use of interleave service policy.

Split service policy differs from interleave in that read operations are assigned to either the M1
or M2 disk, but not both. The objective of the split service policy is to minimize disk actuator
contention by only moving the disk heads on one of the two disks in the mirrored pair. Random
workloads make the best use of the split service policy. In the case of multiple hyper-volumes in
the mirrored pair, certain logical volumes are read exclusively from M1, and certain logical
volumes are read exclusively from M2.

Symmetrix Dynamic Mirroring Service Policy (DMSP) is an EMC-unique enhancement to
Symmetrix that provides intelligent algorithms for processing read operations for mirrored
(RAID 1) and business continuance volumes (BCVs). The major benefit of DMSP is its ability
to dynamically choose between split or interleave depending on the application’s workload.
This algorithm is another step EMC is making towards a self-tuning storage subsystem. As the
access patterns and workloads change, the DMSP algorithms evaluate the new workloads and
adjust service policies as needed to maximize performance.

Mirroring Error Recovery In the unlikely event that one disk in the mirrored pair fails, the Symmetrix instantly and
automatically begins using the second disk drive of the mirrored pair for I/O operations
without any interruption in data availability (see the following section on EMC’s Dynamic
Sparing Option). The Symmetrix system notifies and alerts the EMC Customer Support Center
via an Auto-Call action. The EMC Customer Support Center product support engineer (PSE)
then begins the diagnostic process, and if necessary, dispatches a customer engineer (CE) to the
customer site. Once the suspect disk is non-disruptively replaced, the Symmetrix system re-
establishes the mirrored pair and automatically resynchronizes the data with the new disk.
During the data resynchronization process, the Symmetrix system gives priority to host I/O
requests over the copy I/O to minimize the impact on application performance and user service.

Read

Logical
Volume

Mirror
Mirror Mirror
Policy 2
1 Decision

Symmetrix Mirroring Advantages

In summary, EMC’s RAID 1 Mirroring provides:

• Improved performance over traditional RAID 1 by supporting 100 percent fast write, and two
simultaneous internal data transfer paths.

• DMSP algorithms that evaluate workloads and adjust service policies as needed to maximize per-
formance.

29


• Protection of mission-critical data from any single point of failure.

• Continuous business operation by switching to the alternate disk of a mirrored pair without
interruption to data availability should loss of access occur to one of the mirrored pair.

• Automatic resynchronization of the mirrored pair after repair of the suspect volume.

• Transparency to the host processor and operating system.

Parity RAID (RAID S) Symmetrix 8000 provides parity-based data protection similar to RAID 4 and RAID 5, but with
significant advantages for performance, flexibility, and data availability. Compared to a mirrored
Symmetrix, Parity RAID offers approximately 33 percent more usable capacity than a mirrored
system containing the same number of disk drives. Like mirroring, Parity RAID protection can be
dynamically added or removed. For example, for higher performance requirements and high
availability, parity protection on a RAID group can be turned off and the volumes in the RAID
group mirrored. Within the same Symmetrix system, data can be protected via Parity RAID,
mirroring, and/or SRDF. Dynamic sparing can be added to any of these data protection options.

One of the factors contributing to the higher performance of the Symmetrix Parity RAID
option is that Symmetrix takes advantage of the ability of the latest disk drives to calculate
parity at the disk itself. The Boolean operation “Exclusive Or” (XOR) logic used to calculate
the parity is carried out by a microprocessor with XOR logic and disk cache on each disk drive.
This greatly improves write performance by offloading these calculations from the host or
Symmetrix system, allowing them to continue to service I/O requests. Since the Channel
Directors do not need to calculate parity, and I/Os are serviced from global memory, Parity
RAID will not impose performance penalties on the host processor.

EMC currently recommends data protected by Parity RAID be grouped with a ratio of three
data disks to one parity disk. Though Symmetrix 8000 allows the intermixing of different
capacity disks within a single Symmetrix system, all physical disks participating in a RAID
group must have identical storage capacity. With this approach, 75 percent of the total storage
capacity of each Parity RAID group of volumes is available for storing data. Multiple RAID
groups may exist within a single Symmetrix system. Members of a RAID group can be located
anywhere in the Symmetrix system, spanning multiple Disk Directors.

A logical volume describes the actual unit of data that is discretely protected by Parity RAID. A
logical volume may be as large as an entire physical volume, or disk, or may be a subset of the
physical volume. With Hyper-Volume Extension, up to 128 logical volumes may exist on one
physical volume. Both data and parity associated with logical volumes is distributed across the
RAID group so that the parity for any RAID group always resides on a separate physical drive
from the data volumes in that RAID group.

A rank describes the logical volumes, which are related to each other for common parity
protection. All logical volumes within the rank must be identical in capacity. A minimum of
one rank and a maximum of 128 ranks can exist within a single Parity RAID group. Hyper-
Volume Extension is used when supporting any number of ranks greater than one.

A data volume is similar to a traditional logical volume. It is the “virtual volume” image
presented to the host operating system and defined as a separate unit address to the host. All
data volumes within a rank must be the same size. There can be a maximum of 8,000 data
volumes in Symmetrix 8000 systems.

30


Write Operations with A write operation to a Parity RAID volume is a Symmetrix Fast Write, and the application does
Parity RAID not incur additional delay associated with having to physically calculate parity as is normally
associated with other parity RAID implementations. The Channel Director presents Channel
End/Device End to the host immediately after data is written and veriﬁed in global memory.
When Parity RAID data is later destaged to disk, it follows the following sequence.

In the Parity RAID write process, performing the read old data and XOR functions at the disk
device level reduces the Disk Director’s operations to a single read (difference data) and two
writes (new data to the data volume and difference data to the parity volume). This is a
reduction in the number of disk operations that must occur to write data when compared to
traditional independent access parity RAID levels. The following ﬁgure illustrates how data is
destaged to disk through the following sequence of commands Symmetrix uses in the Parity
RAID write process:

• XD-Write-Read

• XP-Write

Read Operations with During read operations, if the data requested is not in global memory (a Read Miss), a normal
Parity RAID read is initiated from the data drive within the Parity RAID group that contains the requested
data. There is no XORing, and only one disk drive is involved in servicing the request. This
offers advantages over other parity RAID implementations that ‘stripe’ data across multiple
drives. If unrecoverable errors are detected in attempting to read the data, data recovery
utilizing parity and the surviving volumes in the same rank will be initiated.

31


Parity RAID Error Recovery Parity RAID is able to provide continuous data availability for all data in a RAID group
should any single physical or logical volume fail or become unavailable within that RAID
group. Should a data volume report too many errors or fail outright, that volume will be taken
ofﬂine by the Symmetrix system, and the appropriate automatic calls will be logged to the
EMC Customer Support Center to initiate diagnostics and problem resolution.

When a volume within the RAID group fails or becomes unavailable, the RAID group is put in
reduced mode, and parity protection for the data volumes in the RAID group is immediately
turned off. These volumes will now serve all their I/O requests as standard data volumes, or
data will be reconstructed from parity. All data is still available to the host, but is unprotected
against additional failures unless protected by dynamic sparing.

If Symmetrix is conﬁgured with dynamic sparing, Symmetrix copies the data from the failing
volume to the spare, reconstructing the data if necessary from parity. Symmetrix also invokes
available spares for the remaining volumes in the RAID group, if they are available. This
establishes a mirrored relationship between the three data volumes in the RAID group and
three spare drives, which can be located anywhere in the system. The data volumes on the
unaffected disks, along with readable data volumes from the failing disk, are copied to the
spare disks. Any unreadable data is recreated, using parity, and copied to the spare disks.
These volumes will now serve all their I/O requests as normal mirrored data volumes.

No parity data is copied to the spare disks, and no parity generation continues since all the
data is now protected via mirroring. This provides immediate protection from subsequent
failures prior to a service action. Replacement of the failed disk can take place at a time
convenient for the customer. Once replaced, the RAID group will rebuild itself to RAID parity
protection, and the spares will again be made available.

Symmetrix Parity RAID • Offers more usable capacity than a mirrored system containing the same number of disk drives
Advantages through performance-enhanced parity-based data protection.

• Delivers high performance, even in the event of a disk failure within a RAID group. When a disk
failure occurs, all logical volumes that were not physically stored on the failed disk device will
perform at the level typical of standard Symmetrix devices.

• Protects volumes requiring high availability from being a single point of failure as any opera-
tional Parity RAID data volume can continue to service I/Os, regardless of disk failure within
that RAID group.

• Dramatically reduces the “write penalty,” since the XOR calculation is done at the disk level,
and data is not striped. Only three physical disk operations need to occur to perform a write.
This results in superior performance relative to traditional parity RAID protection.

• Since EMC’s Parity RAID does not stripe data from a single volume across multiple physical
disks, no performance tuning is required. When data is striped across multiple volumes (tradi-
tional RAID 4, RAID 5, and RAID 6), the complexity associated with performance tuning is
substantially greater.

32


Symmetrix Remote SRDF is an online, host-independent, mirrored data storage solution that duplicates production
Data Facility site data (source) to a secondary site (target). If the production site becomes inoperable, SRDF
enables rapid automatic failover to the secondary site, allowing critical data to be instantly
available to the business operation. SRDF mirroring of data is transparent to the host operating
system and host applications. It does not require additional host software as the Symmetrix
systems manage all SRDF functions.

SRDF requires a minimum of two Symmetrix systems-one source system (R1) and one target
system (R2). Additionally, there may be one host server sending information to multiple target
systems and/or multiple hosts sending information to one target. SRDF requires a minimum of
two Remote Link Directors (RLDs) for ESCON, or Remote Fibre Directors (RFDs) for Fibre
Channel, to be installed in both Symmetrix systems, source and target, for path redundancy.

SRDF offers differing solutions to meet all business needs:

Campus solution

Extended distance solution

Adaptive copy solution

Please refer to the “Symmetrix Remote Data Facility (SRDF) Product Description Guide” for details.

SRDF Campus Solution SRDF campus environments, deﬁned as source and target up to 60 kilometers apart, will
primarily use synchronous mode. In this mode of operation, Symmetrix maintains a real-time
mirror image of the data of the remotely mirrored volumes. This conﬁguration ensures that
data on the source (R1) volumes and the target (R2) volumes are always fully synchronized at
the completion of an I/O sequence. SRDF Campus implementations can be either uni-
directional or bi-directional.

The sequence of operations is:

1. An I/O write is received from the host/server into the global memory of the source (R1).

2. The I/O is transmitted to the global memory of the target (R2).

3. A receipt acknowledgment is provided by the target back to the global memory of the source.

4. An ending status is presented to the host/server.

5. Each Symmetrix system destages writes to disk as described in previous sections.

4
Synchronous Mode

1 3

2
SRDF
Links

Source Target

33


SRDF Extended In extended distance environments, SRDF primarily operates in semi-synchronous mode. This
Distance Solution mode of operation supports only uni-directional SRDF conﬁgurations, and data on the
remotely mirrored volumes is always synchronized between the source (R1) volume and the
target (R2) volume prior to initiating the next write operation to these volumes.





4. A receipt acknowledgment is provided by the target back to the global memory of the source.


2
Semi-Synchronous Mode

1 4

3
SRDF
Links

Source Target

SRDF Adaptive Copy SRDF adaptive copy mode is used primarily for data migrations and data center moves. SRDF
adaptive copy mode allows the source (R1) volumes and target (R2) volumes to be few or many
I/Os out of synchronization. The number of tracks out of synchronization (skew) is user selectable.




3. I/O is placed in the SRDF queue in R1 global memory.


5. A receipt acknowledgment is provided by the target back to the source.


7. The next I/O in the SRDF queue is processed.

34


2 Adaptive Copy
Mode

1
6

5
SRDF
Links

Source Target
3&4

SRDF Error Recovery In the event of a loss of a disk drive at the source (R1) site, read/write I/O operations for the failed
drive will be performed entirely at the target (R2) site. The Symmetrix system notiﬁes the host
operating system of the disk error and alerts the EMC Customer Support Center via an Auto-Call
action. The EMC Customer Support Center product support engineer (PSE) then begins the
diagnostic process, and if necessary, dispatches a customer engineer (CE) to the customer site. Once
the suspect disk is non-disruptively replaced, the Symmetrix system re-establishes the mirrored pair
and automatically resynchronizes the data with the new disk. During the data resynchronization
process, the Symmetrix system gives priority to host I/O requests over the copy I/O to minimize the
impact on application performance and user service. Even though catastrophic disk failures are
uncommon, it is highly recommended that the source (R1) volume be locally mirrored in the event
that a drive failure occurs.

SRDF Multi-hop Multihop enables only data that has changed since the last update to be mirrored on a
Symmetrix system in a third location. ControlCenter Symmetrix Data Mobility Manager and
the Symmetrix Automated Replication feature, both of which implement an automated SRDF
multi-hop capability, are ideal for cost effectively mirroring data over long distances. By
copying only the changed tracks, less bandwidth is consumed, performance is enhanced, and
transmission times are shorter. Multi-hop mirroring to the third site can take place during off-
peak times over lower transmission lines making long distance mirroring more affordable.
Recurrent monthly line costs can be dramatically reduced, quickly recouping the investment in
a third Symmetrix system.

35


SRDF Advantages • Host-independent, realtime data mirroring solution for mainframe, UNIX, Windows NT,
Linux, Windows 2000, and iSeries system

• Requires no host server resources

• Replicates data over Virtual Private Networks using Internet Protocol (IP)

• Supports high-speed Fibre Channel including fan-in and fan-out source and target relationships
through Connectrix family switches

• Recovers business data and relieves disruptive outages in minutes

• Achieves highest distance performance and communication line efficiencies through multi-hop
capability and SRDF FarPoint

• SRDF transparent integrates into a GDPS environment

Symmetrix Dynamic Sparing Symmetrix systems can provide Dynamic Sparing, an additional level of protection for volumes
that use the Symmetrix mission-critical data protection schemes: Mirroring; Parity RAID;
and/or SRDF. This user-selectable option is capable of providing dynamic reallocation of data to
a standby spare disk drive, thus maintaining data protection in the event of disk failure. A small
pool of spare disks is committed to this option. All that is required operationally is to select the
Dynamic Sparing option during initial Symmetrix system configuration and to reserve the
necessary spare disk drives. The entire Dynamic Sparing process requires no intervention from
customer personnel as it is completely implemented in Symmetrix Enginuity SOE.

Disk
Director Data Data Data Data Data Spare

Disk
Data Data Data Data Data Data
Director

Disk
Data Data Data Data Data Spare
Director

Since errors are usually detected by Symmetrix 8000 systems well in advance of an actual disk
failure, Dynamic Sparing has proven itself to be very effective at being able to copy operational
data to a spare drive prior to that data becoming unavailable on the failing drive. When the
error threshold is exceeded on a disk, data is immediately copied from the failing disk to the
spare disk. Priority is given to host I/O requests during data copying, so high application
performance is maintained. With EMC’s RAID 1 mirroring implementation, a unique feature
copies data from the “good” disk to the spare disk when the error threshold is exceeded, rather
than copying from the failing disk. The spare and original disks then operate as a mirrored pair,
providing additional data protection until the failing disk is replaced.

When the copy operation is complete, notification of the occurrence is made to the EMC
Customer Support Center via an Auto-Call event. The local customer engineer will then
perform a non-disruptive replacement of the failing disk drive. When the physical replacement
is finished, data is dynamically copied from the spare to the new disk in the original location.
The spare remains in use until the copy completes and is then returned to the spare pool,
standing by and ready should another disk drive fail at some time in the future. Because data
volumes are fully protected once Dynamic Sparing is invoked, the disk replacement and re-
synchronization may be deferred to a time convenient to the customer. Throughout this
process, continuous data availability is provided to users and applications without disruption.

36


Symmetrix Dynamic Sparing • Increases protection of all volumes from loss of data
Advantages
• Automatically activates the spare volume without interruption prior to loss of access of a poten-
tially failing volume

• Ensures that the spare copy is identical to the original copy

• Resynchronizes a new disk device with the dynamic spare after repair of the defective device is
complete

• Increases data availability of all volumes in use without loss of data capacity

• Dynamic sparing is transparent to the host and requires no user intervention

37


Chapter 4
Symmetrix Reliability, Availability, and Serviceability
(RAS) Features

EMC Design and EMC’s design philosophy has always been to design-in maximum reliability and then to
Maintenance Philosophy implement the design with the most reliable components available. This philosophy continues
with the Symmetrix 8000-series Enterprise Plus Storage products. The goal for Symmetrix
products is to address all possible aspects of systems operation that contribute to providing
continuous data availability to allow continuous business operation. Once the design and
component selection are complete, the reliability focus continues with Design Verification
Testing (DVT), Highly Accelerated Life Testing (HALT), and Ongoing Reliability Testing
(ORT) to assure customers of an inherently highly reliable product at all times. EMC also
employs extensive leading-edge Environmental Stress Screening (ESS) techniques to detect
possible early life component failures well before any Symmetrix system is delivered to the
customer site.

Building upon this foundation of designed-in reliability and highly reliable components, the
architecture of the Symmetrix focuses on redundancy, so that data availability is assured even
in the unlikely case of a component failure. In addition to redundancy in data paths and data
path components, as previously described, this philosophy continues in all the major
operational units, providing backup should a component failure occur.

Symmetrix has full state-of-the-art self-monitoring, self-diagnosing, and where possible, self-
repairing algorithms. The objective of this philosophy is the avoidance of user-observable
errors. Symmetrix will actively identify internal temporary errors that could potentially lead to
any type of user-observable hard failure and attempt to correct them prior to data being
unavailable to a user or an application. This error avoidance is accomplished through a process
of error detection, error logging, and notification.

EMC Remote Support EMC has a long tradition of providing seamless remote support where we can maintain the
health of our systems and troubleshoot them as needed with experts throughout EMC. The
evolution of the technologies employed in EMC’s remote support has continuously evolved and
improved over time to provide rich diagnostic support functionality.

Typically a remote maintenance session is initiated by a Symmetrix call home. The call home is
an automatic event that is initiated when the Symmetrix service processor detects a condition
that meets the guidelines established by EMC Engineering for warranting further investigation.
SymmRemote instructs the service processor to call an EMC Customer Support Center. The
call is answered by an auto-attendant. Call detail files do not contain any customer data.

Once the EMC remote support center analyzes the call detail file and determines the best course
of action, and only if additional investigation is required, an EMC Customer Service,
Engineering, or Systems Engineering professional is instructed to connect to the designated
Symmetrix and pursue diagnosis and remedy.

Secure Network (SymmIP) SymmIP is an infrastructure and methodology that combines the power of the EMC support
network with hardware components to deliver a secure private conduit for remote
maintenance activities or traffic to protect the customer environment.

38


SymmIP provides the encrypted virtual private session between a Security Server and the
customer-based service processor. Note that this is an additional layer of security on top of the
end-to-end protection already provided by SymmRemote. Using this methodology, all remote
connections with the service processor are secured using Public Key technology.

Redundant Power Subsystem The Symmetrix 8000 has a modular power subsystem featuring a redundant architecture that
facilitates field replacement of any of its components without interruption to processing. Three
power supplies ensure power to the subsystem. The redundancy starts with the power connections:
two dedicated or isolated AC power cords. If AC power fails on one AC line, the power subsystem
automatically switches to the other AC line to provide continuous operation.

Three AC/DC power supply modules operate in a redundant parallel, or load-sharing,
configuration. If any single power supply fails, there is sufficient capacity in the remaining
power supplies to maintain full operation until a non-disruptive repair can be made to the
failed component.

Three DC/DC power supply modules operate in a similar redundant configuration. Symmetrix
senses any failure in a power supply component and reports errors to both the host system and
to the EMC Customer Support Center.

The entire Symmetrix system is made nonvolatile via an onboard battery backup subsystem. In
addition to providing non-volatility to the Symmetrix system, the battery subsystem is fully
capable of maintaining normal Symmetrix operation for a period of over three minutes. This
window allows Symmetrix to provide non-stop operation in the event of short power outages
or fluctuations in DC power. Symmetrix will continue to accept host I/Os during this period.

If normal power is not restored after three minutes, Symmetrix will return a Device Not Ready
condition for all devices to all connected hosts. Symmetrix will then destage all write tracks in
cache currently waiting destage and then perform an orderly shutdown. An orderly shutdown
is a condition where the heads on the disk drives are properly retracted and the drives are spun
down and powered off, eliminating emergency power off situations and extending the useful
life. Should AC power be restored prior to the Symmetrix being powered down, the Symmetrix
becomes immediately operational without requiring a system restart.

Enhanced Battery Testing Batteries are constantly recharged and load tested periodically to ensure that backup power
Procedures will be available if needed. In conjunction with the battery test, a fully comprehensive pre-test
of the Symmetrix power subsystem is carried out automatically. An enhanced battery test
thoroughly verifies the battery’s condition. Load tests ensure the Symmetrix will be fully
operational for the graceful destage and power down if required. The batteries are capable of
being hot swapped if necessary.

Dual Initiator Feature Symmetrix 8000 has a dual initiator feature that ensures continuous availability of data in the
unlikely event of a Symmetrix disk management hardware failure. This feature works by
having two Disk Directors each ‘shadow’ the function of the other. Under normal operation,
each Disk Director services its own disk drives. If Symmetrix detects a disk management
problem, each Disk Director has the capability of servicing any, or all of the devices of the
Director with which it is paired, should either Disk Director be unable to partially or fully
service its own devices. When the source of the failure is corrected, Symmetrix returns the I/O
servicing of the two Disk Directors to its normal state.

39


Dual initiation also provides an additional level of data availability in mirrored configurations.
Normally, if Symmetrix is unable to read from, or write to one of the disks in a mirrored pair,
Symmetrix automatically uses the other disk in the pair. If Symmetrix fails to communicate
with that device, it will transfer access to the volume to the alternate path provided by the dual
initiator function.

Non-disruptive Symmetrix, with its redundant architecture, supports non-disruptive replacement of many of
Component Repair its components. The Field Replaceable Units (FRUs) of Symmetrix include: Channel Directors,
Global Memory Cards, Disk Directors, disk drives, Power Modules (AC/AC, AC/DC, AC
input), batteries, and cooling fans.

This non-disruptive replacement capability allows the EMC customer engineer to install a new
component, initialize it if necessary, and bring it online without:

• Disrupting access to the affected data volume

• Powering down the Symmetrix system

• Stopping the operating system

• Taking the affected channel path offline

• Taking devices offline (except for the affected device)

Non-disruptive As customers continue to implement Symmetrix and EMC Enterprise Storage as the
Microcode Upgrades foundation of their information infrastructures across the enterprises, the ability to provide
non-disruptive hardware and software upgrades has become a critical feature to achieving 100
percent data availability and providing true business value.

40


Load new Enginuity upgrade

Load in-family Enginuity upgrade

SCSI Code Upgrade
- Change System-IDs
- Change Configuration Flags

Change Offline Director Flags
- Change Directors Configuration
- Change Volumes Configuration
- Add new logicals to existing physicals online
- Add new Physicals
- Changing from Normal volume to BCV
- Changing from BCV to Normal volume
- Changing from a Normal volume to a DRV
- Changing from a DRV to a normal volume
- Convert from Mirrors to Raid-S
- Convert from Raid-S to Mirror (additional drives required)
- Changing from Host emulation of SO to S

Change Meta Volumes Configuration

- Change Host

Change Volumes MIGRATION status
- Add RDF to Non-RDF system
- Add and remove devices to the RDF link.
- Swap RDF volumes and resync.
- Change RDF assignments.
- Online Add/Remove empty RDF group.
- Remove RDF to Non-RDF box (currently requires a 2 step process)
- Increase memory size
- Upgrade Eprom

Symmetrix Non-disruptive More than 50,000 non-disruptive upgrades have been carried out on Symmetrix systems in the
Enginuity Upgrade Procedure past two years.

Enginuity upgrades, performed by the EMC product support engineers (PSEs) at the EMC
Customer Support Center, provide enhancements to performance algorithms, error recovery and
reporting techniques, diagnostics, and microcode fixes. The Symmetrix system does not require
manual intervention on the customer’s part to perform this function. All Channel and Disk
Directors remain in an online state to the host processor, thus maintaining application access.

41


The following illustration shows the Symmetrix in-family non-disruptive microcode upgrade
procedures, highlighting the load process for one Director. In the actual non-disruptive upgrade
implementation, all Symmetrix Channel and Disk Director microcode is updated
simultaneously.

Note: Family-to-family non-disruptive Enginuity upgrade procedures include the ﬁve in-family
upgrade procedures shown, with four additional steps for updating the Symmetrix global
dynamic allocation table information.

2
3
Microcode loaded from
Status of Ports
global mailbox to
Set to Busy
EEPROM
4
A EE MP
Microcode loaded
from B CS
EEPROM to
Control Store A EE MP 1
B CS New microcode
loaded from
service processor
5 into global
Status of mailbox
Ports
Cleared
USD4 USD4 USD4 USD4 FCD2 FCD2 RLD4 RLD4
Top – High
Top – Low
Bottom – High
Bottom – Low Cache

DD DD DD DD DD DD DD DD

During a non-disruptive microcode upgrade:

1. The EMC PSE downloads the new microcode to the service processor. The new microcode
loads into a global mailbox via an Ethernet connection.

2. The new microcode is distributed to each Director’s EEPROM from the global mailbox.

3. Status is set to busy for ports controlled by each Director.;

4. Symmetrix will load executable code as selected “windows of opportunity” within each
Director until all have been loaded.

5. Once the executable code has been loaded in each Director, the busy status of the Director’s
ports is cleared, internal processing is synchronized, and the new code becomes operational.

42


Chapter 5
Additional Symmetrix 8000 Mainframe-Class Features
Symmetrix systems provide high performance and high functionality for I/O processing, not
only to the latest z/OS versions of mainframe operating systems, but also to non-traditional
mainframe operating systems and non-current versions of MVS, VM, and VSE. Virtually every
System/370 and System/390 operating system can be supported, including MVS/ESA
MVS/XA, MVS/SP, ACP/TPF, VM/ESA, VM/XA, VM/SP, VM/HPO, VSE/ESA, VSE/SP,
MVT/VSE, AIX/ESA, OS/390, and z/OS.

Enterprise Storage Platform In IBM/PCM mainframe environments, all Symmetrix systems are operating system
independent. The caching algorithms are self-managed, and Symmetrix 8000 systems do not
depend on host cache commands to receive the benefits of read and write caching. This means
that when Symmetrix ESP software is installed on a Symmetrix system, simultaneous
connections for mainframes, UNIX, Linux, Windows NT, and AS/400 (iSeries) systems are
provided. This specialized software enables combinations of serial ESCON and FICON
Channel Directors, Ultra SCSI Channel Directors, and Fibre Channel Directors on the same
Symmetrix system. For configuration flexibility, these Directors can be installed in combination
in the Symmetrix 8000, facilitating the concurrent storage of mainframe and open systems data
in the same system.

Symmetrix systems with ESP appear to mainframe operating systems as a 3990 or 2105. The
physical storage devices can appear to the mainframe operating system as a mix of multiple
3380 and 3390 devices. All models of the 3380 or 3390 volumes can be emulated up to the
physical volume sizes installed. A single Symmetrix system can simultaneously support both
3380 and 3390 device emulations.

The Symmetrix responds to cache commands from the host processor and will respond as a
3990 or 2105, but will not always perform the command in exactly the same manner as a 3990
or 2105. Some host access methods are designed to turn off cache during sequential processing.
This is necessary with conventional cached controllers as their caching algorithms create cache
pollution when processing sequential I/O. The sequential prefetch capability of Symmetrix
allows for efficient sequential operation without having to actually turn off Symmetrix cache.
This allows the Symmetrix to provide the high performance of an integrated cached
environment 100 percent of the time, while the host operating system perceives that cache has
been turned off.

The Symmetrix emulation of the IBM 3990 or 2105 allows it to be compatible with IBM’s
Systems Managed Storage (SMS) and other data management systems. Symmetrix knows how
data is being accessed and will manage its own caching and prefetch processes accordingly.
EMC cache management algorithms select which channel commands to process and which to
ignore for greater efficiency and performance.

43


Parallel Access Volume COMPAV is EMC’s compatible implementation of IBM’s Parallel Access Volumes (PAV). Without
(COMPAV) PAV, access to a volume is limited to one I/O at any one time by the Unit Control Block (UCB). So,
if two or more applications want to issue I/Os to a volume at the same time (or multiple
applications want to issue an I/O to a volume before the current I/O is completed), the second I/O
has to wait because the UCB is being used by the ﬁrst I/O. PAV introduces an alias UCB that also
points to the same volume. If the “base” (original) UCB is being used by a preceding I/O, then the
next I/O can use the alias UCB to access the volume. With this implementation, restrictions still
apply. If a write (update) is taking place at the extent (or track) where the second I/O wants to read,
then the second I/O still has to wait for the ﬁrst I/O to complete.

44


Multiple Allegiance (MA) Multiple Allegiance (MA) is a similar control unit capability to process non-conflicting I/Os
from different systems in parallel. The only requirement to exploit MA is to define the device to
multiple systems. Multiple Allegiance I/O executes concurrently with PAV I/O. The Symmetrix
treats them equally and guarantees data integrity by serializing write I/Os where extent
conflicts exist.

Dynamic Parallel Access Enginuity provides support for dynamic Parallel Access Volumes (PAVs). This feature allows
Volumes fewer aliases to be defined within a logical control unit. With dynamic PAV, aliases are applied
to the base devices that need them the most. This enables the MVS Workload Analyzer (in
Goal Mode) to assign an alias to a device “on the fly.”

IBM ESS 2105 Channel Symmetrix supports the IBM ESS 2105 channel command structure and I/O Priority Queuing
Command Emulation that is required to support EMC’s COMPAV/MA.

Multi-System Imaging Symmetrix supports multiple z/OS or System/390 environments through use of its 3990 or
2105 emulation modes. Symmetrix systems support up to 256 SSIDs providing a maximum of
8,000 logical devices per Symmetrix system. Consistent with IBM and PCM equivalents, up to
eight-path connectivity may exist to any single device within the Symmetrix configuration.

Sequential Data Striping Symmetrix family systems are fully compatible with IBM’s Sequential Data Striping function
for 3990 Model 3 and 3990-6 with Extended Platform in the ESCON environment.

Sequential Data Striping automatically distributes data to balance the workload across disks. It
also provides fast execution on large I/O bound sequential processing requests by allowing I/O
operations to be managed in parallel across as many as 16 devices. The Symmetrix system
handles the smaller blocks of data provided by Sequential Data Striping by performing up to 32
concurrent I/Os over multiple paths.

Sequential Data Striping is available only with DFSMS/MVS (Data Facility Storage
Management Subsystem) with storage management active. Symmetrix must be emulating 3990
or 2105 and running the appropriate level of Enginuity microcode. It must be attached via
ESCON channels and have SMS-managed volumes.

45


Mainframe Systems The Symmetrix system enhances disk system functionality by supporting multiple logical
Hyper-Volumes volumes on each physical device.

The Hyper-Volume Extension feature has two usage options:
• Split-volume Capability - Allows up to 32 logical volumes on each Symmetrix physical disk
device.

• Extended Cylinder Addressing - Establishes a small logical volume at the end of physical disk
device for data requiring high performance on a small volume.

In the mainframe environment, the following IBM cache management software can be used
with Symmetrix volume level cache statistics:

• Resource Management Facility (RMF)

• VM/Monitor and VM Performance Planning Facility (VMPPF)

• Cache RMF Reporter

Peer-to-Peer Remote Copy Enginuity EOS supports IBM Peer-to-Peer Remote Copy commands. PPRC is the synchronous
(PPRC) Emulation remote copying solution available with IBM Enterprise Storage Systems. PPRC is implemented as
a subset of the Dynamic SRDF feature. As a result, Symmetrix will support PPRC commands and
facilitate interaction with other PPRC systems in the framework of a Geographically Dispersed
Parallel Sysplex (GDPS).

FICON Support Symmetric supports a FICON director with two ports. FICON is a new protocol that enables
ESCON traffic to move over Fibre Channel connections.

This has several benefits:

• Removes the connection orientation inherent with ESCON, enabling multiple concurrent I/Os
on a single FICON channel

• Increases link bandwidth

• Sustains throughput (insignificant rate drop (up to 100 km)

• Relieves ESCON addressing limits from 1KB to 16KB Unit Addresses per FICON Channel

• Permits re-use of cabling plant with proper adapters and enables FICON and ESCON to co-exist

46


The idea behind FICON is to encapsulate the ESCON “logical” protocol on top of Fibre
Channel Physical Signaling Protocol (FC-PH). This is called FC-SB-2 and defines the mapping
of FICON and ESCON logical data. Designing FICON in this way enables high levels of legacy
ESCON code reuse.

Symmetrix RAID 10 (Mirrored To improve mainframe volume performance, Enginuity stripes data of a logical device across
Striped Mainframe Volumes) several physical drives. (The idea is analogous to meta volumes on open systems.) Four
Symmetrix devices (each a fourth the size of the original M/F device) appear as one M/F device
to the host, accessible via one channel address. Any four devices can be chosen to define a
group, provided they are equally sized, of the same type (3380, 3390, etc.), and have the same
mirror configuration. Striping occurs across this group of four devices with a striping unit of
one cylinder, as shown in the following diagram.

Intelligent Resource Director EMC Symmetrix fully supports the DCM portion of IRD through enhancements made to the
(IRD) Dynamic Channel Path Enginuity operating environment providing mapping of the appropriate control blocks
Management (DCM) required.

IRD is a new feature in z/OS V1R1 that extends the concept of goal oriented resource
management by allowing users to group system images that are resident on the same physical
server running in LPAR mode, and in the same Sysplex, into an “LPAR cluster.” This gives
workload management the ability to manage resources, both processor and DASD I/O, not just
in one single image but across the entire cluster of system images.

47


DCM lets workload manager dynamically move channel paths through the ESCON director
from one I/O control unit to another in response to changes in the workload requirements. By
defining a number of channel paths as “managed,” they become eligible for this dynamic
assignment. By moving more bandwidth to the important work that needs it, DASD I/O
resources are used much more efficiently. This may decrease the number of channel paths
needed in the first place, and could improve availability. In the event of a hardware failure,
another channel could be dynamically moved over to handle the work requests.

Dynamic Path Reconnection Dynamic Path Reconnection (DPR) permits the Storage Control Unit (SCU) to reconnect to the
Support (DPR) host on any available channel path between the device and the host system if the original
channel is busy with other operations. Without DPR, SCU waits for the original channel path
to become available again.

The DPR option must be invoked in an ESCON environment to facilitate reduction of director
port busy conditions. DPR must also be enabled when using extended platform functions, such
as IBM’s Concurrent Copy.

Host Data Compression Host Data Compression compatibility is provided on Symmetrix 8000 systems via implementation
of Sequential Data Striping support. The MVS instruction-driven data compression function is
supported on high-end air-cooled and water-cooled IBM mainframe processors.

Partitioned Data Set Search Symmetrix systems support IBM’s Partitioned Data Set (PDS) Search Assist feature for 3990
(PDS) Assist Model 3 or Model 6 with Extended Platform in serial channel for ESCON environments. PDS
Assist improves performance on large, heavily used partitioned data sets by modifying the
directory search process.

Multi-Path Lock Facility/ Symmetrix systems support the Multi-Path Lock Facility/Concurrent Access (MPLF/CA) for
Concurrent Access use with the ultra-high performance Airline Control Program (ACP) and Transaction
(MPLF/CA) Processing Facility (TPF) host operating system environments. MPLF/CA allows multiple
concurrent I/O requests to the same logical device from multiple TPF mainframes. The
Symmetrix system maintains the names and status of logical locks currently in use and
responds to requests to obtain or release a lock. This allows multiple hosts to share the same
data storage system through multiple paths in an active OLTP environment while maintaining
data integrity. MPLF/CA is an enhancement and replacement for the Extended Limited Lock
Facility (ELLF) and Limited Lock Facility (LLF).

48


Chapter 6
Symmetrix 8000 Family Software
Symmetrix provides centralized, sharable information storage that supports changing
environments and mission-critical applications. This leading-edge technology begins with
physical devices shared between heterogeneous operating environments and extends to
specialized software that enhances information sharing between disparate platforms.

Symmetrix systems improve the value of information by allowing users to consolidate storage
capacity for multiple hosts and servers. EMC offers powerful software to enable businesses to
raise service levels, lower operational costs, and accelerate time to market.

Automated Information AutoIS™ is EMC’s strategy for reducing storage management complexity in an open
Storage (AutoIS) environment. With new management applications and technologies, customers can automate
and simplify labor-intensive and inefﬁcient processes, in order to do more with less. AutoIS
draws upon EMC’s unprecedented investment in interoperability testing to create a simple,
singular business view of even the most diverse of storage systems.

Customers can unify disparate information storage resources into one seamless infrastructure-
to draw from the best of multiple vendors’ hardware, software, and connectivity devices. A
repository-based architecture lets applications share information on storage resources, policies,
performance, and availability.

WideSky Storage The industry’s ﬁrst storage management middleware, WideSky™ enables storage management
Management Middleware applications to manage offerings from multiple vendors. WideSky solves the problems of
higher costs, lowered service levels, and limited choices that have plagued IT managers who
work with multiple storage vendors. It masks the complexity of multi-vendor environments by
translating across any vendor’s storage software, systems, and connectivity devices so end users
don’t have to. The result: businesses manage all their storage assets from one point of contact,
boosting productivity and driving down costs.

WideSky is open to all software developers. With WideSky, developers can gain the necessary
foundation for building simple and automated products to meet customer needs. WideSky and
related technology can be leveraged to gain a common architecture for writing applications
that will work across heterogeneous network and storage products.

Information Management Software

EMC ControlCenter/ EMC ControlCenter/Open Edition is the most powerful and focused product for the
Open Edition centralized management of multi-vendor storage environments. From a single console,
customers can manage all of their storage platforms, networking devices, and server-based
resources. EMC ControlCenter/Open Edition draws on WideSky technology to populate an
Oracle-based repository with information from a range of storage, connectivity, and server
elements. It allows storage management applications to work in harmony, rather than in
competition for the time of costly IT managers.

49


Replication Manager Replication Manager is industry-first software for managing disk replications. EMC
ControlCenter Replication Manager obliterates the old assumption that the more replicas
made, the harder they are to manage. It discovers, catalogs, integrates, and automates between
disk replication, host applications, and external applications such as tape backup. It makes
replication simple and more powerful, automatically.

StorageScope EMC ControlCenter StorageScope™ is a highly flexible business storage reporting tool.
StorageScope eliminates the time-consuming, clumsy, and costly manual collection of
information. StorageScope allows users to see their entire storage infrastructure from a
business perspective, so they can allocate storage resources with established business processes.
By using WideSky’s middleware, StorageScope can probe a range of servers, SANs, and
attached storage devices to create a single view of a customer’s environment.

Symmetrix Manager Enables the customer to monitor the status and performance of Symmetrix systems, create
Symmetrix Logical Devices and Meta Devices, and modify device size and type from a single
console.

Symmetrix Optimizer Automates performance tuning of a Symmetrix system with an intuitive GUI for easy disk
tuning and data placement changes.

SymmEnabler™ Extends the superior performance and full benefits of an E-Infostructure to a variety of third-
party software applications.

SRDF/TimeFinder™ Manager Enables a customer to manually monitor, provision, and control data replicas and automate the
replication process.

Symmetrix Data Mobility Enables a customer to monitor, provision, and control data replicas in the replication process.
Manager (SDMM)

Database Tuner Enables a customer to report on realtime, recent, and historical information for capacity
planning, problem solving, and performance analysis. Supports Oracle and IBM DB2 UDB in a
Symmetrix environment.

PowerPath™ Integrates multiple path I/O capabilities, automatic load balancing, and path failover functions
for use on open server platforms connected to Symmetrix storage systems.

Resource Availability Simplifies and automates storage resource management across the enterprise. Monitors
operating systems, databases, tape systems, and backup applications; automates host storage
resource management; and reports on the status and usage of storage resources.

ESN Manager Can be used in combination with ControlCenter to actively control SAN management
functions such as zoning and LUN masking. Features integrated storage network discovery,
topology, and alert capabilities.

50


ResourcePak® for Windows Provides integration and functionality enhancements to gain the most value from Symmetrix
systems.

EMC Double Checksum Provides an end-to-end safeguard against data corruption by re-checking Oracle’s data
validation before data is written to EMC Symmetrix systems.

Information Protection EMC information protection software protects information and increases productivity while
Software driving down the cost of storage.

SRDF Duplicates production site data on one or more physically separate target Symmetrix systems
regardless of location.

SRDF/Data Mobility (DM) Replicates or moves data from one Symmetrix storage system to another without any impact
on server or application cycles.

TimeFinder Creates, in background mode, independently addressable local mirror images of active
Symmetrix production volumes for running simultaneous tasks in parallel.

Symmetrix Data Migration Provides end-to-end management of an entire data migration process, including planning,
Services (SMDS) implementation, and post-migration reporting and testing.

GeoSpan™ MSCS Combines SRDF software with Microsoft Cluster Server to enable cluster operations to
continue following a site disaster.

GeoSpan VCS Combines SRDF with VERITAS Cluster Server to enable cluster operations to continue
following a site disaster.

CopyPoint™ Allows AS/400 systems to provide virtually uninterrupted 24x7 production-level support for
an enterprise, while enabling backup protection.

CopyCross™ Copies mainframe tape data to Symmetrix disk storage for enhanced information availability
and protection.

EMC Data Manager (EDM) EDM( combines software, hardware, and services to provide a centralized, high-performance
backup and restore system optimized for Symmetrix-based and distributed UNIX and
Windows NT database environments.

51


EMC Fastrax™ Data Moves backup/recovery data between Symmetrix systems and industry-standard tape libraries.
Movement Platform Backs up, restores, and provides fast data recovery of Oracle and SAP/R3 information residing
on Symmetrix systems in HP-UX and Solaris operating systems using HP OpenView
OmniBack II.

EMC SymmEnabler™ Increases the functionality and performance of select partner applications using EMC
Application Programming Interfaces.

EMC Foundation Suite & Enables seamless integration with other EMC and VERITAS products to bring mainframe-
Database Edition for Oracle class manageability to open systems data.

Information Sharing Software EMC information sharing software instantly draws on timely information across an
infrastructure for better decision making.

InfoMover Transfers ﬁles bi-directionally between any combination of mainframe, UNIX, or Windows
systems using Symmetrix systems and existing I/O channel connections.

Enterprise Storage Platform Enables simultaneous mainframe (ESCON and FICON) and open systems (UNIX, LINUX,
and Windows NT/2000) connectivity to the same Symmetrix system.

52


Chapter 7
EMC Global Services
EMC Global Services delivers a network of services that enables customers to reap the full
benefits of their EMC E-Infostructures. This network provides a continuum of best-in-class
services that support customers through their entire information lifecycles.

EMC Powerlink EMC Powerlink offers continuous value for customers and partners. It offers a 24x7
connection to product and technical information; online services and support; training and
certification programs; and collaboration with product specialists. Hosted by EMC’s Internet
Solutions Group, EMC Powerlink showcases EMC products and services at work in a living e-
business setting, deepening interactive online relationships between EMC and its customers
and partners.

Professional Services As a company, EMC has focused, information storage experience, unparalleled in the industry.
Within Global Services, the Professional Services organization represents more than 1,200
Professional Services experts, boasting an average of 20 years of industry-related expertise
overall. EMC has categorized our areas of focus to areas we know are important to our
customers. Each solution set includes multiple offerings ranging from consulting, planning,
and design services, to consolidation, migration, and operations management.

Operations Management EMC Professional Services analyzes a customer’s current testing processes and methodologies.
Consulting Based on the customer capabilities, Professional Services recommends improvements in the
areas of human resources, process, and technology. EMC Professional Services consultants can
also be engaged to go onsite at the customer’s location to augment in-house staff. These expert
consultants facilitate knowledge transfer to the customer’s staff until the staff is properly
trained and able to manage the environment on their own.

An Operations Management Consulting engagement consists of the development of a storage
management strategy. This includes defining the best approach to managing infrastructure
operations, developing processes and metrics, and the selection, implementation, and
integration of storage management products.

Operations Management Consulting offerings include:

• Operations Management Planning

• Operations Management Design and Implementation

• Operations Management Support

53


Information Storage During an Information Storage Integration engagement, Professional Services presents a
Integration customer with a comprehensive storage systems strategy, based on the unique information
gleaned from an in-depth consultation. From there, the conceptual storage architecture and
design is identified, and a plan for the installation and integration of storage infrastructure
products is created. Information Storage Integration services assists a customer in building a
storage architecture foundation to meet current and future business requirements. As
customers continue to focus on decreasing their storage costs, they will typically experience
increasing data requirements driven by new applications and increasingly complex storage
architectures. Members of the EMC Professional Services team are experts in managing and
protecting information storage, and can give the knowledge and assistance needed to meet
those challenges.

Information Storage Integration offerings include:

• Storage Infrastructure Strategy and Planning

• Storage Infrastructure Design and Implementation

Information Storage EMC Professional Services provides all services, including project management, for a full data
Consolidation center migration project. This engagement uses an EMC best practice methodology called
“Eccelerate,” which includes risk analysis, planning and design, implementation, and
management. There are two offerings Professional Services will deliver in an Information
Storage Consolidation engagement:

• Information Storage Consolidation Planning develops and delivers a strategy and architectural
approach that addresses consolidation needs at a variety of levels. The strategies range from spe-
cific application consolidations to enterprise wide multi-environment situations. The Profes-
sional Services consultants delivering this offering review the current storage architecture and
technology and identify current and future storage requirements for scalability and opportunities
to leverage operational efficiencies. An information storage consolidation strategy and architec-
ture is developed to support both customer business objectives and IT requirements. Technical
components are recommended, and high-level deployment and migration plans are developed.
Overall cost and ROI are calculated and a formal business case is proposed to support the con-
solidation effort.

• The Data Migration offering provides the detailed planning and physical migration of data from
one storage environment to another. This effort could involve a small- to medium-data set migra-
tion that is moving data to a new storage base or a full data reorganization in a data center.

Business Continuity The Business Continuity services that EMC Professional Services consultants offer include the
development of a comprehensive plan for data availability-the availability of storage assets as
part of a business continuance initiative. Included in the plan is a strategy and tactical details on
the implementation of data replication devices and hierarchical storage technologies.
Customers receive a proposal based on the unique needs of their businesses that reveals the
financial impacts of not having an effective, business continuity plan in place.

EMC understands the challenges customers face delivering real-time data, and we also acknowledge
the need to reduce cycle time for backups. EMC provides comprehensive business continuity
solutions that meet multiple levels of availability and business continuance requirements. EMC
Professional Services has the experience and expertise to help determine the right solution for each
customer. Moreover, we’ll implement that solution quickly and cost-effectively.

54


Customer Service Customer Service at EMC starts with highly qualified and dedicated EMC engineers well
trained on EMC equipment. Each customer is assigned a primary and secondary customer
engineer. EMC’s world-wide customer account database contains all information about a
customer’s account, which customer engineers can readily access.

Proactive and Pre-emptive EMC Customer Service delivers 24x7 global support with a Global Services network of over
Support 5,000 technical, field, and support personnel. EMC’s proactive approach to support means
problems are addressed and eliminated before they occur. Remote support provides
notification when there’s a problem-or a potential problem. If onsite service is required, our
field staff can “hot swap” a part without system downtime. Our parts depots also are
accessible 24x7.

Remote Support EMC Symmetrix systems are equipped with automatic “phone-home” capabilities, so our
service experts monitor a system 24x7x365. And by dialing back into the EMC system, we take
action quickly, analyzing events and abnormalities, and resolving most issues before they affect
business. Our highly advanced remote support means we can offer a proactive and pre-emptive
approach that’s unmatched in the industry.

Software Support An all-inclusive, unparalleled software support and maintenance program ensures optimum
availability of mission-critical information. Our software specialists provide 24x7 telephone
support to meet the needs of the most complex multivendor environments. And our e-services
make information, solutions, and software upgrades instantly accessible.

Change Control Our industry-leading change control process enables customers to take advantage of the
outstanding connectivity, flexibility, and upgradeability engineered into every EMC Symmetrix
system. Our experts meticulously plan and orchestrate changes to the EMC solution-from
standard microcode upgrades to massive data center relocations.

Installation Support EMC specialists configure the Symmetrix 8000 systems according to the customer’s
specifications and requirements. During installation, Customer support engineers and
installation specialists install and configure Symmetrix systems based on business
requirements; create file systems and set access rights, as required; export file systems to the
network; mount file systems on individual machines; and provide channel and network
connectivity.

Post-sale Warranty and Coverage of the Symmetrix system includes EMC’s basic two-year hardware and 90-day
Product Support software warranty plan, with 24-hour, 7-day-a-week coverage. Post-warranty service offerings
include 24x7 coverage, technical support, and service and maintenance contracts.

55


Worldwide Organization, The EMC Customer Support Center, headquartered in the United States, directly supports
Local Support EMC hardware and software products. Use the following numbers to contact EMC and obtain
technical support:

U.S.: (800) 782-4362 (SVC-4EMC)

Canada: (800) 543-4782 (543-4SVC)

Worldwide: 1 + (508) 497-7901 (or contact the nearest EMC office)

Global Technical Training EMC Global Technical Training delivers ongoing technical education that gives customers the
knowledge they need to use their E-Infostructures to a competitive advantage.

Educational Services Both elearning and traditional instruction are available. Our Web-based program offers access
to training whenever it is convenient.

The EMC Proven Professional The EMC Proven Professional Certification Program is aligned with other IT industry
Certification Program certification programs, notably Microsoft and Cisco. Students can achieve an Associates or a
Masters level of certification in the Proven Professional program. Four tracks are offered, based
on IT job roles(Operator, Builder, Architect, and Instructor.

• Operator: manage data center operations

• Builder: implement and integrate data centers

• Architect: design enterprise storage networking solutions

• Instructor: knowledge transfer of E-Infostructure

E-learning EMC E-learning incorporates online learning into the suite of training, education, and
certification solutions available to customers, partners and employees.

56

EMC Symmetrix

Where Information Lives www.EMC.com

EMC Corporation
Hopkinton
Massachusetts
01748-9103
1-508-435-1000
In North America
1-800-424-3622, ext. 362

EMC2, EMC, MOSAIC:2000, ResourcePak, and Symmetrix are
registered trademarks and EMC ControlCenter, EMC Enterprise
Storage, EMC E-Infostructure, The EMC Information Orb,
AutoIS, CacheStorm, CLARiiON, Celerra, CopyCross,
CopyPoint, EDM, Enginuity, Fastrax, GeoSpan, HighRoad,
InfoMover, PowerPath, SDMS, StorageScope, SymmAPI,
SymmEnabler, TimeFinder, WideSky, and where information
lives are trademarks of EMC Corporation. Other trademarks
are the property of their respective owners.

© 2002 EMC Corporation. All rights reserved.
Printed in the USA. 05/02
C756.1

8000 guide

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to 8000 guide (20)

Recently uploaded (20)

8000 guide