Dr관련 세미나 자료 v2

Copyright © 2014 Juniper Networks, Inc.1 Copyright © 2014 Juniper Networks, Inc.
DISASTER RECOVERY AND
INTER-DC WORKLOAD MOBILITY
APRIL 2014
HJKIM@JUNIPER.NET

Copyright © 2014 Juniper Networks, Inc.2
연결된 세상의 힘
CONNECT EVERYTHING. EMPOWER EVERYONE.

AGENDA
 Business and technologies requirement
 Storage networking DCI design considerations
 Disaster Recovery and Workload Mobility DCI design
considerations
 Juniper Data Center Connectivity Solution

DISASTER RECOVERY DCI 솔루션:
사업적 요구 사항
 데이터 가용성
• Client to Storage
• Server to Storage
• Storage to Storage
• Measured by RTO and RPO
• Different backup/replication methods give different RTO/RPO as generally as they go down the cost of the method goes up
 재난 복구
 사업 연속성
• One of the critical function of over all BC planning
 정부 규제
• Storage availability and continuity of access
• http://www.drj.com/tools/tools/dr-rules-and-regulations.html
• HIPAA, SoX
 민첩성
• Leveraging DR sites for workload balancing

DISASTER RECOVERY DCI 솔루션: 기술적 요구사항
 IT 서비스의 성공적인 재개를 위한 필요 사항
• 데이터의 가용성
• Consistent data availability
• 어플리케이션/컴퓨팅 자원의 가용성
• Sufficient computing power and application software
• 네트워크 가용성
• For compute, data resource communication and user connections
• 데이터 세션 리다이렉션
• User’s request must be redirected to available application compute resources
• Data access requests must be redirected to available data store
 네트워크 요구 사항
• 블록 디스크 엑세스, 복제 타입, 거리 등에서 오는 고려 사항
• Low Latency
• High B/W
• No loss/low loss
• Reliable/Resilient

DISASTER RECOVERY DCI 솔루션:
사업적 요구 사항
 복구 목표 시간 (Recovery Time Objective)
 복구 목표 지점 (Recovery Point Objective)
Uptime Uptime 년간 최대 허용
다운 시간
Five nines 100.00% 5분 35초
Four nines 99.99% 52분 33초
Three nines 99.90% 8시간 46분
Two nines 99.00% 87시간 36분
One nine 90.00% 36일12시간
가용성 테이블
“Zero-time” / “Zero-loss”
RTO/RPO
높은 RTO 및 RPO 기준은
더 잘 디자인된 DCI 솔루션을
필요로함.
DECLARE
DISASTER
10 a.m.
RPO RTO
RPO: Amount of data lost from
failure, measured as the amount
of time from a disaster event
RTO: Targeted amount of time
to restart a business service
after a disaster event
5
a.m.
6
a.m.
7
a.m.
8
a.m.
9
a.m.
10
a.m.
11
a.m.
12
a.m.
1
p.m.
2
p.m.
3
p.m.
4
p.m.
5
p.m.
6
p.m.
7
p.m.

AGENDA
considerations

DATA CENTER들 사이의 데이터 가용성
 데이터 복제 기술들
• 동기식 복제 (Synchronous replication)
Any write request is acknowledged only after replication to target is acknowledged
• 비동기식 복제 (Asynchronous replication)
A write request is acknowledged as soon as write is complete at local data store, replication request is separately handled
• 스냅샷 복제 (Snapshot replication)
Similar to asynchronous replication, however the replication requests are handled in batches

세가지 데이터 복제 기술들
동기식 복제 비동기식 복제 스냅샷 복제
• Any write request is acknowledged
only after replication to target is
acknowledged
• "Zero-time” / “Zero-loss" RTO/RPO
Mostly across MAN distances
• Low Latency
• Guaranteed Bandwidth
 A write request is acknowledged
as soon as write is complete at
local data store, replication
request is separately handled
 WAN
 Higher Average throughput
 Similar to asynchronous replication,
however the replication requests
are handled in batches
 WAN
 Higher Average throughput

복제 네트워크 영향
 가용한 대역폭이 충분한 시간 동안 제공되지 못한다면,
어플리케이션은 손상되고 복구를 위한 사용자의 개입이 필요
할 수 있음.
 동기식 복제에서 필요 대역폭이 가용 대역폭 이상일 경우,
어플리케이션의 성능은 레이턴시의 증가로 인해 많이 떨어 질
수 있음.
 비동기식 복제는 어플리케이션 성능의 영향 없이 Data Rate을
부드럽게 허용할 수 있음.
 비동기식 복제는 변화를 모아 전달 하기 때문에 필요한
대역폭을 줄일 수 있음.

복제에 대한 접근 및 프로토콜
 원거리 전송 타입 (FC 또는 SAN 확장)
 FC (MAN distance only)
 iSCSI (WAN distance), iSCSI really more for Initiator to target. i.e. client/server system to storage.
 FCIP (WAN distance), FCIP predominantly used for bridging FC SANs over long distances
 iFCP, xWDM
 모든 SCSI의 전송 방법은 기기 사이에서 IP 네트워크를 통해 명령
호스트 기반 복제 어레이 기반 복제

ISCSI STACK이 FCIP STACK 보다 효율적
 ISCSI PDU는 ISCSI CDB (Control Descriptor Blocks) 옵션과 함께 세션
명령과 데이터를 포함
 iSCSI는 3260 포트 사용
 데이터 및 헤더 무결성을 위한 에러 체크 기능
 IPSEC 기능을 통한 보안 세션 및 데이터 사용
 FCIP PDU는 FCIP 세션 제어 정보를 포함
 FCIP는 3225 포트 사용
Data Link
IP
TCP
FCIP
Data Link
IP
TCP
iSCSI
Physical Physical
SCSI FCP
SCSI

ISCSI DCI 적용이 보다 단순
FC
Ethernet
Session
IP Network
iSCSI array to iSCSI array for replication
IP Network
FCIP gateway to FCIP gateway for replication
FCIP
Gateway
FCIP
Gateway
Array
Array
Array
Array

STORAGE OVER IP(TCP) 솔루션
TCP/IP chattiness/statefullness
Windowing
slow start
packet loss
QoS
B/W
 SACK
 Tuning TCP timers
 Compression
 WAN optimization
문제점 솔루션
대부분의 스토리지 기반 문제는 거리와 스토리지 특성에서 기인함

AGENDA
considerations

GbE/10GbE SERVERS
FC
STORAGE
Pooled
storage
iSCSI / NAS
Customer B - IT DC
SRX
MX
Junos Space
DATA CENTER 모델
Public Cloud Users
SMB
GbE/10GbE SERVERS
Pooled Storage
(NAS)
Production Data
Center A
MX
GbE/10GbE SERVERS
FC
STORAGE
Pooled
storage
iSCSI / NAS
Customer A - IT DC
SRX
MX
NAT
FW
LB
IPSec
Junos Space
Inter Data Center
Connectivity
Hybrid Cloud
Junos Space
SRX
GbE/10GbE SERVERS Pooled Storage
(NAS)
Production Data
Center B
MX
Hybrid Cloud VPN
NAT
FW
LB
IPSec
SRX
VPN
Junos Space
EX
QFX μF
Switch Switch
EX
QFX μF

DATA CENTER들 사이의 어플리케이션 자원 가용성
 고 가용 컴퓨팅 클러스터
 A group of compute resources providing compute services for an application
 Shared data source
 확장된 / 지리적 클러스터
 Members of compute clusters are located across DC sites
 Replicated data source across DC sites
 가상화된 컴퓨팅 클러스터
 Virtual machine mobility among compute clusters
 클러스터 인지 어플리케이션
 Application resiliency integrated with compute cluster environment

고 가용 컴퓨팅 클러스터
로컬 클러스터
Private
Network
Public Network
Compute
Cluster
Shared Disk
Site-1
지리적 클러스터
Private
Network
Public Network
Compute
Cluster
Disk-1
Site-1
Disk-2
Site-2
Data
Replication

가상화된 컴퓨팅 클러스터
 클러스터 환경에 가상 머신 사용
 클러스터 그룹내에서 가상 머신의 이동성에 의해 민첩성 향상
 다수의 네트워크들이 필요
 HA/DR의 능력을 올리기 위해 DATA CENTER들 간의 클러스터 구성
Live Migration Network
Heartbeat
Network
Public Network

고 가용성 컴퓨팅 클러스터 네트워킹
 결정적 레이턴시
 Heartbeat communication
 Virtual machine memory and state replication
 네트워크 처리 성능
 Virtual machine memory and state replication
 VM image transfer
 신뢰성 및 복원력
 To avoid Split brain conditions
 네트워크 트래픽 분리
 QoS guarantee
 Layer-2 연결성
 For single hop heartbeat
 For simpler user session redirection

네트워크 가용성
 네트워크 장비 고 가용성
 Redundant components
 Graceful Routing Engine fail-over
 Non stop routing/bridging
 Modular software
 Routing and Forwarding on Separate Planes
 네트워크 링크 고 가용성
 Link Aggregation
 Virtual Chassis
 Multi Chassis LAG
 Bidirectional Forwarding Detection
 Fast Reroute

데이터 세션 리다이렉션
 클러스터 멤버간 Layer-3 연결
 DR site is in different IP subnet
 May require GSLB setup
 DNS update with shorter TTL
 Higher convergence time and hence higher RTO
 클러스터 멤버간 Layer-2 연결
 DR site is in same IP subnet
 Target IP address is same after fail-over
 User session redirect based on forwarding update
 Target machine need to learn the default gateway mac address
 Faster convergence – possibility of no session loss

LAYER 2 확장
 VM/Workload 이동성
 지리적 클러스터/확장된 클러스터를 위한 고 가용성
 이점 : 신뢰성, Workload 이동
 문제점 : 클러스터 제어의 분리 이슈. 양 파트의 클러스터가 통신 할 수 없을 때, 공유
스토리지에 계속 쓸 경우 데이터의 이상이 발생 할 수 있음.
 L2 도메인의 문제가 WAN을 통해서도 내포
 대역폭의 비용
 고 가용성/클러스터 솔루션은 LAN을 위해 만들어 졌으므로 WAN 환경에서 잘 동작 하지
않거나 새로운 문제를 발생 시킬 수 있음.

VMOTION 네트워크 요구 사항
 Vmware는 리던던시 및 스토리지를 위해 최소 2개 이상의 인터페이스를 요구함.
 VMKernel – Clustering interface that is used for synchronization traffic (L2 requirement by
recommendation of VMware support)
 Data Interface – Assumes MAC and IP address of VM upon move (L2 requirement as we would
still be assuming the same segment for IP routing)
 거리 제한
 Distance limitation is based upon local function and memory paging synchronization goal for
‘no interruption of service’ (5ms ~100km)
 동기화를 위해 서버의 트래픽 로드가 크게 증가 할 수 있음

DATA CENTER간의 VMOTION 이슈
 각 DC에서 같은 클러스터와 호스트를 관리 하기 위해 같은 vCenter를 사용하여야함
 멀티티어 어플리케이션
 move one server need to move them all or need to still ensure communication to them all
 ESXi 호스트 당 대역폭 및 QoS
 move one VM or move all of them. Requires a lot of bandwidth no QoS differentiation between VMs on
vswitch/LAN – Bandwidth cost tends to be prohibitively expensive long distance remember the data is the
most important thing if you don’t shift this then many applications are useless
 방화벽과 로드 밸런서들의 세션 정보는 DC간 유지 및 공유가 필요 할 수 있음

AGENDA
considerations

MX는 DC LAN 및 WAN의 연결성 제공
WAN
LANMX
supporting
extensive set
of LAN
features
High scale,
multi-tenancy,
resiliency,
deployment
flexibility
Inline
services,
stateful
services
WAN / COREMX
providing
market
leading WAN
features
Proven platform
Over 24,000 chassis
shipped
Over $3B revenues
Over 2,500 customers
EDGE
COLLAPSED
CORE

DCI MX – L2와 L3 연결성 제공
 L2 와 L3 DCI로 표준 기술 사용
 고 확장성의 멀티태넌시 제공
SRX5800
EX4200
EX/MX
MX
Series
MX
SeriesRemote
Data Center
MX
Series
MX
Series Remote
Data Center
VPLS and L3VPN over
MPLS (or) IP
GbE/10GbE SERVERS
NAT
FW
LB
IPSec
SRX
Switch
GbE/10GbE SERVERS
 멀티 벤더 솔루션
 강력한 고가용성 기능과 빠른 복구 능력 제공
 테스트되어진, 적용되어진, 입증된 솔루션

MX를 통한 L2 DCI – VPLS
 특성:
 Forwarding of Ethernet Frames
 Forwarding of Unicast frames with an unknown MAC address
 Replication of broadcast and multicast frames
 Loop prevention
 Dynamic Learning of MAC address
IP of MPLS
Site 1 Site 2
MULTIVENDOR – PROVEN – FAST RECOVERING – HIGH SCALE
L2 CONNECTIVITY
VPLS
Edge
VPLS
Edge

Virtual Private LAN Service (VPLS)는 공유된 IP/MPLS 네트워크를
통하여 VLAN 확장을 제공함
VPLS 특성
Full Mesh
VLAN Separation
Provisioning
Multicast, Broadcast and
Flooding
Availability
Any-to-Any connectivity regardless of physical path
Separate VPLS instances per VLAN. Allows network-wide
segmentation with very large scale
New site Auto Discovery
Scale forwarding with Multicast & Point-to-Multipoint capabilities
Underlying MPLS offers ECMP, Fast Reroute

LAN
ETHERNET VPN 소개
LAG
 Ethernet-VPN은 새로운 표준 기반의 프로토콜
 L2 도메인을 MPLS 또는 IP 백본을 통해 연결
 멀티 벤더에서 지원
 Leverages BGP와 MPLS의 강점을 접목
 Policy based learning, advertisement for
controlled learning
 L3 aware L2 solution
WAN
BGP based
state exchange
EVPN
router
EVPN
router

PRIVATE MPLS WAN without EVPN
VLAN 10
DATA CENTER 1
VLAN 10
DATA CENTER 2
✕
MAC VLAN Interfaces
AA 10 xe-1/0/0.10
Router 1’s MAC Table
MAC: AA
Server 1
xe-1/0/0.10
xe-1/0/0.10 xe-1/0/0.10
xe-1/0/0.10
MAC: BB
Server 2
ge-1/0/0.10
ge-1/0/0.10
MAC VLAN Interfaces
BB 10 xe-1/0/0.10
ge-1/0/0.10
ge-1/0/0.10
PRE-EVPN: LAYER 2 STRETCH BETWEEN
DATA CENTERS Without EVPN
Data
Plane
• Only one path can be active at a given time
• Remaining links are put into standby mode
Control
Plane
• Layer 2 MAC tables are populated via the data plane
(similar to a traditional L2 switch)
• Results in flooding of packets across WAN due to
out of sync MAC tables

PRIVATE MPLS WAN without EVPN
VLAN 10
DATA CENTER 1
VLAN 10
DATA CENTER 2
MAC VLAN Interfaces
AA 10 xe-1/0/0.10
BB 10 ge-1/0/0.10
MAC: AA
Server 1
xe-1/0/0.10
xe-1/0/0.10 xe-1/0/0.10
xe-1/0/0.10
MAC: BB
Server 2
ge-1/0/0.10
ge-1/0/0.10
MAC VLAN Interfaces
BB 10 xe-1/0/0.10
AA 10 ge-1/0/0.10
ge-1/0/0.10
ge-1/0/0.10
With EVPN
Data
Plane
• All paths are active
• Inter-data center traffic is load-balanced across all
WAN links
Control
Plane
• Layer 2 MAC tables are populated via the control
plane (similar to QFabric)
• Eliminates flooding by maintaining MAC table
synchronization between all EVPN nodes
POST-EVPN: LAYER 2 STRETCH BETWEEN
DATA CENTERS

VM MOBILITY TRAFFIC OPTIMIZER
 By utilizing the control plane learning that EVPN offers, Juniper is able build upon that
technology and bring a significant set of enhancements for customers that are providing Layer
2 stretch but are plagued with the effects of a scenario commonly referred to as “Trombone
Routing.”
 This is a result of a VLAN being “stretched” to reside in two or more data centers but with only
one optimal way into the VLAN from the outside (IGP route preference) and one optimal way out
(single Master VRRP address).
 VMTO fixes both of these problems by:
1) Optimizing routing information sent to the WAN that is specific to the location of each VM
2) Ensuring every router within the VLAN has an active instance of the default gateway
 Ultimately, this allows a customer to provide a better experience to their end users while being
able to take advantage of all resources—efficiently and in multiple geographies.

Scenario with VMTO enabled
PRIVATE MPLS WAN PRIVATE MPLS WAN
VLAN 10 VLAN 10 VLAN 10VLAN 10
Scenario without VMTO
THE NEED FOR L2 LOCATION AWARENESS

DC 2
VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
VLAN 20
Server 2 Server 3
Server 1
PRIVATE MPLS WAN
DC 1
20.20.20.100/24
Active VRRP
DG: 10.10.10.1
Standby VRRP
DG: 10.10.10.1
Standby VRRP
DG: 10.10.10.1
Standby VRRP
DG: 10.10.10.1
Task:
Server 3 in Data Center 3 needs to send packets to Server
1 in Data Center 1.
Problem:
Server 3’s active Default Gateway for VLAN 10 is in Data
Center 2.
Effect:
1. Traffic must travel via Layer 2 from Data Center 3 to Data
Center 2 to reach VLAN 10’s active Default Gateway.
2. The packet must reach the Default Gateway in order to
be routed towards Data Center 1. This results in
duplicate traffic on WAN links and suboptimal routing –
hence the “Egress Trombone Effect.”
WITHOUT VMTO: EGRESS TROMBONE EFFECT

DC 2
VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
VLAN 20
Server 2 Server 3
Server 1
PRIVATE MPLS WAN
DC 1
20.20.20.100/24
Active RVI
DG: 10.10.10.1
Active RVI
DG: 10.10.10.1
Active RVI
DG: 10.10.10.1
Active RVI
DG: 10.10.10.1
Task:
Server 3 in Datacenter 3 needs to send packets to Server 1
in Datacenter 1.
Solution:
Virtualize and distribute the Default Gateway so it is active
on every router that participates in the VLAN.
Effect:
1. Egress packets can be sent to any router on VLAN 10
allowing the routing to be done in the local datacenter.
This eliminates the “Egress Trombone Effect” and creates
the most optimal forwarding path for the Inter-DC traffic.
WITH VMTO: NO EGRESS TROMBONE EFFECT

DC 2
VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
VLAN 20
Server 2 Server 3
Server 1
PRIVATE MPLS WAN
DC 1
20.20.20.100/24
Task:
in Datacenter 3.
Problem:
Datacenter 1’s edge router prefers the path to Datacenter 2
for the 10.10.10.0/24 subnet. It has no knowledge of
individual host IPs.
Effect:
1. Traffic from Server 1 is first routed across the WAN to
Datacenter 2 due to a lower cost route for the
10.10.10.0/24 subnet.
2. Then the edge router in Datacenter 2 will send the
packet via Layer 2 to Datacenter 3.
10.10.10.0/24 Cost 5
10.10.10.0/24 Cost 10
Route Mask Cost Next Hop
10.10.10.0 24 5 Datacenter 2
10.10.10.0 24 10 Datacenter 3
DC 1’s Edge Router Table Without VMTO
WITHOUT VMTO: INGRESS TROMBONE EFFECT

DC 2
VLAN 10
10.10.10.100/24
DC 3
10.10.10.200/24
VLAN 10
VLAN 20
Server 2 Server 3
Server 1
PRIVATE MPLS WAN
DC 1
20.20.20.100/24
Effect:
1. Ingress traffic destined for Server 3 is sent directly across
the WAN from Datacenter 1 to Datacenter 3. This
eliminates the “Ingress Trombone Effect” and creates the
most optimal forwarding path for the Inter-DC traffic.
Task:
in Datacenter 3.
Solution:
In addition to sending a summary route of 10.10.10.0/24
the datacenter edge routers also send host routes which
represent the location of local servers.
10.10.10.0/24 Cost 5
10.10.10.0/24 Cost 10
Route Mask Cost Next Hop
10.10.10.0 24 5 Datacenter 2
10.10.10.0 24 10 Datacenter 3
10.10.10.100 32 5 Datacenter 2
10.10.10.200 32 5 Datacenter 3
DC 1’s Edge Router Table WITH VMTO
10.10.10.100/32 Cost 5
10.10.10.200/32 Cost 5
WITH VMTO: NO INGRESS TROMBONE EFFECT

JUNIPER SOLUTIONS FOR VM MOBILITY
L2 & L3 address no longer
pinned to a site, interface
Ingress and Egress traffic
convergence, optimization
Learning and information
distribution control
L2 & L3 interaction for best
user experience
Fast convergence of network
paths as VM moves
Challenges
L2: Split subnet supported by VPLS
L3: Need provisioning help to advertise split
subnet members
L2 and L3: Split subnet supported by EVPN and
L3VPN
VPLS & L3VPN EVPN & L3VPN
L2: MX implements integrated L2 and ARP
learning (DP)
Fast convergence through flooding
Ingress, egress L2: automatic
Ingress L3: provisioning based
Egress L3: VRRP leverage
L2: DP based learning no advertisement
L3: BGP policies
Limited
L2: MX implements integrated L2, ARP, L3
advertisement (DP, CP)
Convergence through flooding and CP
announcement
Ingress, Egress, L3, L3 automatic
L2 and L3: BGP policies
Full

WAN
VXLAN ON MX – OVERLAY WITHIN LAN
VC
VC VC
POD-1: Hypervisor based
environment, terminating VXLAN
tunnels on MX and on virtual-switch
of servers
POD-2: Legacy server environment,
terminating VXLAN tunnels on MX
and access switches
POD-3: Legacy server and LAN
environment with no VXLAN tunnels,
MX providing gateway function to
VXLAN environment
Virtualized L2, and L3
Bridge-domain,
virtual-switch,
IRB, L2, and L3
Bridge-domain,
virtual-switch,
IRB, L2, and L3 Bridge-domain,
virtual-switch,
IRB, L2, and L3
MX acting as the
VTEP for VXLAN,
legacy LAN with full
BD, VS, IRB, L2, L3
support
High scale
multitenant
VXLAN
implementation
Orchestration &
Controller
Virtualized L2, and L3
DC GW
Intra DC
Network
Intra DC
Network
TOR TOR

THANK YOU

Dr관련 세미나 자료 v2

More Related Content

What's hot (15)

Viewers also liked (19)

Similar to Dr관련 세미나 자료 v2 (20)

Dr관련 세미나 자료 v2