Performance Testing from the Cloud
Download as PPTX, PDF1 like917 views
Digital River's performance testing case study highlights the implementation of a high-capacity order capture solution capable of handling 6,000 orders per second. It discusses challenges such as load generation, tuning, and the advantages of using cloud-based solutions for cost-effectiveness and scalability. The findings emphasize the need for careful monitoring of capacity, accuracy, and connectivity when leveraging cloud resources.
Performance Testing from the Cloud
- 1. Performance Testing from the Cloud Presented at HP Discover 2012 Tom Kleingarn Technical Manager, Performance Engineering Digital River http://www.linkedin.com/in/tomkleingarn http://www.perftom.com
- 2. Agenda > Digital River > Case Study > The Cloud > Challenges > Capacity > Accuracy > Connectivity > Optimization > Summary
- 3. > Global Cloud Commerce Solution Provider > Founded in 1994 > Over $3 billion in annual online sales > Daily Stats > > > > 20 million page views 30 million emails 175,000 orders 5 terabytes of digital content > 8 global data centers > Industries > > > > Software Consumer Electronics Games Payment Processing
- 4. Case Study > Dec, 2011 – Digital River launched high-capacity order capture solution > Capable of capturing 6,000 orders per second > Per hour, that’s… > 2 million orders > 400 stadiums > The state of Nevada > 3 day timeline for external performance validation
- 5. Challenges > Load Generators > Capacity > Tuning > Space > Supporting teams > License cost > Internal network infrastructure > Time!
- 6. Solutions > Add internal generators > Cost, space, infrastructure, installation, time > Not external > External vendor > Cost, connectivity, unfamiliar with app > External generators in public cloud > Connectivity
- 7. The Cloud > Virtual machine (VM) impermanence > Capacity and accuracy > Connectivity
- 8. The Cloud
- 11. The Cloud Capacity and accuracy > http://aws.amazon.com/ec2/instance-types/ > Dedicated CPU, memory, local storage > Shared network, disk subsystem > Larger instances guarantee less variability in shared resources
- 13. The Cloud Connectivity 54345 Controller Generator 50500 Controller <PORT> System Under Test (SUT) 443 MI Listener Generator System Under Test (SUT)
- 14. The Cloud Connectivity 54345 Controller <PORT> Generator System Under Test (SUT) RDP (3389) - SSH (22)
- 15. Installation > Launch an Instance
- 16. Associate an IP Address > Each generator needs a static IP
- 17. Installation - Windows > Windows Server 2008 > Limited to c1.xlarge > 8 Cores, 7GB RAM > I/O performance: High 1. RDP to running instance 2. Download LoadRunner 3. Mount image 4. Install 5. Open port 54345
- 18. Installation - Linux > A little more complicated ... > Red Hat Enterprise Linux 6.2 > Limited to c1.xlarge > 8 Cores, 7GB RAM > I/O performance: High 1. Connect via SSH and Keyfile > Windows - Use PuttyGen to generate .ppk file from .pem file 2. Log in as root user
- 19. Installation - Linux 4. Set environment variables > To set automatically on startup, create a .profile file in the “/root” directory, add this to the file: export M_LROOT=/opt/HP/HP_LoadGenerator export LD_LIBRARY_PATH=${M_LROOT}/bin export PATH=${M_LROOT}/bin:${PATH} export DISPLAY=localhost0.0 5. Copy LoadRunner installation files to the instance > Use WINSCP to connect and copy via SFTP
- 20. Installation - Linux 6. Unzip the archive > unzip <filename> 7. Mount the image Create the directory i.e. mount point: > mkdir -p /mnt/disk Use mount command as follows to mount iso file called disk1.iso: > mount -o loop LG_Linux_Install.iso /mnt/disk Change directory to list files stored inside an ISO image: > cd /mnt/disk > ls -l
- 21. Installation - Linux 8. Run the installer: > /mnt/disk/Linux/installer.sh 9. Check files in path of LoadRunner installation: > /opt/HP/HP_LoadGenerator 10. Install additional required packages > yum install compat-lib* > yum install glibc.i686 > yum install libstdc++.so.5 11. Install daemon process /opt/HP/HP_LoadGenerator/bin/m_daemon_setup -install
- 22. Installation - Linux 12. Create a wrapper script the starts the m_daemon_setup process in /usr/local/bin (to follow common practices) #!/bin/bash # LoadRunner load generator daemon startup script echo Initializing load generator daemon process … echo This process is required to connect to the generator from the controller machine echo without using RSH # Installation command is: # /opt/HP/HP_LoadGenerator/bin/m_daemon_setup -install # If the daemon is already installed, just start it with the following command m_daemon_setup start $M_LROOT/bin # Other args include: -kill, -remove, -start, -stop, -stat
- 23. Installation - Linux 13. Add this to your .profile file to run the script on startup > source /usr/local/bin/init_loadgenerator_daemon > Considerations, additional security > MI Listener configuration > RSH vs. daemon process > Host Security Manager (since LR 9.5)
- 24. Connect Controller to Generators > Windows, no extra steps > Linux, specify UNIX OS > Log in via SSH, verify connectivity and daemon process > Generator Details > Controller > Unix Environment > Check “Don’t use RSH”
- 25. Generator - Unix Environment Settings
- 26. Optimize for High Throughput > Load generators create thousands of connections > Limit of 65,534 connections per network interface > Avoid port starvation, increase the port range used, and allow reuse of sockets in TIME_WAIT > Windows > HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServi cesTcpipParameters > Set value to 65534 > Linux > sysctl -w net.ipv4.ip_local_port_range="1024 65535“ > sysctl -w net.ipv4.tcp_tw_recycle=1 > Linux has a throughput advantage here!
- 27. Use Multiple Data Sources Controller Cloud Generator Physical Generator System Under Test (SUT) External Monitor
- 28. Case Study Results > Cloud generators yielded reliable results > Alignment across internal generators, external generators, and independent client validation > Results from single server yielded 2,000+ orders per second
- 29. Case Study Results > Linear scale confirmed Orders Captured per Second by Server Count 35,000 30,000 Orders per Second 25,000 20,000 15,000 10,000 5,000 0 1 2 3 4 5 6 7 8 Server Count 9 10 11 12 13 14 15
- 30. Summary > The cloud is cost effective but unpredictable > Capacity > Use the largest footprint possible > Accuracy > Compare results across data sources > Monitor physical and virtual utilization (if possible) > Connectivity > 54345 or MI Listener > Linux - LR Daemon or RSH > Configure optimal port range, remember Linux advantage
- 31. Benefits > Cost Savings > Hardware > $4000 for 8 core, 8GB server > $400 monthly maintenance > External vendor - $20K > Time > No dependence on infrastructure team for provisioning/support/maintenance > On demand flexibility
- 32. Questions?
Editor's Notes
- #2: Global E-CommerceIndustries Software Consumer Electronics GamesPayments global payments program local and global payment methods real time fraudE-Marketing email analytics bid management full service or self service
- #16: http://youtu.be/Kr3BN0XctPU
- #17: http://youtu.be/F_PSaOu6kao
- #29: Results from a single web server