Advanced Rational Performance Tester reports

Advanced Rational Performance Tester reports
5 ways to get deeper insight into performance test results

Skill Level: Advanced

Jigar N Kapasi (kjigar@in.ibm.com)
Senior Software Architect
IBM India

Priyanka Arora (prarora1@in.ibm.com)
Performance Analyst
IBM

01 Mar 2011

By default, IBM® Rational® Performance Tester provides essential performance
metrics, such as throughput, response times, concurrency, and success rate.
However, it also includes several advanced features for detailed analysis, many of
which are not commonly used. Proper use of these options provides deeper insight
when analyzing test results. This article gives five tips for using some of these
advanced features, all of which have helped tremendously in real-world performance
testing projects with large companies.
This article provides five tips that a performance test engineer can use to gain
deeper insight into performance results:

1. Handling spikes in results

2. Generating and analyzing percentile reports

3. Creating and using business transactions

4. Determining volumes during a subset of a run

5. Correlating client-side results with server activities

Advanced Rational Performance Tester reports Trademarks
© Copyright IBM Corporation 2011 Page 1 of 13

developerWorks® ibm.com/developerWorks

1. Handling spikes in performance test results
In an IBM® Rational® Performance Tester report, the Response vs Time Detail tab
plots response time for each page. Extraordinarily high response times, or spikes (as
shown in Figure 1), can be a concern if they occur frequently. Analyzing the raw
response time data can give useful information about application performance.

Figure 1. Spikes in page response times

To determine the percentage of such outliers, you need the raw data for the test run
that Rational Performance Tester uses to plot the performance report graphs. These
are the steps to get this data:

1. Right-click any tab of the report, and select Export Report Counters to
CSV.

2. Browse to a location to save the output, and name the file.

3. For Export Type, select Full to get all counters.

4. Check Export overall results and Split output if columns exceed 250
(250 is the maximum column limit for a Microsoft Excel file).

5. In the location that you just specified, multiple .csv files will get created.
Browse the files for the one with this column name: All_Hosts Pages
Response Time Page_Name Average interval.
Note:Page_Name here will be replaced by the page titles in your test
schedule, and there will be a separate column for each page in the test


ibm.com/developerWorks developerWorks®

schedule.

Look for data on pages that show spikes in the graph. This data can now be used to
determine the percentage of outliers for each page and troubleshoot potential
bottlenecks.

2. Generating and analyzing percentile reports
Percentile response times are often better indicators of the application performance
than average response times. Understanding how to generate and analyze
performance reports will help you understand system behavior more accurately.

Generate percentile reports

After a test is finished and the performance report has been generated, you can
generate a percentile report.

1. From the Performance Test Runs window, under the report, right-click
All Hosts.

2. Select HTTP Reports > Percentile Report, as shown in the drop-down
menu selections in Figure 2, to open the percentile report with default
analysis targets.



Figure 2. Generating percentile report

By default, a percentile report shows the 85th, 90th, and 95th percentiles. You can
define a new value or modify the default percentile analysis target.

3. Select Window > Preferences.

4. Expand Test and click Percentile Analysis Targets.

5. Either click Add and enter a required percentile value, as shown in Figure
3, or select an existing target and click Edit.

6. Click OK, and then click Apply to implement your changes.



Figure 3. Custom percentile analysis targets

7. Regenerate the percentile report to see your custom percentile target
values.

Analyze percentile reports

Theoretically, if the nth percentile of a set equals x, it means that n% values in the
given set are below the value of x. So, applying that to percentile response time
reports in Rational Performance Tester, if the 85th percentile response time for a
page is 5 seconds, it means that 85% of virtual users got a response time of 5
seconds or better.

Percentile response times are more relevant than the average times when it is
important that each user of the system gets acceptable response times consistently.
The more stringent a customer is on each user's experience, the higher the
percentile target to be considered. As the number of spikes in the response time set
increases, the probability of getting a high percentile response time value for that
page also increases.

Also see the Handling spikes in performance test results section.



3. Creating and using business transactions
Customers often characterize non-functional requirements in terms of business
transactions. A business transaction typically spans multiple pages; whereas, by
default, Rational Performance Tester reports metrics on a page basis. Generating
performance results represented in terms of business transactions makes it easier
for business users to benefit from what the reports reveal.

For business transaction-based metric reports, define the business transactions first,
and then define the required performance metrics first. Then you can generate the
transaction reports.

Define business transactions

A typical business transaction is representative of a use case consisting of several
pages (navigating through required pages to raise a service request, for example).
After you have identified a business transaction, you can define it by following these
steps:

1. Record the scenario in Rational Performance Tester.

2. Select all sequential pages in the script that form the transaction.

3. Right-click and select Insert > Transaction.

4. In response to the question in the pop-up window that asks if you would
like to move the selected objects into a new transaction, click Yes.

5. Give the transaction a meaningful name, and then Save the test.

Enable performance requirements

A performance requirement specifies acceptable thresholds of performance and
helps validate service level agreements. You can define performance requirements
on several test elements.

Follow these steps to enable performance requirements on business transactions:

1. Select the transaction in the test contents.

2. Under Test Element Details, click the Advanced tab.

3. Click the check box for Enable Performance Requirements.



4. Now define all of the transaction performance requirements that will
determine the transaction status (Pass or Fail) of your test. For example,
if it is a requirement that the transaction must complete in not more than
120 seconds:

a. Select the Maximum Transaction Execution Time [ms]
performance requirement.

b. Choose the <= operator (less than or equal to operator).

c. Enter the value in milliseconds. In this case, enter 120000, as
shown in Figure 4.

5. Ensure that the page think times and pace times in this transaction align
with these requirements. Thus, the sum of all think times, pace times, and
expected page response times should fit within this limit.

Figure 4. Business transaction performance requirements

Larger view of Figure 4.

Generate transaction performance reports

If you have transactions defined in your test suite, then the Summary tab of the
performance report details a section on transaction performance metrics, such as
average execution time, maximum or minimum execution time, and so on, as Figure
5 shows.



Figure 5. Business Transaction Summary report

To get a detailed report for each business transaction, generate the Transaction
Report and the Transaction Percentile Report:

1. In the Performance Test Runs window, right-click the test run.

2. Select either Display Transaction Report or Display Transaction
Percentile Report.

3. The Transaction Report gives the transaction throughput, transaction
count for the run, and a plot of transactions at run time. The Transaction
Percentile Report shows execution (run) times by transaction at the
defined percentile analysis targets. See the Generating and analyzing
percentile reports section for more details.

4. Determining volumes for a specified subset duration
Some performance tests have non-functional requirements for volumes of business
transactions achieved over a specified duration in a steady state. By default, a
Rational Performance Tester report gives the total number of transactions achieved
across the total duration of the run. These results include statistics gathered during
the ramp-up phase, where the page hits are much lower. Getting performance
metrics during the steady state gives you a more realistic measure of how a system
will perform.

You can use the steady state as the time range for your report. Alternatively, you
can also define your own time range in the test run by specifying start and end
times. By doing this, you will then see metrics in the report that pertain only to that
time range.

Follow these steps to define your own time range:



1. Click the Server Health Detail tab on the report. The Hits -- Count [for
run] column lists volumes against each page or transaction.

2. Right-click the report and select Change Time Range.

3. In the Select Time Range view (Figure 6), click the New Time Range
button to define a new range.

4. You then have two choices:
• Either click Set to Steady State
• Or enter values of your choice for Start Time and End Time from
within the test run duration

Figure 6. Changing the time range for the report

5. Finish and wait for the report to generate with the new time range.

The report will then show statistics only from samples in the range that you
specified.

(Also see the Creating and using business transactions section.)



5. Correlating reports with server activities and logs
When monitoring or analyzing performance reports, it is very useful to correlate the
reported performance with activities on the server side (through logs, traces, for
instance). By default, the Rational Performance Tester performance report shows
time across all tabs in seconds (starting at zero). Matching the time scale in the
report with that of the server helps correlate performance results with server events
and aids troubleshooting.

It is easier to correlate client-side performance metrics from Rational Performance
Tester with the server-side log activities if the test report shows absolute times rather
than relative times starting from zero.

To customize the time, open a performance report and complete these tasks:

1. Go to the Response Vs Time Detail tab on the report.

2. Right-click and select Customize.

3. For the X Axis field, click the arrow to open the drop-down menu, and
select Date & Time (Absolute) to change the template, as shown in
Figure 7.

4. Click OK and let the report refresh to reflect the change.



Figure 7. Customizing time on reports

Important:
To ensure precise correlation, synchronize the system time on the Rational
Performance Tester host controller, agent, and all servers under test.



Resources
Learn
• Download and read the PDF titled Using Rational Performance Tester Version 7
(IBM® Redbooks®, 2008).
• Find out more on the Rational Performance Tester product overview page.
Then explore the Rational Performance Tester page on IBM®
developerWorks® for links to technical articles and browse the user assistance
in the Rational Performance Tester Information Center.
• Visit the Rational software area on developerWorksfor technical resources and
best practices for Rational Software Delivery Platform products.
• Stay current with developerWorks technical events and webcasts focused on a
variety of IBM products and IT industry topics.
• Attend a free developerWorks Live! briefing to get up-to-speed quickly on
IBM products and tools, as well as IT industry trends.
• Watch developerWorks on-demand demos, ranging from product
installation and setup demos for beginners to advanced functionality for
experienced developers.

• Improve your skills. Check the Rational training and certification catalog, which
includes many types of courses on a wide range of topics. You can take some
of them anywhere, any time, and many of the “Getting Started” ones are free.
Get products and technologies
• Download the trial version of IBM Rational Performance Tester.
• Evaluate IBM software in the way that suits you best: Download it for a trial, try
it online, use it in a cloud environment, or spend a few hours in the SOA
Sandbox learning how to implement service-oriented architecture efficiently.
Discuss
• Join the Performance Testing forum, where you can share you questions and
knowledge about IBM performance testing products.
• Share your knowledge and help others who use Rational software by writing a
developerWorks article. You'll get worldwide exposure, RSS syndication, a
byline and a bio, and the benefit of professional editing and production on the
developerWorks Rational website.
• Follow Rational software on Facebook and Twitter (@ibmrational), and add your
comments and requests.
• Get involved in the Rational forums, groups, and Rational wikis.



• Connect with others who share your interests by joining the developerWorks
community and responding to the developer-driven blogs.

About the authors
Jigar N Kapasi
Jigar Kapasi is a software performance architect in the IBM India
Software Lab, working with the HiPODS team. He works with large
companies to lead critical performance engineering projects, which
include architecture and design recommendations, performance testing,
infrastructure and middleware analysis and tuning. He has been with
IBM for 12 years and holds a bachelor’s degree in Information
Technology from Bangalore University.

Priyanka Arora
Priyanka Arora is a software performance analyst with the
High-Performance On Demand Solutions team in India. She has more
than three years of experience with strong technical analysis, tuning,
troubleshooting, tooling, and load test execution of end-to-end
performance engagements for several IBM telecom and public sector
customers. She has an MSc Tech in Information Systems from Birla
Institute of Technology and Science in India and is a certified
WebSphere Application Server and Portal Server Administrator.


Advanced Rational Performance Tester reports

More Related Content

What's hot (20)

Similar to Advanced Rational Performance Tester reports (20)

More from Bill Duncan (20)

Recently uploaded (20)

Advanced Rational Performance Tester reports