Medium 9780596520663

The Art of Application Performance Testing: Help for Programmers and Quality Assurance

Views: 202
Ratings: (0)

This practical book provides a step-by-step approach to testing mission-critical applications for scalability and performance before they're deployed -- a vital topic to which other books devote one chapter, if that.

Businesses today live and die by network applications and web services. Because of the increasing complexity of these programs, and the pressure to deploy them quickly, many professionals don't take the time to ensure that they'll perform well and scale effectively. The Art of Application Performance Testing explains the complete life cycle of the testing process, and demonstrates best practices to help you plan, gain approval for, coordinate, and conduct performance tests on your applications. With this book, you'll learn to:

  • Set realistic performance testing goals
  • Implement an effective application performance testing strategy
  • Interpret performance test results
  • Cope with different application technologies and architectures
  • Use automated performance testing tools
  • Test traditional local applications, web-based applications, and web services (SOAs)
  • Recognize and resolves issues that are often overlooked in performance tests

Written by a consultant with 30 years of experience in the IT industry and over 12 years experience with performance testing, this easy-to-read book is illustrated with real-world examples and packed with practical advice. The Art of Application Performance Testing thoroughly explains the pitfalls of an inadequate testing strategy and offers you a robust, structured approach for ensuring that your applications perform well and scale effectively when the need arises.

"Ian has maintained a vendor-agnostic methodology beautifully in this material. The metrics and graphs, along with background information provided in his case studies, eloquently convey to the reader, 'Methodology above all, tools at your discretion...' Ian's expertise shines through throughout the entire reading experience."-- Matt St. Onge, Enterprise Solution Architect, HCL Technologies America / Teradyne

List price: $27.99

Your Price: $22.39

You Save: 20%

 

10 Slices

Format Buy Remix

1. Why Performance Test?

ePub

Faster than a speeding bullet . . .

This chapter poses some fundamental questions concerning the subject of this book. What is performance? Why carry out performance testing in the first place? Here I also define when an application is considered performant versus nonperformant and then discuss some common causes of a suboptimal end-user experience.

Nonperformant (i.e., badly performing) applications generally don’t deliver their intended benefit to the organization. That is, they create a net cost of time, money, and loss of kudos from the application users and therefore can’t be considered reliable assets. If an application is not delivering benefits, its continued existence is definitely on shaky ground—not to mention that of the architects, designers, coders, and testers (hopefully there were some!).

Performance testing is a neglected cousin of unit, functional, and system testing, which are well understood in most businesses and where the maturity level is high in many organizations. It is strange but true to say that executives do not appreciate the importance of performance testing. This has changed little over the past ten years despite the best efforts of consultants like myself and the many highly publicized failures of key software applications.

 

2. The Fundamentals of Effective Application Performance Testing

ePub

For the want of a nail . . .

In this chapter we cover a basic task that seems to be avoided even by companies that take on the task of performance testing: capturing performance requirements. Unlike the “functional” requirements upon which regression and unit testing are based, performance related requirements can be considered “nonfunctional.” Semantics aside, they are vital activities that must be addressed in order to carry out effective performance testing.

The idea of a formal approach to application performance testing is still considered novel by many. Just why is something of a mystery, because (as with any project) failing to plan properly will inevitably lead to misunderstandings and problems. Performance testing is no exception.

With this thought in mind, let’s reiterate my performance testing mantra:

Performance awareness should be built into the application life cycle as early as possible.

In other words: if you don’t start your planning with performance in mind, then you expose yourself to significant risk that your application will never perform to expectations.

 

3. The Process of Performance Testing

ePub

I see plans within plans.

As discussed in Chapter 1, many performance testing projects come together as a last-minute exercise. I would happily wager that you have been involved in a project that falls into this category. In such cases, you are constrained by limited time frames and pressure to deploy by a certain date, even though the application may have serious undetected performance problems. This chapter describes a performance testing approach to follow so that any new projects you participate in don’t suffer from the same pitfalls.

In Chapter 2 my intention was to cover performance testing requirements in a logical but informal way. This chapter is about using these requirements to build a plan: a performance testing checklist divided into logical stages. We’ll also look at how this plan can be applied to a couple of case studies based on real projects. Each case study will demonstrate different aspects of the performance test process and will provide some of the examples used in Chapter 4 to demonstrate the interpretation of performance test results.

 

4. Interpreting Results: Effective Root-Cause Analysis

ePub

Statistics are like lampposts: they are good to lean on, but they don’t shed much light.

OK, so I’m running a performance test—what’s it telling me? The correct interpretation of results is obviously vitally important. Since we’re assuming you’ve (hopefully) set proper performance targets as part of your testing requirements, you should be able to spot problems quickly during the test or as part of the analysis process at test completion.

If your application concurrency target was 250 users, crashing and burning at 50 represents an obvious failure. What’s important is having all the necessary information at hand to diagnose when things go wrong and what happened when they do. Performance test execution is often a series of false starts, especially if the application you’re testing has significant design or configuration problems.

I’ll begin this chapter by talking a little about the types of information you should expect to see from an automated performance test. Then we’ll look at some real-world examples, some of which are based on the case studies in Chapter 3.

 

5. Application Technology and Its Impact on Performance Testing

ePub

IDIC, infinite diversity in infinite combinations.

As mentioned in the Preface, this book is designed to provide a practical guide to application performance testing. However, certain application technologies require us to refine our testing strategy to ensure that we achieve maximum benefit.

It therefore seems appropriate to offer some guidance on how different technologies may alter your approach to performance testing. I’ll try to cover all the important technologies that I’ve encountered over many years of performance testing projects together with some of the newer offerings that have emerged alongside NET.

This is not the Greek hero of antiquity but rather a technology designed to make things better for the poor end users. The key word here is “asynchronous,” which refers to breaking the mold of traditional synchronous behavior where “I do something and then I have to wait for you to respond before I can continue.” Making use of AJAX in your application design removes this potential logjam and can lead to a great end-user experience. Unfortunately, most automated performance tools find this sort of technology difficult to handle.

 

A. Transaction Examples

ePub

This appendix contains some examples of transactions taken from a real project, showing the parameters you generally need to script the transaction using an automated performance testing tool.

Table A-1 demonstrates the sort of detail you should provide for each transaction to be included in your performance test. The columns in the table refer to the following information.

Step. Order of events in the transaction.

Action. What the user interaction is with the application client.

Test Data. The data entered by the user and whether this will come from an external file DATAPOOL refers to an external file of data typically is CSV format.

System Time. The expected time taken for the system to respond during transaction capture.

User Think Time. The amount of time taken by a user to react to what has been displayed by the application in response to the Action for the Step.

 

B. POC and Performance Test Quick Reference

ePub

This appendix contains a convenient quick reference, drawn from earlier chapters, of the steps required at each stage of planning and carrying out a Proof of Concept (POC), performance test execution, and performance test analysis.

A Proof of Concept (POC) is an important prerequisite because it provides the following information (see Chapter 3 for details).

Provides an opportunity for a technical evaluation of the performance testing tool against the target application.

Identifies scripting data requirements.

Allows assessment of scripting effort.

Demonstrates the capabilities of the performance testing solution against the target application.

You should anticipate no more than a couple of days for completion, assuming that the environment and application are available from day one.

The following should be in place before you set up the POC environment.

A written set of success or exit criteria that you and the customer have agreed on as determining the success or failure of the POC.

 

C. Automated Tool Vendors

ePub

This appendix provides a list of the leading automated tool vendors. I have also included entries for popular free tools where appropriate. The tool vendor market is fairly dynamic, but the list contains vendors who have the lion’s share at the time of this writing. The list is in no particular order. For each section I have listed the tool vendor and web site followed by the product name and (where appropriate) a link for more information.

Application Vantage

http://www.compuware.com/solutions/appvantage.htm

Vantage Analyzer

http://www.compuware.com/products/vantage/vantageanalyzer.asp

DynaTrace

http://www.dynatrace.com/en/

Panorama

http://www.opnet.com/solutions/application_performance/panorama.html

Silk Performer

http://www.borland.com/us/products/silk/silkperformer/index.html

 

D. Sample KPI Monitoring Templates

ePub

The following examples, reproduced here by kind permission of Compuware Corporation, demonstrate groups of common server KPIs that I favor in performance testing projects.

The first KPI template, shown in Figure D-1, provides a good indication of when a Windows server is under stress. This is the first level of monitoring that I use when performance testing a Windows OS server tier. You will probably recognize the metrics, since they taken from the Windows Performance Monitor (Perfmon) application. This is a nice mix of counters that monitor disk, memory, and CPU performance data together with some high-level network information measuring the number of errors encountered and data throughput in bytes per second.

I recommend that, in addition to using this template, you monitor the top ten processes in terms of CPU utilization and memory consumption. Identifying a process that is CPU- or memory-hungry is often your best link to the next level of KPI monitoring, where you need to drill down into specific software applications and components that are part of the application deployment.

 

E. Sample Project Plan

ePub

The following example is based on a typical performance project template in MS Project. The “profiling” section involves an additional transaction optimization step aimed at detecting and addressing—prior to performance test execution—application-related problems that may hurt scalability and response time.

The MS Project file is available from the book’s companion web site.

 

Details

Print Book
E-Books
Slices

Format name
ePub
Encrypted
No
Sku
9780596555436
Isbn
9780596555436
File size
0 Bytes
Printing
Not Allowed
Copying
Not Allowed
Read aloud
No
Format name
ePub
Encrypted
No
Printing
Allowed
Copying
Allowed
Read aloud
Allowed
Sku
In metadata
Isbn
In metadata
File size
In metadata