Stress Testing Process

Stress Testing Process

	Stress testing is seen as part of the process of performance testing which tries to identify the breaking point in a system under test by overwhelming its resources or by taking resources away from it. This helps to ensure that the system fails and recovers gracefully. Stress Testing is done to uncover memory leaks, bandwidth limits, transactional problems, resource locking, hardware limitations and synchronization problems that occur when an application is loaded beyond the limits determined by the performance statistics. Stress testing tools generate extremely high load on the Web server by simulating multiple client connections.
Stress testing process is very simple and straightforward which is done in three easy steps: Record Your Stress Test Scripts Add dynamic data to the test and set load level Run stress test and analyze stress test results
Stress Testing Features
The key features that help you perform stress testing are: Stress Test Recording Stress test recording is made simple and easy with a point and click user action. You can record stress test scripts to capture any HTTP/HTTPS/SSL/AJAX requests. Flexible User Scenarios and Dynamic Data Generation To emulate the real-user activities, stress testing provides the flexibility to split the number of users accessing different parts of the web application performing different operations. You can also capture user diversity to simulate a variety of virtual users characteristics such as different connection speeds, different browser types, IP Spoofing and different servers/ports. To capture real-life stress testing, you can generate dynamic data for the stress test. The values of session IDs or request parameters (Get/Post data) can be fetched from a number of ways (from dataset, hidden elements, previous response/url, cookies, etc). Real-world Load Simulation To stress test and identify the potential breaking points in your web application. Burn-in Mode Load Test: The burn-in mode helps you to verify the acceptability of your web sites/web applications when abnormal or extreme load conditions are encountered for an extended period of time, such as diminished resources or extremely high number of users. You can exit the stress test only based on some specified exit criteria. Exit criteria allows you to set multiple criteria based on which the stress test should exit or the system should break, if one of the criterion is not met during the stress test execution. This goal based test methodology enables stress testers to set thresholds and parameters during stress test configuration to identify the breaking points. If not attainable, either the system should break or the stress test should exit.  Exit Criteria Options Following are the options available to set the goal for the stress test: Exit on Server Crash: Select this option to exit the stress test, if the web-server crashes.   Exit if Test Duration exceeds specified time: Select this option to exit the stress test, if the test duration exceeds the specified time in milliseconds. Exit if Response Time is not less than specified millisecs for % of the time: Select this option to exit the stress test, if the response time is less than the specified milliseconds for the specified percentile of the time. For example, if the response time is given as 10 milliseconds for 95 % of the time. Then the response time is sorted in ascending order from the lowest to the highest. Assume there are 20 values, then find the 95th percentile value (95/100*20=19). If this value is less than 10 milliseconds then exit the stress test.   Exit if CPU% on Server exceeds specified %: Select this option to exit the stress test, if the CPU % of the specified server exceeds the specified CPU %. You can also configure new server monitors and check the CPU % of the same. Mixed Load Test: This helps you to identify the breaking point in your web application when a combination of workloads is added which emulates the real-world load behavior such as steady-state, ramp-up or ramp-down.  Distributed Stress Testing To simulate a very high load hitting your web site for stress testing, you have the option of simulating the users in a single machine with a high configuration or distribute the load across multiple machines using the distributed playback option. Single load test controller that centrally manages automatically generates and distributes the load across multiple play engines. Web-based Play Engine Configuration UI allows you to quickly and easily configure the distributed machine IPs and the maximum virtual users to be simulated in each machine (Windows or Linux machines). Stress Test Reports Provides a clear and comprehensive range of stress test reports and graphs which includes both summary and detailed reports. You can also refer to the white paper on Performance Testing Report Analysis. Some of the key stress test reports and graphs that helps you to quickly and easily analyze the potential bottlenecks in your web application are as follows. Request/Response Status Summary to know whether the stress test configuration is appropriate. Error Rate Graph, Error Distribution Graph, Response Validation Error Details helps you to analyze the stress test errors and adjust the stress tests in order to achieve meaningful and accurate results. Graph to identify performance bottlenecks provides a consolidated view which shows the server response time for individual URLs, error percentage and the number of active users over elapsed time. User Details Graph helps you to know the number of virtual users generated by the load generator. The Hits per Second graph shows the number of HTTP requests made by Virtual users to the Web server during each second of the run. Response time graphs help you to know whether the response time meets your target requirements. User capacity graph and Response Time vs User Load Graph helps you to know how many simultaneous virtual users your web server can handle. Server and Database graphs help you to know how your web application performance affects due to server and database parameters.

Web Services Performance Testing

Web Services Performance Testing using SOAPUI
Recently i had a chance to recommend a performance testing tool to carryout performance testing for web services system. I evaluated a series of open source and licensed tools and finally concluded SOAPUI would be meet the requirement for our system.
SoapUI provides extensive load testing functionality allowing you to do the following:

Functional LoadTesting: validate functionality under load using standard TestCase methods.

Behavioral Load Testing: analyze performance behavior under varying load with different load strategies.

Performance LoadTesting: find maximal performance available using thread strategies and Command Line Load Test execution.

Requirements Driven LoadTesting: define performance requirements and continuously validate using Load Test assertions.

SOAPUI helps in the following.
> Validate a Web Services performance under different Load scenarios.
> Maintain Functional validations to see that they don't "break" under load.
> Run several load tests simultaneously to see how they affect each other.
It’s a freeware tool which best suits web services load testing. Its commercial version is called SOAPUI Pro includes productivity enhancements.
In Overall SOAPUI meets most the requirements for a Web services performance testing and it is a apt tool to meet most of the web services.

WCF Services Performance Testing

Most of the market available performance testing tools available for web services performance testing does not provide support to WCF (Windows Communication Foundation) web services performance testing.

The following are the tools which support WCF web services performance testing.

> WCFStorm
> WCF Load Test
> SOAtest

SOAPUI doesn’t support WCF web services performance testing. Out of above 3 tools, we have finalized on the SOAtest 5.5 tool to carryout WCF web services performance testing.

Do think about Think Time

I got a chance to interact with lot of performance testers in various organizations , not only in India, but also in abroad & found that most of us are interested in spending time for setting realistic performance test goals & identifying the frequently used business scenarios (business flows) , but very few analyze the think time of various business flows.

It becomes very important that the average think time of the end users of the application needs to be analyzed rather than estimating specific time limit by counting the total fields to be filled by the end user. For the same application (let’s take Yahoo new account creation), a new user might take 4-5 minutes or more to fill the required details to create a new account. But at the same time, a frequent old user might take 1-2 minutes to create a new account. These think times (4-5 minutes for new user & 1-2 minutes for old user) might impact the server load.

Let’s assume 100 new users are accessing the yahoo application for the time interval 13.00 to 14.00 hours & 100 old users are accessing the application for the time interval 14.00 to 15.00 hours continuously. Though there are 100 users (100 active sessions created) load on the server during both 13.00 to 14.00 hours and during 14.00 to 15.00 hours, the actual server load (requests) is going to be different during these 2 hours.

When you have a close look at the server load in requests/sec (Hits/sec) , you will find that the server load during 13.00 to 14.00 hours might be lesser than 14.00 to 15.00 hours, because of high think time used by the new users. If you look at the quantified numbers, you will be shocked to know that during the first hour, the average load could be apprx 12 hits/sec whereas during the second hour, the server load could be around 25 hits/sec.

Having performance test goals in mere user load alone doesn’t make the test more realistic. Detailed analysis on the end user's think time makes it more realistic.