Stress Testing Vs Load Testing

Load testing examines the entire environment and database, while measuring the response time, whereas stress testing focuses on identified transactions, pushing to a level so as to break transactions or systems.

System stress testing: also known as stress testing, is loading the concurrent users over and beyond the level that the system can handle, so it breaks at the weakest link within the entire system. During stress testing, if transactions are selectively stressed.

 
The other way of dealing this is... testing the system with heavy load & with less resource to handle.
Where as Load Testing mean to test the system with full load & with full resources in hand.

Relation Between Hits Per Sec & Throughput

As the hits per second increase....

Throughput: - Increases with increasing hits on the web server for a certain range. Generally it also depends on the bandwidth available. Then at a certain range the Throughput becomes constant & at this point if hits per sec increases then the throughput decreases.

So whenever we derive a relation between Hits per sec & Throughput then within the threshold limits they are directly proportional to each other...

Relation between Hits Per sec, CPU Usage & Response times

As the hits per second increases....

Response times:-

1. If the application is good enough to high user load, we don’t see any changes in the response times with respect to hits per sec

2. If the application is not good enough & with performance bottlenecks then response times gradually increase with increase in hits per sec. 

3. Finally if there is any network delay which occurs intermittently in some instances will increase the response times of all the transaction at that moment & hence due to which hits per sec & throughput decreases.

JAVA Related Performance Tuning Tips - Part 2

Servlet performance Tips:

• Use the servlet init() method to cache static data, and release them in the destroy() method.
• Use StringBuffer rather than using + operator when you concatenate multiple strings.
• Use the print() method rather than the println() method.
• Use a ServletOutputStream rather than a PrintWriter to send binary data.
• Initialize the PrintWriter with the optimal size for pages you write.
• Flush the data in sections so that the user can see partial pages more quickly.
• Minimize the synchronized block in the service method.
• Implement the getLastModified() method to use the browser cache and the server cache.
• Use the application server's caching facility.
• Session mechanisms from fastest to slowest are: HttpSession, Hidden fields, Cookies, URL rewriting, the persistency mechanism.
• Remove HttpSession objects explicitly in your program whenever you finish the session.
• Set the session time-out value as low as possible.
• Use transient variables to reduce serialization overheads.
• Disable the servlet auto reloading feature.
• Tune the thread pool size.

JSP performance Tips:

• Use the jspInit() method to cache static data, and release them in the jspDestroy() method.
• Use the jspInit() method to cache static data.
• Use StringBuffer rather than using + operator when you concatenate multiple strings.
• Use the print() method rather than the println() method.
• Use a ServletOutputStream rather than a PrintWriter to send binary data.
• Initialize the PrintWriter with the optimal size for pages you write.
• Flush the data in sections so that the user can see partial pages more quickly.
• Minimize the synchronized block in the service method.
• Avoid creating a session object with the directive <%@ page session="false" %>
• Increase the buffer size of System.out with the directive <%@ page buffer="12kb" %>
• Use the include directive instead of the include action when you want to include another page.
• Minimize the scope of the 'useBean' action.
• Custom tags incur a performance overhead. Use as few as possible.
• Use the application server's caching facility, and the session and application objects (using getAttribute()/setAttribute()). There are also third-party caching tags available.
• Session mechanisms from fastest to slowest are: session, Hidden fields, Cookies, URL rewriting, the persistency mechanism.
• Remove 'session' objects explicitly in your program whenever you finish the session.
• Reduce the session time-out as low as possible.
• Use 'transient' variables to reduce serialization overheads.
• Disable the JSP auto reloading feature.
• Tune the thread pool size.

Web application scalability Tips:

• Web application scalability is the ability to sustain the required number of simultaneous users and/or transactions, while maintaining adequate response times to end users.
• The first solution built with new skills and new technologies will always have room for improvement.
• Avoid deploying an application server that will cause embarrassment, or that could weaken customer confidence and business reputation [because of bad response times or lack of calability].
• Consider application performance throughout each phase of development and into production.
• Performance testing must be an integral part of designing, building, and maintaining Web applications.
• There appears to be a strong correlation between the use of performance testing tools and the likelihood that a site would scale as required.
• Automated performance tests must be planned for and iteratively implemented to identify and remove bottlenecks.
• Validate the architecture: decide on the maximum scaling requirements and then performance test to validate the necessary performance is achievable. This testing should be done on the prototype, before the application is built.
• Have a clear understanding of how easily your configurations of Web, application, and/or database servers can be expanded.
• Factor in load-balancing software and/or hardware in order to efficiently route requests to the least busy resource.
• Consider the effects security will have on performance: adding a security layer to transactions will impact response times. Dedicate specific server(s) to handle secure transactions.
• Select performance benchmarks and use them to quantify the scalability and determine performance targets and future performance improvements or degradations. Include all user types such as "information-gathering" visitors or "transaction" visitors in your benchmarks.
• Perform "Performance Regression Testing": continuously re-test and measure against the established benchmark tests to ensure that application performance hasn?t been degraded because of the changes you?ve made.
• Performance testing must continue even after the application is deployed. For applications expected to perform 24/7 inconsequential issues like database logging can degrade performance. Continuous monitoring is key to spotting even the slightest abnormality: set performance capacity thresholds and monitor them.
• When application transaction volumes reach 40% of maximum expected volumes, it is time to start executing plans to expand the system

JAVA Related Performance Tuning Tips - Part 1

Improving J2EE performance improves Tips:

• Set performance goals before development starts.
• If supporting clients with slow connections, consider compressing data for network communication.
• Minimize the number of network round trips required by the application.
• For applications to scale to many users, minimize the amount of shared memory that requires updating.
• Cache data to minimize lookup time, though this can reduce scalability if locks are required to access the cache.
• If there are more accesses than updates to a cache, share the access lock amongst all the accessors, though be aware that this reduces the window for updators to lock the cache.
• For optimum performance, zero shared memory provides a cache per user.
• Be methodical to ensure that changes for performance do actually improve performance.
• Eliminate memory leaks before tuning execution speed.
• Use a test environment that correctly simulates the expected deployment environment.
• Simulate the expected client activity, and compare the performance against your expected goals.
• Consider which metrics to measure, such as: Max response time under heavy load; CPU utilization under heavy load; How the application scales as additional users are added.
• Profile the application to find the bottlenecks. Correct bottlenecks by making one change at a time and testing for improvement.
• Generate stack traces to look for bottlenecks which are multi-thread conflicts (waiting for locks).
• Improving the performance of a method that is called 1000 times and takes a tenth of a second on average each call, is better than improving the performance of a method that is only called 10 times but takes 1 second each call.
• Don?t cache data unless you know how and when to invalidate the cached entries.

J2EE Performance tuning Tips:

• Call HttpSession.invalidate() to clean up a session when you no longer need to use it.
• For Web pages that don't require session tracking, save resources by turning off automatic session creation using: <%@ page session="false"%>
• Implement the HttpSessionBindingListener for all beans that are scoped as session interface and explicitly release resources implementing the method valueUnbound().
• Timeout sessions more quickly by setting the timeout or using session.setMaxInactiveInterval().
• Keep-Alive may be extra overhead for dynamic sites.
• Use the include directive <%@ include file="copyleft.html" %> where possible, as this is a compile-time directive (include action <jsp:include page="copyleft.jsp" /> is a runtime directive).
• Use cache tagging where possible.
• Always access entity beans from session beans.
• If only using an entity bean for data access, use JDBC directly instead.
• Use read-only in the deployment descriptor.
• Cache access to EJB homes.
• Use local entity beans when beans are co-located in the same JVM.
• Proprietary stubs can be used for caching and batching data.
• Use a dedicated remote object to generate unique primary keys.

EJB performance Tips:

• EJB calls are expensive. A method call from the client could cover all the following: get Home reference from the NamingService (one network round trip); get EJB reference (one or two network roundtrips plus remote creation and initialization of Home and EJB objects); call method and return value on EJB object (two or more network rountrips: client-server and [mutliple] server-db; several costly services used such as transactions, persistence, security, etc; multiple serializations and deserializations).
• If you don't need EJB services for an object, use a plain Java object and not an EJB object.
• Use Local interfaces (from EJB2.0) if you deploy both EJB Client and EJB in the same JVM. (For EJB1.1 based applications, some vendors provide pass-by-reference EJB implementations that work like Local interfaces).
• Wrap multiple entity beans in a session bean to change multiple EJB remote calls into one session bean remote call and several local calls (pattern called SessionFacade).
• Change multiple remote method calls into one remote method call with all the data combined into a parameter object.
• Control serialization by modifying unnecessary data variables with 'transient' key word to avoid unnecessary data transfer over network.
• Cache EJBHome references to avoid JNDI lookup overhead (pattern called ServiceLocator).
• Declare non-transactional methods of session beans with 'NotSupported' or 'Never' transaction attributes (in the ejb-jar.xml deployment descriptor file).
• Transactions should span the minimum time possible as transactions lock database rows.
• Set the transaction time-out (in the ejb-jar.xml deployment descriptor file).
• Use clustering for scalability.
• Tune the EJB Server thread count.
• Use the HttpSession object rather than a Stateful session bean to maintain client state.
• Use the ECperf benchmark to help differentiate EJB server performances.
• Tune the Stateless session beans pool size to minimize the creation and destruction of beans.
• Use the setSessionContext() or ejbCreate() method to cache bean specific resources. Release acquired resources in the ejbRemove() method.
• Tune the Stateful session beans cache size to and time-out minimize activations and passivations.
• Allow stateful session beans to be removed from the container cache by explicitly using the remove() method in the client.
• Tune the entity beans pool size to minimize the creation and destruction of beans.
• Tune the entity beans cache size to minimize the activation and passivation of beans (and associated database calls).
• Use the setEntityContext() method to cache bean specific resources and release them from the unSetEntityContext() method.
• Use Lazy loading to avoid unnecessary pre-loading of child data.
• Choose the lowest cost transaction isolation level that avoids corrupting the data. Transaction levels in increasing cost are: TRANSACTION_READ_UNCOMMITED, TRANSACTION_READ_COMMITED, TRANSACTION_REPEATABLE_READ, TRANSACTION_SERIALIZABLE.
• Use the lowest cost locking available from the database that is consistent with any transaction.
• Create read-only entity beans for read only operations.
• Use a dirty flag where supported by the EJB server to avoid writing unchanged EJBs to the database.
• Commit the data after the transaction completes rather than after each method call (where supported by EJB server).
• Do bulk updates to reduce database calls.
• Use CMP rather than BMP to utilize built-in performance optimization facilities of CMP.
• Use ejbHome() methods for global operations (from EJB2.0).
• Tune the connection pool size to minimize the creation and destruction of database connections.
• Use JDBC directly rather than using entity beans when dealing with large amounts of data such as searching a large database.
• Combine business logic with the entity bean that holds the data needed for that logic to process.
• Tune the Message driven beans pool size to optimize the concurrent processing of messages.
• Use the setMesssageDrivenContext() or ejbCreate() method to cache bean specific resources, and release those resources from the ejbRemove() method.