Thursday, March 1, 2012

LoadRunner Analysis: Hints ‘n’ Tips.


The LoadRunner Analysis tool can either be a godsend or the devil’s daughter. I think most Performance Analysts have a love-hate relationship with the Analysis tool…and refer to it as some sort of necessary evil.

With a little bit of patience, the Analysis tool is actually quite powerful and can be used to produce some fantastic graphs and information for your reports.
Everyone seems to learn their own tricks, if you have any more of your own, please submit these as comments.

In no particular order:-
  • Setting the Granularity.
    The granularity affects the graph smoothing, or number of data points. A high granularity value will reduce clutter and improve readability, but it may hide events such as spikes. The trick is to adjust the granularity to maximize readability but accurately represent any important events. Caution, however for larger performance tests setting the granularity too low may cause the Analysis tool to hang for a few minutes.
  • Annotations – Comments and Arrows.
    Make use of comments and arrows to annotate important events on the graph. In addition, these are useful for labeling data lines…particularly when there are lots of lines and you want to highlight just one of two of them.
  • Focus on just a few transactions.
    Performance tests can have sometimes hundreds of transactions and this can generate some very busy graphs. Decide on 3-5 transactions that have the highest business value or are of particular interest…focusing your analysis on just a few transactions is more valuable to the readers of your report.
  • Set your Y-Axis to be the same for all graphs.
    Graphs should be easily comparable and “tell the same story”. By default the Y-Axis is set to Automatic, which sets the minimum and maximum scale to match your data. Go to Display Options > Advanced > Axis tab, and change Minimum and Maximum values from “Auto” to something sensible. Percentage graphs should always be ranges from 0% to 100%.
    As a general rule, your Y-Axis scale should always start at zero and your ideal or SLA level should be approximately one-third up the scale. People we automatically assume response times in the lower third of your graph are “good” and the upper two-thirds are “poor”.
  • Tell a Story
    All graphs you present via the Analysis tool should tell a story. Performance Testing is there to answer the question of whether the application will scale to the load level required in production…and as a performance analysis you should produce graphs to provide evidence of either the success or failure of this performance testing.
  • Don’t Take Screenshots with the Print Screen button
    Screenshots look bad, instead use Edit > Copy To Clipboard > Graph. Alternatively, you can export graphs as image files. Go to Display Options > Advanced > Export tab. By exporting in Metafile format you will avoid any blocky pixelation which looks great for presentations or printed reports. I also recommend setting the border background colour to White for a nicer look (Display Options > Advanced > Chart > Panel > Background > Color).
  • Filter on Peak Load
    When analysing a peak load test, make use of a global filter to limit your analysis to the time that was spent AT PEAK LOAD, so that your response times do not include values measured during ramp up.
  • Percentile Graph Analysis
    Look at each response time in the percentile graph to see if there are any weird “step” patterns. The lines in the percentile graph should always be smooth, if there is a weird “step” pattern that could indicate abnormal behaviour.
  • Import External Data
    Remember that you can import data from external monitors/text files (Tools > External Monitors > Import Data).
  • Merging graphs
    This is useful but don’t put too many values on the graph, or it will be hard to interpret. Combine multiple data types can be confusing. Use Annotations.
  • Edit the Transaction Summary in Excel or Word.
    The Transaction Summary can be easily copied and pasted into either Word or Excel, this allows it to be easily edited. For example, removing columns and reduced the decimal points displayed.
  • Only use Complete Data
    After initially opening a LoadRunner result, only summary data is available. In the lower-left corner, you will notice the generation of Complete Data. You should always wait until this background process completes before editing graphs.
  • Remove the data point markers
    Use Display Options > Graph Type to quickly remove all the data point marks for your line graph. This makes it look a lot less cluttered.
  • Make use of Templates
    This feature (Tools > Templates) allows you to quickly apply the same graphs and formatting from one Analysis scenario to another. This is great for comparing results; however always double-check filters to after applying a template to ensure you are not filtering out any important data.
I hope you all find this bag of tricks useful. I know I’ve learnt a thing or two by putting it together. ;-)

Scripting RMI with Loadrunner VuGen 9.5x and 11


The Java Remote Method Invocation (RMI) is a Java application programming interface that supports an object-oriented way of  remote procedure calls (RPC). In this article I show you two ways to create performance tests scripts with Loadrunner for RMI-driven applications.
As RMI is a Java-specific protocol, the Java vuser or the Java Record replay protocols mush be used in the Virtual User Generator.
I demonstrate two approaches:
  1. Code RMI calls manually – using Java VUser script type
  2. Record the network traffic of the RMI application’s client and then play it back – using Java Record replay script type
Note: The respective Loadrunner licenses are necessary to run the created tests in the Controller.

The test application

I created a small RMI client-server test application using the Java Spring Framework. For simplicity the RMI server application is packaged as a WAR and can be run in a java web container (e.g. Tomcat). The client has been packaged into a single JAR file for easy execution. This JAR file will be reused in VUgen.  It contains the RMI interface of the application under test and the necessary Spring Framework APIs for easy RMI access.
If you would like to try the test application, here are the necessary tools:
  • Java Development kit (e.g. 1.6)
  • Apache Maven for build and dependency management
  • Apache Tomcat to execute the RMI test server application
The test application can be downloaded here: spring-rmi-poc-test-app.zip

Using Java VUser script type (using Loadrunner Virtual User Generator 9.5x)

  1. Create a new Java VUser script
  2. Use File/Add files to script… menu to add the test client jar to the script. Adding JAR files this way ensures that the contained classes will be placed to the Classpath, therefore will be available in script.
  3. Code the RMI call manually, as it would be done in any other Java program. In our example we used Spring Framework to facilitate that.
JAR file added to Java virtual user script
  1. /*
  2. * LoadRunner Java script. (Build: 3020)
  3. *
  4. * Script Description:
  5. *
  6. */
  7. import lrapi.lr;
  8. import org.springframework.remoting.rmi.RmiProxyFactoryBean;
  9. import rmitest.intf.AccountService;
  10.  
  11. public class Actions
  12. {
  13.  
  14.   RmiProxyFactoryBean proxy = new RmiProxyFactoryBean();
  15.  
  16.   public int init() throws Throwable {
  17.     // TODO update here the name of the server and the port:
  18.     proxy.setServiceUrl("rmi://localhost:1199/AccountService");
  19.     proxy.setServiceInterface(AccountService.class);
  20.     proxy.afterPropertiesSet();
  21.  
  22.     return 0;
  23.   }//end of init
  24.  
  25.   public int action() throws Throwable {
  26.     try {
  27.       lr.start_transaction("getAccounts");
  28.  
  29.       Object obj = proxy.getObject();
  30.       AccountService service = (AccountService) obj;
  31.       service.getAccounts("akarmi");
  32.  
  33.       lr.end_transaction("getAccounts", lr.AUTO);
  34.     } catch (Exception e) {
  35.       lr.end_transaction("getAccounts", lr.FAIL);
  36.     }
  37.     return 0;
  38.   }//end of action
  39.  
  40.   public int end() throws Throwable {
  41.     return 0;
  42.   }//end of end
  43.  
  44. }

Using Java Record replay script type

(This script type available in VuGen 9.5, however it functioned for me only in Loadrunner 11 well.)
This approach is useful, if it is possible to run the RMI client application together with Loadrunner capturing. For complex or platform- or environment-dependent application it is usually not the case however.
To start the example RMI client the following batch file has been created. It is to be specified in the Start Recording dialog, via seting Application type to „Executable\Batch”. I set the working directory to point the directory of the jar file.
1
%JAVA_HOME%\bin\java -jar spring-rmi-jar-with-dependencies.jar
Ensure, that in the Recording Options the RMI protocol is selected.
RMI is selected as a recorded protocol
The client application should now start and the RMI traffic will be recorded and transformed into java code.
Start recording dialog to capture RMI client application’s traffic
Here is a sample generated raw code:


  1. /*
  2. * Code generated by RMI support (Build: _build_number_) - Vugen [JAPATA] for LoadRunner Version 11.0.0
  3. * Session was recorded on: Thu Dec 10 14:34:35 2011
  4. *
  5. * Using VM version : Executable
  6. * Script Author    : Sudhakar
  7. * Script Description:
  8. *
  9. */
  10.  
  11. import lrapi.lr;
  12. import java.util.Properties;
  13.  
  14. public class Actions
  15. {
  16.   // Public function : init
  17.   public int init() throws Throwable {
  18.     // Set system properties...
  19.     _properties1 = System.getProperties();
  20.     _properties1.put("sun.desktop", "windows");
  21.     _properties1.put("sun.jnu.encoding", "Cp1251");
  22.     _properties1.put("sun.management.compiler", "HotSpot Client Compiler");
  23.     _properties1.put("sun.java.launcher", "SUN_STANDARD");
  24.     System.setProperties(_properties1);
  25.  
  26.     // Installing RMISecurityManager
  27.     if (System.getSecurityManager() == null)
  28.     System.setSecurityManager(new java.rmi.RMISecurityManager());
  29.     return 0;
  30.   }//end of init
  31.  
  32.   // Public function : action
  33.   public int action() throws Throwable {
  34.     // Lookup a remote object...
  35.     _rmiinvocationhandler1 = (org.springframework.remoting.rmi.RmiInvocationHandler)java.rmi.Naming.lookup("rmi://localhost:1199/AccountService");
  36.     _string1 = "org.springframework.remoting.support.RemoteInvocation __CURRENT_OBJECT = {" +
  37.     "java.lang.Object arguments[] = {" +
  38.     "java.lang.Object arguments[0] = #akarmi#" +
  39.     "}" +
  40.     "java.util.Map attributes = _NULL_" +
  41.     "java.lang.String methodName = #getAccounts#" +
  42.     "java.lang.Class parameterTypes[] = {" +
  43.     "java.lang.Class parameterTypes[0] = {}" +
  44.     "}" +
  45.     "}";
  46.  
  47.     _remoteinvocation1 = (org.springframework.remoting.support.RemoteInvocation)lr.deserialize(_string1,0);  // RMIComponent
  48.     _object1 = _rmiinvocationhandler1.invoke((org.springframework.remoting.support.RemoteInvocation)_remoteinvocation1);
  49.     return 0;
  50.   }//end of action
  51.  
  52.   // Public function : end
  53.  
  54.   public int end() throws Throwable {
  55.     return 0;
  56.   }//end of end
  57.  
  58.   // Variable section
  59.   java.lang.Object[] _object_array1;
  60.   java.util.Properties _properties1;
  61.   org.springframework.remoting.rmi.RmiInvocationHandler _rmiinvocationhandler1;
  62.   org.springframework.remoting.support.RemoteInvocation _remoteinvocation1;
  63.   java.lang.String _string1;
  64.   java.lang.Object[] _object_array2;
  65.   java.lang.Object _object1;
  66. }