Chapter 5: Load/Performance Testing of Websites

Preparing for Load Testing

In preparing for load testing it is utterly important to address a number of concerns with regards to the target server under test. As load testing helps to benchmark performance behavior of a server, it is important to be able to identify the general expectations and other matters that would normally be taken into account in order to carry out a successful load testing.

Need to Know

As noted earlier, load testing in this matter subjects the application server to work approaching its limits. Obviously, the limits will need to be clearly defined, understood, and agreed upon by the stakeholders, namely your superior(s). In addition, performance metrics need to be clear in order to keep the performance goals in check.

Important expectations as for any load testing include:

• A suitable time to load-test the application, for instance when no development work is taking place on the server (load testing may cause the server to crash) and/or no other users are accessing the server (else the testing results would not yield the correct measures)
• The performance metrics, accepted levels, or SLAs and goals
• Objectives of the test
• The Internet protocol(s) the application is(are) using (HTTPS, HTTP, FTP, etc.)
• If your application has a state, the method used to manage it (URL rewriting, cookies, etc.)
• The workload at normal time and at peak time

It is often advisable that any form of performance testing, inclusive of load testing, is performed on a functionally stable application, regardless of the environment where the application is located. Load testing is best done when the functionality of the Application under Test (AUT) is stable enough to yield consistent and correct results.

Some Helpful Tips to Get Better Results

• Use meaningful test scenarios (use cases are helpful) to construct test plans with 'real-life' test cases.
• Run JMeter on a machine other than that running the application you are testing.
• Make sure that the machine running JMeter has sufficient network bandwidth, so the network connection has little to no impact on the results. Also, the machine running JMeter should have enough computing power (memory, CPU) to generate load.
• Let JMeter Test Plan run for long time periods, hours or days, or for a large number of iterations. This may yield a smaller standard deviation, giving better average results. In addition, this practice may test system availability rate and may highlight any decay in server performance.
• Ensure that the application is stable and optimized for one user before testing it for concurrent users.
• Incorporate 'thinking time' or delays using Timers in your JMeter Test Plan.
• Conduct tests under a monitored and controlled environment, to prevent other users from affecting JMeter results.
• Keep a close watch on the four main things: processor, memory, disk, and network.
• Only run JMeter against servers that you are assigned to test, else you may be accused of causing DoS attacks.

Using Jmeter Components

For practical and realistic reasons, we will use an existing remote server to test its performance. First of all, we will create some useful scenarios as the baseline of our test.
First, we will need to determine the test cases. Generally, we will test five key scenarios:

• Homepage
• Keyword Search—New Visitor making a keyword search
• Create Account—New Visitor creating an Account
• Select A Title—Registered Visitor selecting a featured title
• Add to Cart—Registered Visitor adding selection to cart

These scenarios will be included in a single JMeter Test Plan, for simplicity reasons.

Recording HTTP Requests

A fast way to capture the HTTP pages of this application is to record every request made to the server. For this we will need to use a special-purpose Configuration Element: HTTP Proxy Server.

Since Proxy Server element records any page requests, besides user requests, it will also record background requests made by the browser as modern browsers do. As we would like the Proxy element to record only requests to the target server, consider setting web filter to allow the current browser to make requests only to that server. Otherwise, unnecessary requests will only clutter your recording. You may find these web filters as an option in any Internet security tool or software currently running on your machine.

Run JMeter (double-click JMeter.bat from jmeter/bin folder). You will see the default Elements, which are Test Plan and Workbench.

1. Right-click on Workbench 
2. Select Add | Non Test Elements | HTTP Proxy Server

Figure 5.5.1: Add HTTP Proxy Server

An HTTP Proxy Server configuration Element will appear as a child of Workbench. Bear in mind that child elements of Workbench are not saved as part of Test Plan, but you can save them separately. In the HTTP Proxy server configuration Target Controller lets you determine where the recorded requests will be placed and the Grouping option allows the recorded pages to be grouped or left as individual requests. Notice that the port we are using is the default port, following the browser setting in the next instructions.

Figure 5.1.2: HTTP Proxy Server

Next, we will change the setting of your favorite browser to access a proxy (in our case JMeter HTTP Proxy), listening to the same port. You may also use port 90 for both the browser and JMeter Proxy setting.
The following figure shows the setting for Mozilla Firefox:

Figure 5.1.3: Proxy Settings Mozilla

The following screenshot shows the setting for IE:

Figure 5.1.4: Proxy Settings IE

We are expecting that a single page request will make several embedded requests for images, JavaScript files, CSS files, etc. Therefore for a more managed recording, it is a good practice to create controllers that can contain the sub-requests for each request.

1. Right-click on HTTP Proxy Server 
2. Select Add | Logic Controller | Simple Controller

Repeat so we have five Controllers, or you can simply copy and paste.
Name each Controller:
Homepage, Keyword Search, Create Account, Select Title, and Add to Cart. Then configure the Target Controller to HTTP Proxy Server | Homepage, while the other defaults remain.

Figure 5.1.5: Select Target Controller

Now we are ready to record our first page request. As we return to JMeter, simply press the Start button at the Element Controller, and use your browser. Remember that JMeter records all HTTP requests from the browser you are using; therefore, with all the rich browsers and with all the add-ons we have today, you might want to filter that only requests to the targeted server are allowed. You may configure your browser or firewall to do so.

Click the Start button, and type the URL of the server you want to test in the Address bar. You will see that JMeter has begun to record the request to the homepage and its sub-requests into Homepage Controller.

Figure 5.1.6: Requests Target Controller

Next, on the Target Controller, select HTTP Proxy Server | Keyword Search, return to the browser and type in a keyword and click search. JMeter will record the following search page and its sub-requests in the Keyword Search Controller and nowhere else.

Figure 5.1.7: Requests another Target Controller

Repeat for the Create Account, Select Title, and Add to Cart Controllers, switching to the respective Target Controllers.

The final recording screenshot is as shown in the following figure:

Figure 5.1.8: Requests of Last Target Controller

Once the recording is over, we will save these by right-clicking on the HTTP Proxy Server Controller and saving it in a folder of your choice. As you will see, each request may or may not generate sub-requests for files. The caching feature of today's web browsers allows these files (*.png, *.jpg, *.css, *.js, etc.) to be stored in the browser's cache the first time they are downloaded. Unless there were changes in the main request, the browser will not make new requests for these cached files as it will simply load them from the local cache. How is this feature helpful in benchmarking the performance of an application? To iterate, this has become a test goal decision to make whether the application should be tested to evaluate performance for first-time visitors or existing visitors. If we are testing for first-time visitors, we can use the first recording to simulate first-time users. Subsequent recordings can be used to simulate existing users.

Alternatively, we can configure the Proxy Server Configuration Element to exclude recording of particular file type(s), as the following figure indicates

Figure 5.1.9: URL to Exclude

Subsequent recording of similar actions using the new configuration of HTTP Proxy Server will exclude the caching files.To remove elements, highlight the Controllers in the Test Plan and press delete.

Creating the Test Plan

We will begin by creating a single Thread Group (Users Group) that we will configure later as we expand the Test Plan.

1. Right-click on the Test Plan element
2. Select Add | Thread Group

A Thread Group Element will appear. Configure the Number of Threads to 10, Ramp-Up Period to 1 second, and Loop Count to 50. If you wish, you can set the Scheduler so your test plan can run automatically on the pre-determined time and date.

We want each request to target only one server, therefore, we set a request default to serve this purpose. Add to the Test Plan Config Element | HTTP Request Defaults:

Figure 5.2.1: Add HTTP Request Defaults

JMeter provides this capability by Pre-Processor | User Parameter Element or Config Element | CSV Data Set Config. The User Parameter Element allows the user to specify unique values for User Variables specific to individual threads. The CSV Data Set Config element serves the same purpose; however, these values are read from lines from a file, and split into variables that later can be used throughout the life of the thread.

For providing a large number of users, CSV Data Set Config would be a better choice. The following figure will compare both the elements.

Figure 5.2.2: Add User Parameters

Figure 5.2.3: Add CSV Data Config

Since we need to create 500 unique accounts, the natural choice would be using CSV Data Set Config Element. It is advisable that the CSV file created for this test be located in the same directory as the Test Plan. This element gives you the option to define the delimiter of your data file. Since we want only these 500 unique accounts created and nothing more, we do not choose recycle on EOF, which would reread these pairs from the beginning of the file once the whole file is parsed.
These data will need to be parsed into the appropriate parameter(s) using the function syntax: ${VARIABLE-Name}. We will use these account pairs in Create Account | /login Sampler where you see the corresponding variable names or parameters (email, password, confirm password) are captured in the Sampler. In the following figure you may notice the Value for these parameters corresponds with Variable Names defined earlier in the CSV Data Set Config Element.

Figure 5.2.4: Parameterized the appropriate Parameter

Adding Listeners

We are now ready to add Listeners to our Test Plan. As we are evaluating performance based on scenarios, each scenario Controller will have its own Listener. One Listener is sufficient to capture the performance data, as the saved data can be represented in various ways according to the Listener selected to view these data. The following steps will give you a better walk through.

1. Right-click on the Controller
2. Select Add | Listener

Figure 5.3: Add Listeners

In the Filename text-field simply type the name of the XML or JTL file, along with its extension, that will store the results for the requests in this Controller. By default this will be located in the \bin folder of your JMeter installation path; however, you may choose to specify a different location. Follow similar steps for all other Controllers.

Running the Test Plan

We are now ready to run the Test plan we have built. We may rename the Thread Group to "10 Users" for better documentation. Look at the tiny gray indicator box at the top right of the Control Panel. The numbers beside the box indicate number of active threads vs. total number of threads in the Thread Group.

JMeter requires saving the Test Plan before running; unless indicated otherwise, it will save the Test Plan in the \bin folder of your JMeter installation path. To run the Test Plan, go to the Run menu and select Start. As soon as it runs, the gray box will turn green as JMeter ramps up the total number of active threads to 10. You will see that JMeter takes approximately 5 seconds to activate the total number of users with a delay of 100ms (1000ms per 10 users) between subsequent threads. This demonstrates the 1 second ramp-up time we have set in Thread Group earlier.

Figure 5.4: Add View Results in Table

Interpreting the Results

Once the test is completed, we can now retrieve the results we have saved for each Controller. With the exception of the Assertion Result Listener, the saved data can be viewed in numerous forms. Let us use HomePage.xml as our specimen dataset. Add more Listeners to this Controller: Summary Report, Aggregate Result, and Graph Results. To retrieve the results for this Sampler, type in the name of the file to which you saved data for this Sampler, and press Enter. The following snapshots show the result views.

For Graph Results, the Data legend shows us the widely dispersed data, representing the large value of the Standard Deviation across all samples for this Homepage Sampler. In the case where the results are highly skewed or not symmetrical using 'mean' would result in inaccurate representation of response time. The Median value, which is found in Aggregate Report and Graph Results would closely approximate the response time.

Figure 5.5.1: Add Graph Results

The saved results can be viewed in various forms. The following snapshot is of the saved test result viewed using the Summary Report Listener.

Figure 5.5.2: Add Summary Report

In the case where data distribution is even, or follows the 'bell' distribution, median and average will have the same or only slightly different values. We can see the data distribution pattern by adding a Spline Visualizer Listener using the filename of the file used to store the sampler results. The following shows us the distribution graph for requests to this page.

Figure 5.5.3: Add Spline Visualizer

The following snapshot is of the saved test result viewed using the Aggregate Report Listener. Note that Summary Report and Aggregate Report display the same set of test results differently, following different computation over the same data.

Figure 5.5.4: Add Aggregate Report

Monitoring the Server's Performance

The JMeter Plugins project provides us with the tools necessary to integrate critical performance data directly into our JMeter test plan.  When used in conjunction with a JMeter Monitor Test Plan for retrieving statistics from the Java Application Server, we should be able to identify any resource related bottlenecks that exist in the system and take action accordingly.

Deploy the JMeter Plugins
Downloaded latest version of the JMeter Plugins from Google Code.  To extend JMeter functionality, it is a simple matter of extracting the JMeterPlugins.jar file from the archive into the /lib/ext directory of the JMeter installation.

Figure 5.6.1: JMeter Plugins

Figure 5.6.2: Apache Jmeter 2.7

We can add the Server Monitor component to any of our JMeter Test Plans by selecting the 'jp@gc - Servers Performance Monitoring' (JMeter Plugins @ Google Code) component from Listeners.  For simplicity, I suggest adding a new Thread Group to store all of the performance monitors you might add.  Since there is a limitation of one metric (CPU, Memory, etc.) per listener, you will need to add an additional listener for each metric you with to capture.
You can configure the Thread Group with the default settings.  The listener only captures data while any thread group is running, so it isn't necessary to configure it specifically.

Figure 5.6.3: Add Server Monitoring

Figure 5.6.4: Add PerfMon Metrics

For more details check the below link:

https://code.google.com/p/jmeter-plugins/wiki/PerfMon