Remember Excel can change your data!

I am not suggesting that Excel is bad but I received a list of account numbers and passwords for use in a performance test. These account numbers are used as the username. The file came as a .CSV so I opened in Excel manipulated the file a bit (removed the header etc) and saved the file ready for the performance test.

During the test there where lots of login errors. After some investigation many where found not able to login, going back to the admin that provided we discovered that Excel had removed the leading zero once it had been imported into Excel. e.g 01234567 became 1234567 which didn’t exist and hence failed to login.

A bit of a duh moment and probably a humbling last post for 2013.

Jmeter random string example

Random String Example

Today I wanted to create a random string to emulate a different name for each iteration in my jmeter script. Luckly Jmeter provides the __RandomString function where you specify the number of characters in the random string you want and then the characters you want to make up the random string. simple!

${__RandomString(10,qwertyuiopasdfghjklzxcvbnm)}

Getting the SQL from a sp_execute statement in Microsoft SQL Profiler

While using Microsoft’s SQL Profiler to tune an application I noticed the developers have been using prepared SQL, so all I see is sp_execute and the parameters being passed. But I wanted to see the SQL that is being passed. The first pass through google provided some SQL that could be run against the management tables but I didn’t have permission to do that..

Luckily a colleague told me all I had to do was enable capturing the SP:CacheHit event and this event fires right before the sp_execute and has the text of the query in it.

Playback LoadRunner through Fiddler

If you are concerned that your LoadRunner script is not emulating user behaviour correctly and want to make sure it is download all the correct files etc, then you can connect LoadRunner to Fiddler and play the script so that fiddler captures the HTTP calls. For those that don’t know Fiddler is a debugging proxy that captures and displays the HTTP requests between a browser and the server (it does more but that is a quick summary). An example output is shown below:

fiddler example

You can then run VuGen or (don’t do it at load) through the controller by setting the proxy under runtime options.

loadrunner fiddler

PowerShell script for analysis of Jmeter results

Working on a windows 2008 server this week and wanted to get a quick analysis of the Jmeter results at the end of the test before I go through the hassle of downloading a large file to my PC for analysis. On this companies corporate network believe me it is a hassle.

Anyway, I wondered whether Windows Power Shell could be used. I think LogParser would have been better but that would require going through configuration control to get this added to the Jmeter server I have been assigned. I have configured Jmeter to produce a .csv file and this comes out in the following form:

1363470556493,48,/,200,OK,06 BrowseSite 2-2,text,true,23632,6
1363470546578,49,0601HomePage,200,”Number of samples in transaction : 1, number of failing samples : 0″,06 BrowseSite 2-2,,true,23632,6

The transactions I am interested are wrapped in a transaction controller so we see the first line is a call to the page and the second line records the response time measured in the transaction controller.
For analysis I am interested in the first 3 columns the second (page/transaction) and third (response time). The test output is written to output.csv
From the PowerShell command prompt type the following:

import-csv output.csv -header "ts","resp","page" | select page,resp | sort page | group page | select Name, {($_.Group | Measure-Object -Average resp).Average}, {($_.Group | Measure-Object -Maximum resp).Maximum}, {($_.Group | Measure-Object -Sum resp).Count}

This will give you the average, maximum and count for each page and transaction controller in the test similar to below.

jmeter powershell

There are many improvements to be made such as reporting separately failed pages but thought I would share what I have done today. Thanks to Scott Hansel for a suitable similar example to use (http://www.hanselman.com/blog/ParsingCSVsAndPoorMansWebLogAnalysisWithPowerShell.aspx).

Ignore every thing to the right of the decimal point

I am reading “The Every Computer Performance Book” by Bob Westcott (http://www.treewhimsy.com/TECPB/Book.html)

I really like his rule #7 “Ignore everything to the right of the​
 decimal point”

A performance model or test is never going to be an accurate representation of real life then why pretend by quoting figures to X decimal places. All this does is attempt to confuse the end user and imply a level of accuracy the is not there.

I like Scott Barbers emoticon approach of using happy,sad and neutral faces to represent performance results although I often just stick with red, green and amber (possibly a little more professional).

I will stick to the sentiment rather than letter of the rule. At the moment I have a 2 second response time target from a performance test. I think this deserves at least results reported to 1 decimal places to understand if we have a near miss or a long miss and of course standard deviation to get an idea of spread.

Does Jmeter give inconsistent results?

I am always interested in understanding the limitations of Jmeter so that I can make sure I avoid them and don’t use Jmeter inappropriately. I was there for interested in the following blog post by Michael Czeiszperger (http://www.webperformance.com/load-testing/blog/2013/05/measuring-web-page-load-times-using-jmeter/). Admittedly Michael is the driving force behind WebPerformance a company that has its own performance test tool. I thought Michael tried to provide a reasonable balanced.

What did concern me was the results from his test that showed that Jmeter provided inconsistent results for what sounded like a simple test.

It is always difficult to try to replicate results from a high-level blog. However, it seemed that the problem occurred at 50 threads running a 3 page test with a 1 second delay between each thread. I ran the test within the data centre from a VM against the target application. I only tested 3 static pages and did not download the embedded static content (image, javascript).

The test run from a 2 vCPU VM with 4G of memory, the test ramped up to 50 threads. I cleared the statistics (ctrl-E) and then ran the test for 10 mins before noting the results. This was repeated three times and the results are shown below:

jmeter consistant results

As you can see a more consistent set of Jmeter results. Admittedly the application is not as complex as the application tested by Michael but in away the tool should be independent of the application.

In a follow up message from Michael he said that the problem was noticable on the page that downloaded embedded static content. I enabled for one of the pages to “Retreive All Embedded Resources from HTML File” and rerun the test. Here are the results.

jmeter consistant results2