Description of JMeter core components

Test Plan

A test plan is the highest level of organization within JMeter. It describes a series of steps JMeter will execute when run. A complete test plan will consist of one or more Thread Groups, logic controllers, sample generating controllers, listeners, timers, assertions, and configuration elements.

Within BIP, test plans will perform a series of BI related functions, such as viewing a Web Intelligence report, creating an Analysis for OLAP workspace, or interacting with an Exploration View.

Complete details on test plans can be found in the JMeter User's Manual in Section 3. Building a Test Plan

Below is a screenshot of a test plan that includes several controllers, configuration elements, and results analysis listeners.

JMeter test plans are saved in Java Management Extensions (JMX) format. JMX is an open, text based format that allows for ease of portability and use across disparate systems. Because of this format it is possible to open test plans in a text editor and use the "find and replace" function to quickly alter a CMS name, web server name, or document ID that might appear hundreds of times throughout the test plan with very little effort.

HTTP Proxy Server

JMeter tests can be built manually if you know each of the URL you wish to test. However, with a complex web application such as BIP, you will rarely have a good understanding of which URL to call let alone all of the various headers and query string components that comprise an action.

To solve this problem, JMeter provides a component called an HTTP Proxy Server, that allows you to interactively record a particular workflow such as viewing and refreshing a Web Intelligence Report,
and play it back any number of times within JMeter. In this fashion, you can simply emulate a regular user of BIP and let JMeter record the actions on your behalf.

Most of the test plans on this wiki will be created using the HTTP Proxy Server component and will assume you have read this section and know how to use it.

Detailed instructions on the use of the HTTP Proxy Server can be found in section 18.9.5 HTTP Proxy Server of the JMeter User's Manual.

You can watch this short video on SCN for the step-by-step process involved in using the HTTP Proxy Server component.

Samplers

Samplers are the components that perform the actual work in JMeter. An HTTP Request Sampler, for example, executes an HTTP/HTTPS request to the web server. This could be an Open Document call, a request for the BI Launch Pad logon page, or any other single HTTP call.

Other samplers include JDBC requests, which could be used to correlate database response time with BIP response time, Java requests, WebService (SOAP) requests, and many more.

Complete details on JMeter samplers can be found in Section 18.1 Samplers

Below is an example HTTP request sampler demonstrating the core components that make up a logon request in BI4.

Logic Controllers

Logic controllers control the order in which samplers are processed. They range in functionality from Simple Controllers, which do nothing more than store a series of samplers, to the more useful and complex Random Order Controller, Transaction Controller, and Loop Controller.

Random Order Controllers can be used to randomize the order of execution in a series of Web Intelligence reports, for example, to provide a variable degree of processing to your business reports. If used in conjunction with a Loop Controller, it is possible to execute a random series of reports a configured number of times.

The Transaction Controller is one of the most valuable entities for processing SAP BI test plans in JMeter. A series of HTTP samplers containing calls for JavaScript, AJAX, and JSP requests might comprise a general activity such as "Navigate BI Launch Pad Folder Hierarchy". While you might not care how long each individual JavaScript call takes, it is very likely that the execution time for the entiretransaction would be of interest. Transaction Controllers can be used to group a series of samplers and record the execution time for the entire group during the results analysis phase.

Logic Controllers allow you to emulate business user workflows, as well as organize and orchestrate the execution of the test plan.

Details and example test plans describing the use of Logic Controllers can be found in Section 18.2 Logic Controllers

Timers

Timers allow you to add delays in between the execution of samplers, enabling you to apply think-time to your test plan. Think-time is extremely important, especially in BI workflows, because it is critical to emulate real life business usage in your test plan.

A common oversight in designing performance tests is to simply hammer the server relentlessly with consecutive requests. However, is it realistic to think a business user would view a dashboard and immediately move on to the next tab without performing some type of analysis? It is certain that some amount of analysis, or think-time, would be performed on one or more facets of the BI object.

Timers range from constant, applying the same amount of think-time between each request, to random, allowing a range of think-time to be applied to a series of requests.

Effective use of timers is critical to creating test plans that mimic real business usage, which is the ultimate intent of performance testing.

Complete details on Timers and their usage can be found in Section 18.6 Timers

Regular Expression Extractor

The regular expression extractor is both the most valuable and, at times, most daunting component used in JMeter test plans. It is part of a greater collection of JMeter post-processors whose role is to take some action after a sampler is executed.

Rather than cover all of them in detail, I will focus on the regular expression, or regex, extractor which is the single most imoportant component in designing BI4 test plans.

The first use case for the regex extractor is its use in parsing the session token from a BI4 login. Each time a user logs into BI Launch Pad, the CMC, or any other web based application, a unique token is generated to represent that user's session. In order for JMeter to process a complete workflow, it is necessary for the tool to capture and use the dynamically generated session value.

The session token, called bttoken in BI4, is returned in the response body of the logon page. By leveraging a regex extractor, it is possible to extract that value and store it in a JMeter variable for use later in the test plan.

Regex extractors are used throughout BI4 test plans, and their uses include extraction of Web Intelligence sessions, serialized session ID, temporary image retrieval, and SOAP actions within dashboards and Analysis for OLAP workspaces.

For details on building regular expressions, please refer to the Apache tutorial on the subject

Comprehensive documentation about the regular expression extractor can be found in Section 18.8.1 Regular Expression Extractor

Assertions

Assertions allow you to test facts about responses received from the application being tested. Using an assertion, you can test that BI4 is returning the results you expect it to.

This component has incredible possibilities to enable automated regression testing of the BI4 application suite. For example, an assertion attached to a sampler that executes a Web Intelligence report could assert whether the expected HTML, or more specifically, an individual calculation, is correctly returned before and after an SAP patch is applied.

Another use of assertions might be in high volume load testing, where the intent is to determine how many requests the application can successfully process before crashing or becoming unavailable. An assertion attached to a Crystal Report view request could test for a 500 Internal Server Error string and log the exact request where the test starts failing.

Assertions are covered in Section 18.5 of the JMeter User Manual.

Listeners

Listeners provide a means to collect, save, and view the results of a test plan. Listeners store results in a detailed XML format, or a more efficient (but less detailed) CSV format. Their output can also be viewed directly within the JMeter console.

Some examples are:

1. Graph Results, which provides a graphical representation of response time, throughput, and other metrics associated with a test. 
   
2. View Results Tree, an extremely helpful tool in debugging, that returns the entire request and response body of every sampler in the test. While valuable in the design phase of a test, this component is very resource intensive and is generally not recommended during high volume test executions. 
   
3. Monitor Results, which can be used to provide a graphical output of the health of an application server, such as Tomcat, and demonstrate utilization and health over the course of a test plan execution. 
   
   
4. Summary Report, a high level overview of each independent logic controller, or request, in your test plan. Measures things like average response time, error percentage, and throughput.

There are many other listeners and examples of the most useful will be covered in greater detail in the design part of this wiki.

Listeners are covered in Section 18.3 of the JMeter User Manual.

Config Elements

Configuration elements can be used to set up default values and variables for later use by samplers. While there are a wide variety of config elements available for use with JMeter, here we will focus on 3 in particular. Those are the CSV Data Set Config, HTTP Request Defaults, and HTTP Cookie Manager.

1. CSV Data Set Config - This element is used to read lines from a file, and associate them with a JMeter variable for use in a test plan. In conjunction with BI 4 it is possible to pass unique values for usernames, prompt values, report ID and much more. By configuring a CSV Data Set you can, for example, have each of 10 threads access the system with a different user ID. It is also possible to pass a wide variety of prompt values into a Web Intelligence report. The use cases for the CSV Data Set Config are very extensive and allow you to add legitimate business logic to your test plans. 
   
   
   
2. HTTP Request Defaults - This element allows you to set properties, such as server name and port for every HTTP request in a test plan. This can be especially helpful when comparing performance in 2 different, identical environments. Rather than update the server name in every single HTTP sampler, you can set it in a single location and have that propagate to the entire test plan. 
   
   By modifying the port it is possible to compare performance between an environment with a Tomcat application server, and the same environment with a cache enabled Apache web server in front of it. This enables quick performance comparison between the two configurations. 
   
   
   
3. HTTP Cookie Manager - This element enables cookies to be stored uniquely for each thread in your test plan. BI 4 uses cookies to store some critical information such as the JSESSIONID of the application server instance. 
   
   It is also possible to use the cookie manager to pass a username to the Trusted Authentication engine for Single Sign On (SSO). The HTTP Cookie Manager should be added to every test plan you design unless your application server specifically disables the use of cookies. 
   
   
4.