Author: AutomatedQA Corp.
Last updated: May 4, 2009
Applied to: TestComplete 7
Preface
Load testing consists of simulating a load on web servers and services to
determine how web applications function under a massive load. TestComplete supports
load testing of web servers and web services and can be used to perform relative
testing types like stress testing and scalability testing. This article describes
how you can create load tests with TestComplete.
About Load Testing
Web applications work in an open environment. They process requests sent to them
by different clients. In most cases, the applications have to work with several
clients at one time. The purpose of load testing is to determine whether the application
functions properly under a massive load, to find the stress point that causes crashes
or delays when processing requests and to determine whether the application is scalable
enough to handle an increased workload.
Performing load tests manually is often impossible because companies usually do
not have enough testers that can perform load testing manually, at the same time.
Load testing is performed with automated testing tools
like TestComplete. Below is a list of conditions that should be met by any
load testing tool to perform quality load testing:
- It should provide features for creating load tests easily. It is very important
that load tests can be recorded and then visually modified if needed.
- It should be able to work with different web servers and browsers. Ideally, it should
not depend on the web server or browser type.
- It should be able to simulate a heavy workload by automating an arbitrary number
of virtual users. Virtual users are called virtual because they don’t exist. Only
the actions performed by these users exist. When virtual users are simulated the
tested web server "thinks" that it is working with "real" users.
- The load testing tool should be able to simulate real-life load testing conditions
as close as possible. It should be able to simulate users working with different
parts of the application, working from different workstations, different web browsers,
with different connection speeds and other real-life conditions.
- To help you perform load testing of your web application, the automated testing
tool must provide access to requests and responses. For instance, it should let
you modify request variables or the body of the request.
- Finally, the load testing software must generate detailed result logs, support result
comparison, help you find crash points and check the scalability.
Load testing tools that match these criteria will help you easily emulate real-life
conditions for your web applications and simulate the desired behavior of "users"
during load testing.
Load Testing with TestComplete
TestComplete provides support for load testing of web applications. The support
is included into the TestComplete Enterprise edition. The load testing subsystem
meets all of the criteria mentioned above.
Record Traffic for Load Testing
Load testing consists in simulating virtual users each of which sends requests to
the tested web server and receives responses from the server. To simulate the traffic
with TestComplete, you record user actions over the tested web application and these
actions are then reproduced during load testing.
TestComplete allows you to record both requests and responses sent via HTTP, HTTPS
or SOAP protocols.
HTTP is a standard protocol of the Web. It is used to access a vast majority of
web sites.
HTTPS is a secure version of HTTP. This protocol is used to secure data sent from
one computer to another via the Internet. For example, when an e-shop asks users
for their credit card numbers and other daily duties performed with an e-commerce
type web application.
Another type of request that can be recorded with TestComplete is SOAP. The SOAP
protocol is used for communication between web services and their client applications.
Since TestComplete is able to record and play back SOAP traffic, you can use it
to perform load testing of web services.
TestComplete records and stores traffic in the form of self-contained modules (tasks).
Each task is a sequence of user actions to be simulated by one or more virtual users.
TestComplete allows you to record as many tasks as you need.
TestComplete also provides a special editor that allows you to view and visually
modify properties of recorded requests and responses.
Server and Browser Support for Load Testing
TestComplete records and plays back traffic, but not browser actions, so you can
use it to test web sites regardless of the web browser you are using (Internet Explorer,
Netscape Navigator, Mozilla, Opera, or any other), regardless of the web server
type (Internet Information Services, Apache, and so on) and regardless of the platform
(Windows, Unix, Linux) on which the web server is working.
Visual Creation of Load Tests and Virtual User Simulation
Recorded load testing tasks are played back via virtual users. To simulate virtual
users, you create load tests in TestComplete.
The number of virtual users that can be added to your load test is specified by
your TestComplete license. The basic TestComplete Enterprise license supports five
virtual users. This may be enough to test a simple web application. If you want
to emulate more of a load on your web server, you can buy other versions that support
more virtual users (TestComplete Enterprise Edition with 50, 250 or unlimited number
of virtual users).
In TestComplete, load tests can be created and executed visually or in scripts.
TestComplete provides special program objects to execute load tests created visually
and to create load tests programmatically. Both of these approaches let you create
the desired number of virtual users and specify their attributes (for example, start
delay, web browser, connection speed, and others).
The visual approach is easier to use and allows even inexperienced users to quickly
create load tests. The scripting approach lets you specify test attributes from
scripts, but it requires that you have more programming experience.
A load test may contain dozens or even hundreds of virtual users that work simultaneously.
Virtual users can belong to the same load test but perform different load testing
tasks. That is, you may simulate several users that work simultaneously but perform
different actions on the server.
Real-Life Load Testing Conditions
For better emulation of real-life load testing on web servers, TestComplete offers
a number of attributes that you can use to load test your web application:
Cookie Handling during Load Testing
Cookies are data stored by the web server on the client computer. They are used
by the server for authenticating users and maintaining client-specific information.
TestComplete includes special options that help you manage cookies when the recorded
traffic is simulated by virtual users. These options help you avoid some typical
problems that may happen in your load testing: cookies may expire and this can make
your load tests invalid; cookies may cause cross impact of virtual users on each
other, and other related issues. TestComplete lets you avoid these problems.
Proxy Servers in Load Testing
With TestComplete you can play back load testing tasks even if the computer, on
which TestComplete is installed, is behind a proxy. This feature is useful if TestComplete
does not have direct access to the tested web server. TestComplete includes specific
settings that control the proxy settings.
Authentication during Load Testing
Web servers may provide anonymous or authenticated access to the contents of the
server. If an anonymous connection to the web server is used, this server can be
tested without any restrictions. However, there are some restrictions when testing
web servers with certain types of authenticated connections. TestComplete supports
most of the authentication types:
- Basic authentication
- Digest authentication
- NTLM
- Kerberos authentication
- Negotiate authentication
It provides special properties in which you can specify authentication information
that is used to connect to web sites during load testing.
Modifying Traffic during Load Testing
TestComplete provides access to properties and variables of requests to be sent
to the tested web server and allows you to modify the recorded traffic, if needed.
For example, if your web application’s pages contain an order form, then for advanced
load testing of this application you have to emulate several users inputting different
data when completing the order data.
Web browser can send data in a URL string or within the request body depending on
the request type (GET or POST type). TestComplete supports both request types and
allows you to change the request data visually or from scripts using special events
and program objects. Server response properties can be changed in the same way.
Analyzing Results after Load Testing
Results of the load test execution are stored in TestComplete’s Load Testing Log.
This page contains detailed information on the load test and specifically about
the simulated virtual users, information on each connection that was created during
the test run, and much more.
Figure 3 – The Load Test Log page of TestComplete
If you need to compare load testing results for several test runs, you can do this
on the Load Testing Analysis page. This page shows the table including the summary
results of each load test run (average process time, average request size, the total
number of requests, and more) and a diagram that graphically represents the load
test results.
Figure 4 –The Load Testing Analysis page
Load Testing Sample Project
Below is an example that demonstrates how to perform load testing of a web application
with TestComplete. Let’s perform load testing on a LoadTesting sample web application
which is located at http://localhost/loadtesting/loadtestingdemo.dll. This
web application is installed automatically with other samples when TestComplete
is installed and can be found in the <TestComplete Samples>\Load Testing
folder. To install and invoke this sample, Microsoft Internet Information Services
(IIS) 5.0 or later must be installed on your computer.
Below are instructions on how to perform load testing of this application with TestComplete.
Creating a TestComplete Project for Load Testing
Before recording traffic and creating load tests, create a new TestComplete project:
Then add the HTTP Load Testing project item to the created project. This project
item will hold all information needed to perform load testing. To add the item:
Recording Traffic for Load Testing
To record the load testing traffic:
- Start recording a keyword test or a script by selecting Test | Record | Record
Keyword Test or Test | Record | Record Script from the main menu of TestComplete:
Figure 7– Starting recording
The Recording toolbar will appear:
Figure 8 – The Recording toolbar of TestComplete
- Press Record an HTTP Task on the Recording toolbar to record a load testing
task (TestComplete records traffic as self-contained modules called tasks). TestComplete
will display the Select Load Testing Test dialog asking you to specify the
task and test names. Enter MyTest as the test name and specify local as the
name of the new task:
Figure 9 – The Specify Load Testing Test dialog
Click OK. TestComplete will start recording the traffic. - Launch your web browser and open the tested web page in it. In our example, the
page’s URL is http://localhost/loadtesting/LoadTestingDemo.dll. Enter it
into the Address box of the web browser.
- After the page opens, perform the desired actions over it. For instance:
- Click Start Web Application on the page to begin viewing the images.
- Click Continue to go to the next image.
- Click Finish after you get to the last page.
- Stop recording the task by pressing Stop on the Recording toolbar.
Figure 10 – Stopping recording
Viewing the Recorded Load Testing Traffic
Now, let’s explore the contents of the recorded local task. This task is
displayed as a child node of the LoadTesting | Tasks node in the Project
Explorer. To view its contents, simply double click it in the panel. TestComplete
will open the Load Testing Task editor in the Workspace panel:
Figure 11 – The local task in the Load Testing Task editor
On the left of the editor you can see the contents of the recorded HTTP traffic.
As you can see, it is organized into a tree of connections and requests. The Load
Testing Task editor contains two tabbed pages on the right: Request and Response.
You can select a request in the traffic tree on the left of the editor and in the
corresponding tabbed page view and modify the request’s properties or properties
of its response. For example, expand the first connection node, choose the Request
0002 item and switch to the Requests tabbed page, as it is shown in the figure above.
On this page you will see a table containing values of the selected request properties.
You can modify values, if needed. For instance, in the User-Agent field you can
change the browser to be simulated.
On the Response tabbed page, you will see the contents of the response that was
received from the tested server for the currently selected request:
Figure 12 – The Response page of Load Testing
In the Response page you can view and modify response properties.
In our tutorial, we will not modify the recorded traffic. So, leave it unchanged.
Creating a Load Test
Now we can create a load test and simulate virtual users. When recording the traffic,
TestComplete automatically created new load test named MyTest. You can see
this test in the Project Explorer panel under the LoadTesting | Tests node:
Figure 13 – The MyTest load test in the Tests collection
Let’s open this load test for editing. To do this, double-click the MyTest node
in the Project Explorer. TestComplete will display the load test editor in the Workspace
panel:
Figure 14 – The MyTest load test in the Load Test editor
In the Virtual Users table you can view and modify information on virtual users
to be simulated during the load test run and their properties.
As you can see, in our case the table contains only one row that corresponds to
one virtual user (the number of virtual users is specified in the User Count column).
This is the user that was created automatically by default.
Let’s change the load test. As you may remember, the default number of virtual users
available in TestComplete is 5. Let’s create the test that contains 4 virtual users.
This way you don’t have to download additional packages. These users will "work"
through different web browsers.
- Click within the User Count column of existing row in the Virtual Users table.
- Specify 2 in the in-place editor and press ENTER to confirm the input.
- In the User Name column specify the string for identifying testing results
of the virtual user simulation in the log. For example, enter TestUsers1.
In the log, virtual users will be identified like TestUsers1 (1) and TestUsers1 (2).
- Make sure the other columns contain the following data:
- Task: local - This means virtual users will simulate the traffic that
is stored in the local task.
- Workstation: Master - This means virtual users will be simulated on
the computer, on which TestComplete is installed.
- Browser: <Recorded> - This means virtual users will "work" through
the same browser that was used for recording the task.
- Start Delay: 0 - Start Delay is the time between the start of the
test and the start of virtual user simulation. If Start Delay is 0, the virtual
user simulation starts when the test starts.
- Connection Speed: Maximum speed - The download traffic will be simulated
at maximum allowed connection speed.
Let’s create two more virtual users:
Executing the Load Test
In TestComplete, you can run load tests from the Project Explorer, from keyword
tests, from script code or your can run them as a project test item. Let’s use the
project test item approach:
Now our project contains the test item that will run the load test:
Figure 17 – The Test Items page
To run the test, select Test | Run Project from TestComplete’s main menu.
Note that execution of a load test is invisible. TestComplete simulates recorded
traffic (sends requests to the tested web server and receives responds to these
requests from the server), but it does not simulate user actions over the tested
web application’s pages (mouse clicks, keystrokes, and so on).
After the test run is over, you can view the load testing results.
Analyzing Load Testing Results
You can get detailed information about the results of your load testing from TestComplete’s
Load Testing page:
Figure 18 – Results of the load test run
On this page you can find detailed information about simulated virtual users (user
name, task name, workstation on which the user was simulated, time of the task execution,
and more), information about each connection that was created by the task during
load testing (the number of requests the user sent during the connection, the number
of bytes sent by all requests, and others), information about requests sent during
each connection and other useful information about the load test run.
If you run the load test several times, for example, with a different number of
users, it may be necessary to compare results of these test runs. For this purpose,
you can use the summary results from the Load Testing Analysis page:
Figure 19– Summary results in the Load Testing Analysis page
Conclusion
In this article we provide an overview of load testing and its basic concepts and
also describe TestComplete’s features that are used to record load testing requests
sent to the tested server and simulate this traffic via a number of virtual users.
We hope this information will help you with your load testing. If you are interested
in trying TestComplete for your load testing needs, download and try it today.