Author: AutomatedQA Corp.
Last updated: April 13, 2009
Applied to: TestComplete 7
Introduction
TestComplete, is AutomatedQA's automated software testing tool for Windows and web applications. Automated tests provide a number of benefits to QA engineers: automated tests are more consistent, they make the testing cheaper and faster than manual testing. QA teams can perform more tests when running automated tests, however, for some applications even automated tests may take a significant amount of time and run for more than a day.
QA teams often wonder whether it is possible to increase the automated test execution speed and how they can do this. To answer these questions, let’s first consider what happens when TestComplete executes an action on an object during an automated test run, for instance, selecting a tree view node:
- TestComplete searches for the tested object (tree view control) in the system. If the object does not exist at that time, TestComplete waits for it.
- After TestComplete locates the object in the operating system, it determines its type and adds the appropriate methods and properties to the corresponding test object. For instance, if it is a tree view control, TestComplete adds tree view specific methods and properties to the appropriate test object.
For each object it finds, TestComplete analyzes every possible option for extending the set of automated test methods and properties. This process doesn’t take long to explain, but it can be time consuming because it must be performed many times. For instance, if the control resides in an Open Application, TestComplete reads debug information and retrieves information about the control’s methods and properties and then adds these methods and properties to the scripting object. If the control’s methods and properties can be exposed by the MSAA or UI Automation technologies, TestComplete also adds the appropriate methods and properties to the object.
- After TestComplete finds the method specified in the automated test it calls this method and proceeds with the next automated test command. However, this command may also have to wait until the previous action is finished. For instance, if you expand a tree view item and the tree view uses animation effects for drawing the items, the next command will not be able to work with the tree view until the animation is over.
Keeping in mind the described actions, we suggest the following approaches for improving the performance of your automated tests:
1. Disable Unused Plug-Ins in TestComplete
TestComplete can access internal objects, methods and properties of tested applications. We call these applications Open Applications. The support for some Open Applications and third-party controls is implemented by various plug-ins in TestComplete. TestComplete uses these plug-ins to associate the control (or window) with a scripting object. By disabling the plug-ins, you can narrow down TestComplete’s functionality and speed up the search for objects and methods.
You can disable the plug-ins in the File | Install Extension dialog. We recommend that you disable the plug-ins that concern Open Applications and third-party controls. For instance, if you test a Visual Basic application that uses standard controls, then you can disable the .NET Classes and MSAA Support plug-ins as well as the plug-ins that provide support for third-party controls (like Infragistics or Component One). But use caution! Don’t disable the plug-ins that are used in your automated tests.
2. Filter Open Application Processes in TestComplete
You can limit the number of processes to be recognized as Open Applications by commanding TestComplete to only recognize the tested applications. In this case, TestComplete will not spend time trying to expose objects of other applications.
To specify the process filter:
- Open your TestComplete project.
- Open the project editor and then switch to the Properties page.
- Choose the Open Applications | Process Filter settings group. This group contains a list of processes to be included in the process filter and the Process filter mode box.
- Select Use only tested applications in this box to command TestComplete to recognize only those Open Applications that are included in your project’s list of tested applications.
You can also specify the desired processes name in the Processes list and then choose the Use only selected processes mode. This will tell TestComplete that only the processes added to the list should be recognized as Open. - Select File | Save from TestComplete’s main menu to save the changes.
3. Turn Off the Debug Agent™ in TestComplete
Open Applications expose objects, methods and properties defined in the application code, so you can use these objects, methods and properties in your automated tests. Some applications are always Open to TestComplete while others are "closed" by default and you typically need to compile them with debug information to make them Open. Debug information allows TestComplete to detect the names of object methods and properties, names and types of method parameters and to obtain other critical information.
To parse debug information and get access to the application’s internal objects, methods and properties, TestComplete uses a special subsystem called Debug Info Agent™. Reading and parsing debug information can take some time, and this can significantly slow down the automated tests execution. So, if your automated tests do not use internal methods and properties of the application, you can disable the Agent and decrease the time TestComplete needs to obtain information about objects’ methods and properties.
To disable the Agent:
- Open your test project in TestComplete.
- Open the project editor and then switch to the Properties page.
- Switch to the Open Applications | Debug Agent page.
- Clear the Enable Debug Agent check box.
- Select File | Save from TestComplete’s main menu to save the changes.
4. Optimize the Wait Time for Automated Tests
When simulating user actions on controls and windows, TestComplete’s automated tests may need to wait until the desired window, control or object becomes available. The test engine lets you specify the wait time for individual test commands as well as for the entire project:
- Keyword tests. In keyword tests you specify the wait time for individual operations in the Auto-wait timeout column. By default, this column is hidden. To make it visible, right-click somewhere within the keyword test editor, select Field Chooser from the context menu and then drag the column from the ensuing window to the editor.
The operations, for which the Auto-wait timeout value is specified, will use this wait time. The operations, for which the Auto-wait timeout value is 0, will use the wait time specified by the project settings (see below).
- Scripts. In scripts, the wait time depends on the method that is called. If you pause the automated test’s execution by a call to a
Wait method (for instance, WaitWindow or WaitProperty), the wait time is specified by the method’s Timeout parameter. If you call methods that simulate user actions on a window or control, then the wait time is specified by project settings.
To speed up the automated test’s execution, you can decrease the wait time by decreasing the timeouts specified by project settings and using operation-specific timeouts (or the Wait methods) in the appropriate places of your automated tests. If you do this and use the keyword test operations or Wait methods with shorter timeouts than the timeouts specified by the project settings, the speed of your automated tests will increase.
There are two project settings that specify the wait time. They affect both keyword tests and script routines:
- Auto-wait timeout
When TestComplete attempts to perform an action on a control or window and this control or window is not available, TestComplete waits for the object. The Auto-wait timeout project setting specifies this waiting time. To modify it:
- Open your project’s properties for editing.
- Select the Playback settings page and decrease the value in the Auto-wait timeout box.
- Method invoke timeout
This setting is used when an automated test calls the method or property of an Open Application but the application is busy and cannot execute the method. The Method invoke timeout project setting specifies the waiting time that will be used in this case. To modify the setting:
- Open your project’s properties for editing.
- Select the Open Application | General settings page and decrease the value in the Method invoke timeout box.
Note that using too small timeouts may cause your tests to function improperly: the automated test may decide that the needed object does not exist while it should just wait for a little while longer. To work around the problem, use the operation-specific timeouts (or Wait methods) in the appropriate places of your automated test.
Some methods that simulate user actions may contain parameters that specify the time to wait. For instance, the Close method of the Window object has a parameter that specifies the number of milliseconds the method will wait until the specified window is closed. By default the delay is 2 seconds. To speed up this method, you can pass 0 as a parameter. For example, call wndObj.Close(0) and then call the WaitProperty method to check if the window is closed, by “waiting" until the window’s Exists property is False.
A possible scripting alternative to the Wait methods are loop statements that pause the automated test's execution until the desired condition is met (for instance, until a property equals a value or until the desired window appears on the screen). The Wait methods are faster than the appropriate loop statements, so, to speed up your test, replace these loops with the appropriate Wait method (WaitProperty, WaitWindow, WaitChild, WaitProcess or any other).
If you use the WaitWindow or WaitProcess method just to check whether the desired window exists, then set the Timeout parameter to 0. In this case, the method will not wait until the window (or process) becomes available and will return the result immediately. The same concerns the WaitProperty method.
5. Adjust Delay Options for Automated Tests
The Playback group of TestComplete’s project properties contains a number of settings that specify time values that are used during the automated test's execution:
- Delay between events – Specifies the number of milliseconds to wait after the automated test script simulates any user action and before it continues with the next action. The more this value is, the slower the automated test's execution is.
- Key pressing delay – Specifies the number of milliseconds to wait after each keystroke is simulated. The more this value is, the slower the keystroke simulation is.
- Dragging delay – Specifies the number of milliseconds it takes the mouse cursor to pass 20 pixels while performing a dragging operation during automated test playback. The more this value is, the slower the dragging events simulation is.
- Mouse movement delay – Specifies the number of milliseconds it takes the mouse cursor to pass 20 pixels during automated test playback. The more this value is, the slower the mouse moving event simulation is.
Check the values of these options. If for some reason the values are large, decrease them. Note however that using minimal values may cause your automated test to function improperly.
6. Increase the Double-Click Speed in the OS
To simulate the double-click events faster, increase the double-click speed in the operating system’s settings:
- Open the Control Panel | Mouse window.
- Switch to the Buttons tabbed page.
- Drag the slider in the Double-click speed section to Fast.
- Press OK to save the changes.
This will not only increase the speed of double-clicks, but also speed up single clicks that are simulated at two points that are close to each other. When simulating these single clicks, TestComplete pauses the execution after the first click and waits until the double-click timeout elapses. This is done so that the tested application does not confuse two single clicks as a double-click.
Note however that fast double-clicks may be inconvenient for the tester working on the computer.
7. Turn Off Windows UI Visual Effects
The Windows operating systems include a number of settings that let users apply various visual effects to menus, tooltips and windows. To increase the test speed, use the settings configuration that will provide the best performance.
The way you modify the settings depends on the operating system you use:
- In Windows XP or Windows 2000:
- Open the Control Panel | System dialog box.
- Switch to the Advanced tabbed page.
- Press Settings in the Performance section of the page. This will invoke the Performance Options dialog.
- On the Visual Effect page of this dialog, select Adjust for best performance.
- Press OK to save the changes and to close the dialog box. You will return back to the System dialog box.
- Press OK to close the Control Panel | System dialog box.
- In Windows Vista:
- Open the Control Panel | System window.
- Select Advanced system settings from the Tasks panel. This will invoke the System Properties dialog.
- In the dialog, switch to the Advanced tabbed page.
- Press Settings in the Performance section of the page. This will invoke the Performance Options dialog.
- On the Visual Effect page of this dialog, select Adjust for best performance.
- Press OK to save the changes and to close the dialog box.
The disadvantage of the described approach is that the changes may affect the tested application and it will behave differently than what you expect.
8. Use Alternate Methods in Automated Tests
TestComplete has more than one way to simulate most user actions. Recorded automated tests normally use high level methods because they are more concise, easy to read and encapsulate more verifications. However, more basic commands are available that may playback faster. For instance, the Sys.Desktop.KeyDown and Sys.Desktop.KeyUp methods work on a lower level than the Keys method of the Window or onscreen objects. Similarly, the Sys.Desktop.MouseDown and Sys.Desktop.MouseUp methods work on a lower level than the Click routine, and the Click routine, in its turn, works on a lower level relative to the ClickItem, ClickItemXY and other similar routines.
Typically, lower level methods make fewer calculations and verifications and therefore they work faster than higher level methods. So, if you are going to execute a group of high level methods, you can perform the needed verification operations before this group is executed and then use lower level methods to simulate user actions in your automated tests.
The disadvantage of the described approach is that replacing high level methods with the lower level ones can make the automated test complex, less stable and less portable.
9. Optimize Recorded Low-Level Automated Tests
TestComplete normally records mouse and keyboard actions without delays so that automated tests are efficient and quick. The low-level procedures record all user actions along with the delays between each action. If you are using the low-level automated test type then it may be possible to edit them to save time.
Recorded low-level procedures capture the exact actions and timing of the tester so they can often contain extra mouse movements that aren’t needed to playback the test. And they often capture delays between events that are longer than necessary.
You may be able to increase low-level automated test speed by deleting non-critical mouse movements and decreasing the delay time between events. If you try this tip, be sure to verify that the test playback still works as expected since your edits may significantly modify the low-level procedure behavior.
10. Optimize Working with PDA’s in Automated Tests
When you test applications designed for mobile platforms, for instance, WinCE applications, you may increase the performance of working with the connected device from TestComplete.
So, if you use the
PDA Controller panel to interact with the connected device, you can turn off the
Auto Refresh button. In this mode TestComplete will only refresh the PDA’s screen picture by demand, that is, by clicking the
Refresh button. This measure increases the speed of collecting the information about the connected device, and in the final analyses, speed up the TestComplete’s work. This happens because the refreshing screen in the
PDA Controller takes a major part of the time spent exchanging data with the device.
Are These Tips Helpful?
These optimization techniques are general, and we hope that they will be helpful to most users. However, you should remember that every automated test project is unique and can be improved in certain areas. So, some of the described techniques may have little effect on your project while others may help tremendously. If your optimization efforts succeed, do not hesitate to share your experiences on the TestComplete forums or community web site. We appreciate your comments.