A.: Regression testing means repeating a test already run successfully, and comparing the new results with the earlier valid results. This process is useful when you run a test on your project and then correct the project code. So, regression testing means that tests are reused after the application changes. This lets you ensure that the changes do not harm existing functionality.

How are regression tests performed in TestComplete?

• First, test and debug the application.
• Next, add something to the tested application.
• Then design a test for features added to the new build.
• Run both the old and the new tests over the new build.
• Fix and rerun until everything is clean.

This means that a new test or tests (for example, one or several script routines or one or several load tests) are added to your test project for each new build or for each new feature in a build. Then, these tests are added to the test sequence of the project:

Both the old and new tests should pass without errors. If errors exist, you keep fixing and testing. You can use whatever numbering system you want for the builds. One system uses the integer parts to indicate new functionality, or rebuilt functionality, and the decimal part for fixes (you normally use existing tests for these). So, for iteration 123 in regression testing, you would first test Build123.0, and perhaps reach Build123.7 before you could get back to work and do Build124.0, with a new script added to MainScript -- Test Build 124.0. You can read more about this on the AutomatedQA - http://www.automatedqa.com/techpapers/regression.asp. You can also look at http://www.extremeprogramming.org/.

A.: Yes, you can. TestComplete Enterprise includes the Network Suite project item that lets you create, maintain and execute tests on several workstations. These workstations must have TestComplete or TestExecute installed. You can simulate as many users as the number of computers that are involved in the testing process: the client application (TestComplete or TestExecute) will be launched on each of the computers and the script on the master machine will control activity on every client. Please see the “Distributed Testing” help topic for details.

TestComplete Enterprise also includes the HTTP Load Testing project item that allows you to test client-server applications that communicate using HTTP, HTTPS or SOAP. TestComplete can simulate up to 300 virtual users on each computer involved in testing. If you are interested in this approach, please read the “Load, Stress and Scalability Testing of Web Servers With TestComplete” help topic.

A.: The way you increase the test execution speed depends on your test project, TestComplete settings and the application under test. For detailed information about this, read the Test Playback Performance Tips article.

A.: You can record, write and run TestComplete scripts that simulate user actions with grid controls. TestComplete version 6 provides extended support for the most popular grid controls:

• Microsoft DataGridView
• Microsoft DataGrid
• Microsoft PropertyGrid
• Developer Express XtraGrid
• Developer Express QuantumGrid
• Infragistics UltraGrid
• Syncfusion Essential Grid
• Borland TDBGrid

You can retrieve and modify data in these grids, record, and replay various user actions over individual grid cells, rows and columns.

As for other grid controls, you can test them in “black-box” mode by simulating mouse clicks and keyboard input, or in “white-box” mode by using their internal properties and methods.

To learn more about testing various grid controls, see the “Working With Grids” section in TestComplete’s help. You can also download grid samples from the following pages:

A.: You can’t. To run a test, you must install either TestComplete or TestExecute on the computer where the tests will run, and then open and execute your test projects using this instance of TestComplete or TestExecute (TestExecute is a resource-friendly utility that is used to run test projects on computers where TestComplete is not installed).

You may also export the recorded script code to your Visual Basic, C++, C# or Delphi application, and this code will become part of the application’s code. You can then run this code from your application. For more information on this, see “Connected Applications” and “Self-Testing Applications” in TestComplete’s help system.

A.: You can obtain the exception call stack: compile your application with the debugger information and run under TestComplete using the DbgServices object. If an exception occurs, TestComplete will post the call stack to the log. See the "Tracing Events and Exceptions in Tested Applications" topic in TestComplete help and try the <TestComplete>\Samples\Scripts\DbgServices example.

You can find the call stack another way: compile the application with debug information, run it under the Exception Trace profiler in AQtime and then launch a TestComplete script. If an exception occurs, AQtime’s Event View panel will display its call stack.

A.: It is difficult to make scripts universal since the solution depends on the application and the development tool that the application was created with. TestComplete uses an object-oriented recording model, so changing the control’s position within a window will not break the script, so there is no need to modify the script. However, the script may fail if the control’s recognition attributes are changed (for instance, the button’s caption is changed).

To make your scripts less dependent on the application’s UI changes, you can map desired windows or controls using TestComplete’s NameMapping project item. This mapping process is called name mapping. It consists of assigning a custom name to a control and specifying property values that will be used for recognition of this control. That is, once you map an object, TestComplete attempts to recognize the control using the appropriate name mapping settings rather than recognition attributes that are used by default.

After mapping an object, you can address it in scripts by its custom name. If developers change the object’s properties, you may need to modify the name mapping settings, but not the script.

Note, however, that this option does not work for all changes. For instance, if developers will remove a control from a form, you will have to re-write your scripts.

A.: To check a value of an object property, you can use the if ... then script statement to compare the object property with the desired value and then perform actions according to the comparison results. For example:

[VBScript]

If EditBoxObj.wText = "TestValue" Then
  Call Log.Message("Correct")
Else
  Call Log.Error("Incorrect")
End If

Additionally, in TestComplete 6 you can use the following checkpoint types:

• Property checkpoints - verify an object’s property value.
• Object checkpoints - verify collections of object properties.
• Table checkpoints - verify data in tabular controls (grids, list views, list boxes, etc.)
• File checkpoints - compare files.
• XML checkpoints - compare XML documents.
• Database checkpoints - verify data stored in a database.
• Region checkpoints - compare images, or searches for an image within another image.
• Web service checkpoints - verify responses of web service’s methods.
• Web accessibility checkpoints - perform various checks of web pages (for example, checks for broken links).
• Web comparison checkpoints - compare web pages.

You can create checkpoints when recording tests, or add them at any time when writing or editing test scripts. TestComplete automatically generates the script code needed to perform the comparison based on the settings that you specify. To learn more about checkpoints, please see the “About Checkpoints” topic in TestComplete’s help.

A.: TestComplete can only trace exceptions if the application under test is an Open Application and it includes debug information. TestComplete only handles exceptions when the script calls methods or properties of an application object. This is not possible with ordinary “closed” applications. Suppose that in a Delphi application an exception occurs after pressing a button on a form. TestComplete will handle the exception if the script simulates the button press with the following instruction:

w.VCLObject('Button1').NativeDelphiObject.Click; 

NativeDelphiObject is used to call a native Delphi method. For information on namespaces for other languages see the “Using Namespaces” help topic.

If you use the following code to simulate the button press, TestComplete will not handle the exception --

w.Window('TButton','Button1').Click

// Exception is not handled!!!

Note that, if you have Internet Explorer 6.0, the following problem may occur: Some exceptions that occur in VBScript, JScript, C++Script or C#Script routines are not passed to TestComplete and the script stops without any error messages. In this case, you can use the following code to determine the description of the error.

[VBScript]

Sub Test
  Set p = Sys.Process("Project1")
  Set w = p.Form1
  On Error Resume Next
  w.VCLObject("Button1").NativeDelphiObject.Click ' We call "native" object method
  If Err.Number > 0 Then
    ' Posts the exception message to the log and 
    ' clears the Err object
    Log.Message(Err.Description)
    Err.Clear
  End If
End Sub

[JScript]

function Test()
{
  var p, w
  p = Sys.Process("Project1")
  w = p.Form1
  w.Activate()
  try
  {
    w.VCLObject("Button1").NativeDelphiObject.Click  // We call "native" object method
  }
  catch(exception)
  {
    // Posts the exception message to the test log
    Log.Message(exception.description)
  }
}

[DelphiScript]

procedure Test; 
var
  p, w: OleVariant;
begin
  p := Sys.Process('Project1');
  w := p.Form1;
  w.Activate;
  try
    w.VCLObject('Button1').NativeDelphiObject.Click; // We call "native" object method
  except
    // Posts the exception message to the test log
    Log.Message(ExceptionMessage);
  end;
end;

[C++Script, C#Script]

function Test()
{
  var p, w;
  p = Sys["Process"]("Project1");
  w = p["Form1"];
  w["Activate"]();
  try
  {
    w["VCLObject"]("Button1")["NativeDelphiObject"]["Click"](); // We call "native" object method
  }
  catch(exception)
  {
    // Posts the exception message to the test log
    Log["Message"](exception["description"]);
  }
}

A.: Tests that do not interact with GUI (for example, load tests) can be run on locked computers. GUI tests or tests that simulate user actions cannot be run on locked computers, since when the computer is working in this mode, the user session is frozen (no windows can be shown) and the testing engine cannot simulate user actions.

To work around the problem with tests interacting with GUI, you can use a remote desktop connection, or virtual machine.

Using the Remote Desktop Connections

The Remote Desktop Connection is a built-in feature of the operating system. In order to use it, the workstation, that TestComplete is running on, must have a special server-side software. This software may be included in the operating system, like it is included in Windows XP, or it can be a third-party software like VNC or DameWare.

• Lock the workstation where TestComplete is installed.
• Log onto the TestComplete workstation via the remote desktop connection from another computer.
• Start executing tests with TestComplete.

TestComplete will function in the remote connection session even if the TestComplete workstation is locked.

Note: While running tests that interact with GUI, the Remote Desktop window must be visible on the screen. If you minimize the Remote Desktop window or disconnect from the Remote Desktop Connection session, the GUI tests will fail.

Using the Virtual Machine Server Software

To workaround the problem with a locked computer, you can also use Virtual Machine software, VMware server, VMware ESX or Microsoft Virtual server 2005:

• Install TestComplete and your application on a virtual machine. Copy all files that your application and TestComplete project may require for running and testing.
• Lock the virtual machine’s host computer.
• Log onto the virtual machine from any other computer.
• Start executing tests.

TestComplete will run tests on the virtual machine despite the host being locked.

Note: Do not disconnect from the virtual machine while running GUI tests, otherwise the tests will fail.

A.: Yes, it is possible if you have TestComplete Enterprise edition with the MSBuild Integration package installed. To execute a TestComplete project suite from MSBuild projects, you should register a special task type. The registration is done via the UsingTask element whose attributes specify the task name and the name of the assembly that will perform the task. Upon registering the task type, you can use the ExecuteSolution element to initiate execution of the desired TestComplete project suite.

The sample code is similar for the fourth and fifth versions of TestComplete, except for the task registration information.

Here is a sample code snippet for TestComplete ver. 4:

File: MSBUILD.proj   
<!-- Root element of the MSBuild project -->
<Project DefaultTargets="Build"
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 
  <!-- Declares the task type, ExecuteSolution, that will be used to run TestCOmplete projects
  (TC4 is a namespace). --> 
  <UsingTask TaskName="TC4.ExecuteSolution" AssemblyName="msbuildtc4task, Version=1.1.0.0,
Culture=neutral, PublicKeyToken=948a15855538eecd"/>
 
  <!-- Target -->
  <Target Name="Build">
 
  <!-- Executes the task.
   Note that the tag name, ExecuteSolution, coincides with the registered task name.   -->
    <ExecuteSolution
      SolutionFile="C:\MyProjects\MyProjectSuite.pjs"
      StopIfFail="true"
      AdditionalOptions="/project:MyProject"
      GUIInteractive="true"
      LogFile="c:\MyProjects\MyResults.mht"
    />
  </Target>
</Project>

Here is a sample code snippet for TestComplete ver. 5:

File: MSBUILD.proj   
<!-- Root element of the MSBuild project -->
<Project DefaultTargets="Build"
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 
  <!-- Declares the task type, ExecuteSolution, that will be used to run TestComplete projects
  (TC5 is a namespace). --> 
  <UsingTask TaskName="TC5.ExecuteSolution" AssemblyName="msbuildtc5task, Version=1.1.0.0,
Culture=neutral, PublicKeyToken=5a74e076ecc789f9"/>
 
  <!-- Target -->
  <Target Name="Build">
 
  <!-- Executes the task.
   Note that the tag name, ExecuteSolution, coincides with the registered task name.   -->
    <ExecuteSolution
      SolutionFile="C:\MyProjects\MyProjectSuite.pjs"
      StopIfFail="true"
      AdditionalOptions="/project:MyProject"
      GUIInteractive="true"
      LogFile="c:\MyProjects\MyResults.mht"
    />
  </Target>
</Project>

The following is an example for TestComplete 6:

File: MSBUILD.proj   
<!-- Root element of the MSBuild project -->
<Project DefaultTargets="Build"
  xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 
  <!-- Declares the task type, ExecuteSolution, that will be used to run TestComplete projects
  (TC6 is a namespace). --> 
  <UsingTask TaskName="TC6.ExecuteSolution" AssemblyName="msbuildtc6task, Version=1.1.0.0,
Culture=neutral, PublicKeyToken=2833c3fd10be0f0d"/>
 
  <!-- Target -->
  <Target Name="Build">
 
  <!-- Executes the task.
   Note that the tag name, ExecuteSolution, coincides with the registered task name.   -->
    <ExecuteSolution
      SolutionFile="C:\MyProjects\MyProjectSuite.pjs"
      StopIfFail="true"
      AdditionalOptions="/project:MyProject"
      GUIInteractive="true"
      LogFile="c:\MyProjects\MyResults.mht"
    />
  </Target>
</Project>

A.: You can execute scripts via TestExecute in one of two ways: from the command line or by running TestExecute and selecting the needed project from its menu. In the second instance, the application icon is displayed in the system tray and you can invoke the application menu by right-clicking on the icon. The menu contains items that let you open a TestComplete project or project suite, run and stop tests, view results and modify various parameters (such as issue-tracking templates or engine settings) needed for running the tests. To learn more about running TestExecute from the command line, please read the “TestExecute Command Line” help topic.

A.: The most natural solution is to use the project suite and the project test items. Project suite test items specify which projects to run and set the execution order. Project test items define which test action to execute and in what order. A test action can be a script routine, a low-level procedure, a manual test, and so on. Project and project suite test items are defined on the Test Items Edit Page of the Project and Project Suite editors, respectively.

Project suite items are executed sequentially: once a project suite is started, it opens the first project in the list of test items. The first project runs all of its test items and then transfers execution flow to the second project suite test item, and so forth.

As an alternative way, you can use the Windows command line to start TestComplete tests. So, you can create a batch file that would run tests one after another. See the "TestComplete Command Line" help topic for syntax details.

A.: Other tools require an object repository in order to work with objects on screen. TestComplete has access to all objects on screen at any point in time, so it does not need an object repository in order to run tests.

One of the advantages that an object repository provides is if an object’s name (or some other property) changes, only the item in the object repository needs to be updated. TestComplete provides similar functionality via the Name Mapping project item. This item let’s you assign a custom name to an object and use object’s properties to find the object in the system.

If you expect that a control’s name will be changed, you can add that object to the Name Mapping feature. After this, TestComplete will use this name to address the object in both recording and run time. That is, when TestComplete records any scripts against that object, it will use the name specified by the Name Mapping setting. And like in an object repository, if the object’s name changes, you can just modify it in the name map and the scripts will run.