When your applications become larger, a potential problem arises: new bugs are added when you make changes in the code. The program is relatively stable and a change is requested. The change is implemented, but errors begin to appear in places where everything worked before. That is a nightmare for any developer, in such a way that some avoid making substantial changes in the code. That becomes worse when we check our code and see that it's a complete mess and we need to clean up everything (we can call this refactoring).

How can we make these changes without creating a new problem? The answer lies in automated tests: just create a test routine, which is executed a modification is implemented, checking if new errors were introduced. This test routine could be done manually, using a checklist - when there is a change, the list items are verified and, if there are no errors, you go to the next step. With a large program, it takes time to execute all tests, and you can skip some. After some time, the tests are relaxed or they aren't executed anymore, and the problems arise later.

A solution is to use Unit testing, which can be done using DUnit (for Delphi) or NUnit (for .NET): you create automated tests for the class methods and they are executed at each change, checking if new errors were introduced. That's a large step, because when we have tests we can trust, we can be sure that we haven't introduced new errors in our methods and that we have a test base that is executed always in the same way.

Unit testing has some limitations:

• How can I guarantee that the interaction between methods doesn't generate new errors? We have tested the methods individually, and we cannot guarantee that the interaction between classes is correct. For example, a class works fine with just one instance, but how does it work with two active instances?
• How can I do visual tests: if a method is visual (a drawing routine, for example), how can you say the result is correct without verifying it manually?
• How do I test the user interface? We must emulate mouse clicks or keyboard key presses to get some result - how can we do that?
• How can we verify how the application behaves after some time? How do we know if, after an hour, the application won't hang or show errors?

We could create tests that overcome these limitations, but this isn't always easy, many times the test to be created is more complex than the code we are testing (in that case, should we create a test project to test the test project?).

When I faced a problem like these, to test a visual code which generated an image from biological data, I asked to people that used unit testing and someone suggested me the use of TestComplete, from AutomatedQA.

After downloading it, I saw it was a very complete testing tool and could be the answer to these limitations and for many others that could appear. It allows us to create a complete test suite, testing both the UI and its internal properties.

A 30 day evaluation version of TestComplete is available here.

After downloading, installing and executing the program, a screen like Figure 1 is shown.

Figure 1 - TestComplete main screen

Below the toolbar are the tabs representing the program's windows. Each tab corresponds to a window, which floats when you double click it:

• Script editor is where the project scripts are shown (and can be edited)
• Suites shows the test suites available for this project
• Test log allows us to view the test results and logs from past tests
• Object browser shows the executing processes is the machine. Each process has the Windows associated to it and, in some cases, even the window properties. Some properties can be edited here, and they affect the value in the executed program.
• Stores shows the storage components used for the program tests, like generated images (that will be used as standards for compares) or files.
• ActiveX wizard shows the list of events handled by TestComplete
• Low level recorder shows the low level procedures, where mouse and keyboard events are recorded individually.

To evaluate TestComplete, we will create a program that draws Lissajoux curves. These curves can be obtained using an oscilloscope, where we input sinusoidal signals in the horizontal and vertical directions. Some samples of this kind of curves are shown in figure 2.

Figure 2 - Lissajoux curves samples

The program calculates the points' positions at many angles, drawing lines between them. The code to draw the images is:

procedure TForm1.DesenhaCurvas;
var
  i : Integer;
  Ang : Double;
  PosX,PosY : Integer;
begin
  PaintBox1.Canvas.FillRect(Paintbox1.ClientRect);
  PaintBox1.Canvas.MoveTo(Trunc(120*sin(FAlfa))+150,Trunc(120*cos(FBeta))+150);
  Ang := 0;
  for i := 0 to 720*100 do begin
    Ang := Ang + pi/720;
    PosX := Trunc(120*sin(FM*Ang+FAlfa))+150;
    PosY := Trunc(120*cos(FN*Ang+FBeta))+150;
    PaintBox1.Canvas.LineTo(PosX,PosY);
  end;
end;

The program to be tested is shown in Figure 3. We fill the values of M, N, Alfa and Beta, clicking the button. The program then draws the image.

Figure 3 - Lissajoux curves program

The curves have some special characteristics:

• If M and N are the same and Alfa is 90+Beta or Alfa is 90-Beta, a line is drawn
• If M and N are the same and Alfa and Beta are the same, we have a circle

With these specs, we can test our program. We open TestComplete and create a new project there. When we create a new project, TestComplete asks the script language to use. We can choose any of the script languages available, it's not tied to the language which the tested program was written. We'll choose DelphiScript, as it is closer to Delphi, thus easier to use. We choose Lissajoux as the project name, putting it in the same directory of our application. A Form and an unit, similar to Delphi's are shown.

Then we will add our application to the tested applications. We select the File/Tested Applications menu option, adding our application by clicking on the Add button. We close the dialog box and execute our application, by selecting File/Launch Applications.

When the program is running, if we go to the Object Browser tab we see our application listed there: all windows are shown and we can also change their properties. Selecting a window in the left pane, its properties are shown in the right pane. The properties marked with an arrow are read only, but the other ones can be altered. For example, if we select a TEdit window and change the wText property value, we see that the edit box text in the program is also changed.

The next step is creating a test script for the application. We could do it manually, by programming the script, but we will do it in the easy way, by recording the script from executed actions. We click the first button in the toolbar (the record button) and a small window, with the available tools for the recording mode appears (Figure 4).

Figure 4 - Toolbar in recording mode

We will record our script, creating the tests in the application: we fill the editboxes with 30, 0, 30 and 0, to obtain a circle. We click the button to draw the curve and, in the recording toolbar, we click the third button (the camera), dragging the mouse cursor to the panel where the curve is drawn. When we release the mouse button, a new window opens, where we save the captured image to the file circulo.bmp. When saving the image, a dialog bBox appears, asking the code to be added to the script. We input this code:

if not Regions.Compare(w.Window('TPanel'), 'Circulo') then
  Log.Error('Regiões não são idênticas');

We change the last editbox text to 90 and click the button to get a line, capturing the image to the file linha1.bmp, adding a code similar to the last one. Finally, we change the text in the two editboxes at the right to 90 and 0, getting a mirrored line, which we save to the file linha2.bmp.

Then we click the stop button in the recording toolbar, closing it. TestComplete shows the generated script:

procedure Test1;
var
  p, w: OleVariant;
begin
  p := Sys.Process('Project1');
  w := p.Window('TForm1', 'Curvas de Lissajoux');
  w.Activate;
  Sys.Keys('30[Tab]0[Tab]30[Tab]0');
  w.Window('TButton', 'Gera').Click;
  if not Regions.Compare(w.Window('TPanel'), 'Circulo') then
    Log.Error('Regiões não são idênticas');
  w.Window('TEdit', '', 1).Click(14, 10);
  w.Window('TEdit', '', 1).Drag(14, 10, -28, 0);
  Sys.Keys('90');
  w.Window('TButton', 'Gera').Click;
  if not Regions.Compare(w.Window('TPanel'), 'Linha1') then
    Log.Error('Regiões não são idênticas');
  w.Window('TEdit', '', 3).Click(10, 13);
  w.Window('TEdit', '', 3).Drag(22, 8, -60, -6);
  Sys.Keys('90');
  w.Window('TEdit', '', 1).Click(25, 7);
  w.Window('TEdit', '', 1).Drag(25, 7, -27, 0);
  Sys.Keys('0');
  w.Window('TButton', 'Gera').Click;
  if not Regions.Compare(w.Window('TPanel'), 'Linha2') then
    Log.Error('Regiões não são idênticas');
end;

With this script, we can test our application. The Log class allows adding messages or images to test log of the application. Log.Error sends an error message, indicating that a test failed. We can also send other kinds of messages, such as information or warning messages, or even files, with the methods Message, Warning or File.

Before testing the application, we should tell TestComplete which is the main function of our script. It is set by default to the function Main, created like this:

procedure Main;
begin
  try
//    Test;
  except
    Log.Error('Exception', ExceptionMessage);
  end;
end;

We should make three changes in the code:

• Uncomment the line that calls Test.
• Change the function call to Test1 (the function we've created call Test1, not Test).
• Uncomment the line that declares the Test function (it's above the beginning of Main procedure), changing its name to Test1.

With these changes, we can execute the script. Save it as Unit1.sd, in the project's directory and click the green arrow button. Our program must be running when the test runs. After the test is finished, TestComplete shows the Test log tab and, if nothing special happens, the test should fail. Why should it fail? Our recorded script depends on mouse movement and keyboard tabs, admitting the active control is the first editbox, which is not true if we execute our test twice (the active control remains on the last editbox after the test is run). We must make the first editbox as active before starting the test. To do this, we insert the line:

w.Window('TEdit', '', 4).Click;

before the Sys.Keys line. That selects the first editbox. When we execute the test again, we see that, if our test passes in the first time, it doesn't pass on the second time. Although the editbox is selected, it is not empty and the new text is inserted after the old text. We must clean it assigning an empty string to its wText property, by inserting this line after the line we've just inserted:

  w.Window('TEdit', '', 4).wText := '';

Now, if we execute the test again it passes, no matter how many times we run it. We can make some changes to our test, so it doesn't depend on mouse clicks and keyboard tabs anymore: we can set the values to the editboxes directly. Our final script is like this:

procedure Test1;
var
  p, w: OleVariant;
begin
  Sys.LaunchApplications;
  p := Sys.Process('Project1');
  w := p.Window('TForm1', 'Curvas de Lissajoux');
  w.Activate;
  w.Window('TEdit', '', 4).Click;
  w.Window('TEdit', '', 4).wText := '30';
  w.Window('TEdit', '', 3).wText := '0';
  w.Window('TEdit', '', 2).wText := '30';
  w.Window('TEdit', '', 1).wText := '0';
  w.Window('TButton', 'Gera').Click;
  if not Regions.Compare(w.Window('TPanel'), 'Circulo') then
    Log.Error('Regiões não são idênticas');
  w.Window('TEdit', '', 1).wText := '90';
  w.Window('TButton', 'Gera').Click;
  if not Regions.Compare(w.Window('TPanel'), 'Linha1') then
    Log.Error('Regiões não são idênticas');
  w.Window('TEdit', '', 3).wText := '90';
  w.Window('TEdit', '', 1).wText := '0';
  w.Window('TButton', 'Gera').Click;
  if not Regions.Compare(w.Window('TPanel'), 'Linha2') then
    Log.Error('Regiões não são idênticas');
end;

The first line launches our application and there is no need to execute it manually. Now we have a reliable test, which can be executed when we change the program code. The Stores tab shows the three images we've saved as defaults.

What we have here a black box test, where we don't know anything about the application, only the data retrieved by Windows OS. This kind of test can be done in any application where we don't have the source code and we want to test its functionality (we want to see if all menu options behave well after some change and if the answers are coherent with the input data, for example).

We could transform our application into an open application, which exposes more properties to TestComplete. To do this, we must add the unit TCClient, which is in the OpenApps TestComplete subdirectory to the tested project and recompile it. When we run the program and go to the Object browser tab in TestComplete, we see two changes: a scale besides the project name, indicating it's an open app, and the Form's published properties in the properties list.

When we have an open app, we can use its published components and properties in a TestComplete script. We can create a new test, similar to the first one, using the components and their properties. In the script editor, we create a new function, Test2, with this code:

procedure Test2;
var
  p, w: OleVariant;
begin
  p := Sys.Process('Project1');
  w := p.Form1;
  w.Activate;
  w.Edit1.Text := '30';
  w.Edit2.Text := '0';
  w.Edit3.Text := '30';
  w.Edit4.Text := '0';
  w.Button1.Click;
  if not Regions.Compare(w.Panel1, 'Circulo') then
    Log.Error('Regiões não são idênticas');
  w.Edit4.Text := '90';
  w.Button1.Click;
  if not Regions.Compare(w.Panel1, 'Linha1') then
    Log.Error('Regiões não são idênticas');
  w.Edit2.Text := '90';
  w.Edit4.Text := '0';
  w.Button1.Click;
  if not Regions.Compare(w.Panel1, 'Linha2') then
    Log.Error('Regiões não são idênticas');
end;

It is equivalent to the last test, but we are using the component's properties, as if the test was part of the tested program. AutomatedQA calls this test as a "white box test", where we can access the components and properties as we were coding tests in our own application. We can go a little deeper in our tests. To do this we must compile our project with some debugging options.

In Project/Options, in the Compiler tab, we must check Debug Information and Local Symbols, unchecking Optimization. On the Linker tab we check the Include TD32 Debug Info option and recompile our application.

When we select the application in the Object Browser tab of TestComplete, we see that there is more information in the Fields and Methods tabs: the private methods and variables are also visible, allowing to see how they behave during program execution. For example, the variables FM and FN are 0 when we start the program. We fill the editboxes, click the button and we see that the values change (if this doesn't happen, just right click in the variable list and select Refresh). The script can also access the DesenhaCurvas method. For our next test, we will make a change in the program's source code, creating Access properties do the FM, FN, FAlfa and FBeta variables, so we can change them in our script:

  public
    { Public declarations }
    property M : Double read FM write FM;
    property N : Double read FN write FN;
    property Alfa : Double read FAlfa write FAlfa;
    property Beta : Double read FBeta write FBeta;

When we compile and execute the program, we see the new properties in the Properties tab. We can now create a new test function in our script, like this:

procedure Test4;
var
  p, w: OleVariant;
begin
  p := Sys.Process('Project1');
  w := p.Form1;
  w.Activate;
  w.M := 30;
  w.N := 30;
  w.Alfa := 0;
  w.Beta := 0;
  w.DesenhaCurvas;
  if not Regions.Compare(w.Panel1, 'Circulo') then
    Log.Error('Regiões não são idênticas');
  w.Beta := 90*pi/180;
  w.DesenhaCurvas;
  if not Regions.Compare(w.Panel1, 'Linha1') then
    Log.Error('Regiões não são idênticas');
  w.Alfa := 90*pi/180;
  w.Beta := 0;
  w.DesenhaCurvas;
  if not Regions.Compare(w.Panel1, 'Linha2') then
    Log.Error('Regiões não são idênticas');
end;

As we can see, this code accesses the internal variables of the program, testing the function DesenhaCurvas. That is unit testing for our program. We've created an automated test script, independent of our program, which tests its internal functions.

We haven't made any significant changes to our tested app until now. If we wish, we can create a test function in the main program. It can interact with TestComplete, sending messages to its test log. We can do this when we want to access the program's internal data, not accessible by external scripts.

To include the test code in the program, we must add the unit TCConnect, which is in the Connected Apps\Delphi TestComplete's subdirectory in the Uses clause of the unit where you want to create test functions. That gives access to all classes defined by TestComplete.

One note must be made, here: if the test routine needs data entry, with the mouse or the keyboard, it must be in a separated thread, to avoid deadlocks and consequent application hangs.

As we won't be inputting data in our test, there is no need to create a new thread. We can copy the script test in TestComplete to the end of the main program's unit and add a new button in the Form, to call our test. The OnClick event handler code for this button is:

procedure TForm1.Button2Click(Sender: TObject);
begin
  Test4;
end;

We must add the unit TCConnnect to the Uses clause and the directory where it's located in the search directories in Project/Options. We recompile the project, noting that there's no need to make any changes in the test routine: it could be copied with no changes to our Delphi program. Once the program is compile, we return to TestComplete and change the Main function in the script to call our application:

procedure Main;
var
  p : OleVariant;
begin
  try
    p := TestedApps.Items[0].Run;
    while p.Exists do 
      Sys.Delay(500);
  except
    Log.Error('Exception', ExceptionMessage);
  end;
end;

When we execute the script, the application is opened. We click in the test button we've just created and the test is executed. When the test is finished, nothing happens until we close the tested application. When it's closed, the test log is shown. If we want to show the log when the test is finished, we should add the line:

Runner.StopTests;

at the end of the test routine, in our program. This ends the test at the end of the routine and shows the test log, with no need to stop the application.

After creating the tests for our application, we will port it to .NET and create tests for it. On Delphi 2005, we will create a WinForms .NET application, adding 4 textboxes, 4 labels and 1 panel, setting its property Dock to bottom. In the Click event, we add this code:

  FM := Convert.ToInt32(TextBox1.Text);
  FAlfa := Convert.ToInt32(TextBox2.Text)*pi/180;
  FN := Convert.ToInt32(TextBox3.Text);
  FBeta := Convert.ToInt32(TextBox4.Text)*pi/180;
  DesenhaCurvas;

The DesenhaCurvas function looks like this:

procedure TWinForm.DesenhaCurvas;
var
  i : Integer;
  Ang : Double;
  PosX, PosY, XAnt, YAnt : Integer;
  PaintCanvas : Graphics;
begin
  PaintCanvas := Graphics.FromHwnd(Panel1.Handle);
  PaintCanvas.FillRectangle(Brushes.White,Panel1.ClientRectangle);
  Ang := 0;
  XAnt := 120*sin(FAlfa)+180;
  YAnt := 120*cos(FBeta)+150;
  for i := 0 to 720*100 do begin
    Ang := Ang + pi/720;
    PosX := 120*sin(FM*Ang+FAlfa)+180;
    PosY := 120*cos(FN*Ang+FBeta)+150;
    PaintCanvas.DrawLine(Pens.Black,XAnt,YAnt,PosX,PosY);
    XAnt := PosX;
    YAnt := PosY;
  end;
end;

It is very similar to the Win32 function, the only change is that we aren't using the PaintBox anymore. The drawing surface is obtained using Graphics.FromHwnd, with the Panel's window handle. When we execute the application we see in TestComplete's Object Browser, the scale symbol showing it's an open app: .NET applications are open apps by default. We can inspect its variables and methods the same way we did in the Win32 application. In this case, the tests are very similar to the previous ones, the only difference is in the test images. We must capture new images to compare to what is generated in the tests.

To do this, we start our application, fill the editboxes text and click the button to draw the curve, then we click the recording button in TestComplete. When the toolbar is shown, we click the camera button to capture the control image, drag it to the application panel and save the captured image as circulo1.bmp. We do the same with the other two images and stop recording.

The recorded test isn't important, because we will copy the last test we've created for the Win32 application, making the changes needed. The test for the .NET application is:

procedure Test5;
var
  p, w: OleVariant;
begin
  p := Sys.Process('LissajouxNet');
  w := p.TWinForm;
  w.Activate;
  w.field_FM := 30;
  w.field_FN := 30;
  w.field_FAlfa := 0;
  w.field_FBeta := 0;
  w.DesenhaCurvas;
  if not Regions.Compare(w.Panel1, 'Circulo1') then
    Log.Error('Regiões não são idênticas');
  w.field_FBeta := 90*pi/180;
  w.DesenhaCurvas;
  if not Regions.Compare(w.Panel1, 'Linha11') then
    Log.Error('Regiões não são idênticas'+IntToStr(i));
  w.field_FAlfa := 90*pi/180;
  w.field_FBeta := 0;
  w.DesenhaCurvas;
  if not Regions.Compare(w.Panel1, 'Linha21') then
    Log.Error('Regiões não são idênticas');
end;

As we can see, there are not many changes in the code. There is no need to create access properties for the Form's private variables, they can be accessed adding field_ before their name. We can add the new application to the tested applications, using File/Tested Applications, byadding the new test to the test suite going to the script editor and declaring the new function in the top of the script and adding the call to the Main function::

procedure Test5; forward;

procedure Main;
begin
  try
    Sys.LaunchApplications;
    Test1;
    Test2;
    Test3;
    Test4;
    Test5;
  except
    Log.Error('Exception', ExceptionMessage);
  end;
end; 

As we can see, we can use our tests when we are porting a program to .NET, making only some small changes.

Download Example Source Files

A 30 day evaluation version of TestComplete is available here.