Synchronizing Distributed Tests

Author: AutomatedQA
Last Updated: July 23, 2009
Applies to: TestComplete 7

Introduction

Distributed tests are tests that consist of several parts performed on different computers. Test parts interact with each other and strive for a single common result. TestComplete supports distributed testing and provides you with features that allows you to create and manage distributed tests. For more information about this, see the Distributed Testing Tutorial article.

To ensure that all of the distributed test parts perform in a certain order, you have to synchronize them. Synchronization is one of the key features when distributed testing. TestComplete provides several features for synchronizing distributed tests:

• Network Suite Variables
• Events
• The WaitForNetVarChange and WaitForState Methods
• Critical Sections
• Synchronization Points

This article provides detailed information about these features and demonstrates how you can use them in your distributed tests.

Using Network Suite Variables in Distributed Tests

To create distributed tests, you use the NetworkSuite project item. This item also lets you define a list of specific variables that can be used to exchange data between projects that participate in distributed tests and to synchronize these projects. These variables are called network suite variables (or NetworkSuite variables).

Network suite variables store data between test runs and are common for master and slave projects engaged in the distributed test and if one project changes a NetworkSuite variable, the others will use the new value.

Network suite variables are managed by a master project. That is, to create or modify them, you must load the master project in TestComplete.

To create and modify network suite variables, use the Variables page of the NetworkSuite editor. To view this page, do the following:

• Select the NetworkSuite item in the Project Explorer panel and choose Edit from the context menu, or simply double-click this project item.
• In the resulting NetworkSuite editor of the Workspace panel, switch to the Variables page.

  

By using this page, you can specify the variable’s name, type, description, default value and local value. The default value is used when you open the master project on a computer for the first time and is common for all computers that use the project. The local value depends on the computer where the project has been opened.

You can use network suite variables for synchronization purposes. TestComplete includes specific OnNetVarChange event that occurs when the value of a network suite variable changes. Thus, using this event in your distributed testing scripts or keyword tests, you can perform certain actions when changing the network suite variables. For information about how to create an event handler for the event, see below.

TestComplete also provides the WaitForNetVarChange method that lets you pause the distributed test execution until the network suite variable gets the desired value. The method returns True if the expected value is set to this variable and False if it is not. By using this method, you can stop the test execution if the variable does not get the desired value during a specified time period.

To change suite variables during a test run, it is recommended to use critical sections. Otherwise, there may be various problems or invalid results. For more information about this, see the Critical Sections section.

Network Suites Events and Methods in Distributed Tests

When a distributed test is being executed, it passes through several testing stages. Some of them are executed consecutively, while others occur only when particular conditions are met. In TestComplete, the states of the network suite indicate the stages of the distributed test. During the test run, you may need to know which state the network suite is in or perform actions when the state is changed. To do this, TestComplete includes special events that check the state of a particular job, task and host: OnNetJobStateChange, OnNetTaskStateChange and OnNetHostStateChange events.

These events and the OnNetVarChange event are the network suite events. TestComplete lets you create script routines that handle these events. This allows you to perform specific actions when the events occur.

In order to process network suite events, the TestComplete project must include an Event control that contains the events. Creation of this control requires that your project contains the Events project item. By default, it is added. If for some reason you deleted it, you should include this item in your project using the Create Project Item dialog. To do this --

• Right-click the project node in the Project Explorer panel and choose Add | New Item from the context menu. This will call the Create Project Item dialog.
• In the dialog, select the Events project item, specify its name and location in the appropriate edit boxes and press OK to save the changes.

The Events project item will appear under the project node in the Project Explorer.

By default, after its creation the Events project item includes one Event control, GeneralEvents. This control contains a number of TestComplete events that you can use to create event handlers and perform certain actions. To see which events are included in this control, double-click the GeneralEvents node in the Project Explorer. TestComplete will show the Event Control editor in the Workspace panel.

The Event Control editor contains two lists: Available Events and Events to Handle. The Available Events displays the events of the object that is specified in the Source object box. These events can be added to the Event control. The Events to Handle list displays the events that were added to the control:

The network suite events are not added to the Events to Handle list by default. To add these events to the list, right-click Network Suite events on the Available Events list and select Add selected from the context menu.

Then you can create an event handler routine:

• Select the desired network suite event in the Events to Handle list. Activate the Event Handler cell and click the New button that is displayed within this cell. TestComplete will show the New Routine dialog.
• In the dialog, specify the name of your event handling routine (for example, GeneralEvents_OnNetVarChange) and the unit that will hold this routine (for example, Unit1), then click OK.

TestComplete will create an event handler and display it in the Code Editor. Now you can write code for the event handling routine. You can add any operations, like an operation that will post a message about the changed state of a variable, job, task or host into the test log. The added operations will be performed when the specified event occurs.

You can also check the state of jobs, tasks, hosts and even the entire network suite, using the Job.State, Task.State, Host.State or NetworkSuite.State properties. A little more sophisticated method is to suspend the test execution until the object gets the expected state: use the Job.WaitForState, Task.WaitForState or NetworkSuite.WaitForState methods to do this. These methods let you specify whether the job, task or network suite get the desired state during the specified time period.

Critical Sections in Distributed Tests

In distributed tests, it is quite possible that several projects use the same resources simultaneously. Network suite variables are shared for all projects engaged in distributed testing and any project can change the value of any network suite variable. Suppose, you have two projects that change the same network suite variable simultaneously, or one right after the other, but you cannot say for sure what value the variable will have in the end. This may cause a loss of your test results or other unexpected and undesired behavior. To solve the problem, use critical sections. A critical section is a specific marked section of code which can be performed by one project at a time. To avoid losing test results, we simply put the code that changes the value of the variable within a critical section. While one project works with the critical section, the resources it works with (in our case, the network suite variable) are accessible to it alone. Other projects that are going to use the same resources will have to wait until the first project leaves the critical section. Thus, critical sections let you synchronize projects that participate in distributed testing. Critical sections can include not only code that modify network suite variables but also code that writes to any critical resource available from distributed test scripts and keyword tests (files, objects, programs, and so on).

TestComplete provides special methods for the NetworkSuite program objects to enter and leave a critical section. To enter the critical section, use the NetworkSuite.EnterCriticalSection method, where you need to specify the unique name of the critical section you want to enter. When a project’s test calls this method, it will access the specified critical section. The state of this project changes to Running in CS. Other projects that are requesting this critical section will wait for the first project to leave it (have the Waiting for CS state) and then will enter the section in the order that they came in. Using the Timeout parameter of the NetworkSuite.EnterCriticalSection method, you can also specify how long a project will wait for access to the desired critical section.

Synchronization Points in Distributed Tests

To synchronize the execution of different projects participating in a distributed test, you can also use synchronization points (synchpoints) . Synchronization points let you run the distributed test parts on several remote computers at the same time. In this case, execution of the participating projects is paused until all of these projects have reached the same synchpoint. After that, the projects simultaneously proceed. This allows you to make sure that different projects start doing specific actions at the same time.

In order to use synchronization points in a distributed test, you should add them to the SynchPoints collection in the NetworkSuite project item of each project you want to synchronize. To do this:

• Right-click the NetworkSuite | SynchPoints item in the Project Explorer panel and select Add | New Item from the context menu.
• In the resulting dialog, enter the desired synchpoint name (it must be unique within the project) and press OK.

To use synchpoints in the distributed test, add the NetworkSuite.Synchronize method before the code portion that needs synchronized. You should add the method to each project which needs to synchronize. Using the Name parameter of the method, specify the name of the desired synchpoint.

When the project reaches the NetworkSuite.Synchronize method, the distributed test will pause and wait until the other distributed test projects that use the same synchpoint call the Synchronize method. At this point, the state of these projects change to Synchronizing. Using the Timeout parameter of the method you can specify a time limit so that the given project waits for synchronization. As soon as the last project calls the NetworkSuite.Synchronize method, the execution of all distributed test projects will simultaneously continue to run, and their state changes back to Running.

Synchronizing Distributed Tests - Example

Want to see how to synchronize projects in your distributed test? There is a sample distributed test that illustrates how you can synchronize scripts running on several network computers and also shows how the scripts can exchange data. You can find the distributed test in the following folder: <TestComplete 7 Samples>\Scripts\Distributed Testing.

This folder includes two projects: Master.pjs and Slave.pjs. The Master project is used to manage the distributed test. The Slave project will run on two remote computers as part of your distributed test. Each of the projects creates text files in one shared folder using Windows Notepad and the deletes the files. Also, these computers write text to the same file, one wating for the other to finish using synchronization.

The Master and Slave projects share the Common unit that holds several common routines. These routines will be used to delete, create and change text files in the shared folder. These routines are called from the Master and Slave units.

The network suite is run by the Master unit. When the Main routine of the Master unit calls the UsingOneJob routine, the distributed test runs using one job in the network suite and synchronizes the tasks. Then the Main routine calls the UsingMultipleJob , and the test runs using several jobs and synchronizes the tasks automatically when starting the jobs.

When we run the distributed test using one job (the Main routine of the Master unit calls the UsingOneJob routine), we should synchronize the tasks manually. To do this, we use the synchronization points: BeforeFileCreation, FileCreationCompleted and Finish. The BeforeFileCreation and FileCreationCompleted points are used to simultaneously start creating and changing text files on all computers. The Finish synchpoint is used to wait for the network suite run to finish; it will also be used, when running the distributed test when several jobs exist.

We add these synchpoints to the SynchPoints collection of the NetworkSuite project item both in the master and slave projects:

Then we add the NetworkSuite.Synchronize method for each synchpoint to defined places of the Master and Slave units. The NetworkSuite.Synchronize("BeforeFileCreation") method is added to the units before calling the CreateFiles routine:

Thus, when one project reaches the NetworkSuite.Synchronize(BeforeFileCreation)method , its execution will pause and wait until the other distributed test projects call this method. Once the last project calls the method, all projects will simultaneously start creating the files.

The NetworkSuite.Synchronize method for the FileCreationCompleted synchpoint is added to the Master and Slave units before calling the ChangeFile routines. So, all the projects that use the same synchpoint will start changing the file after the last project calls this method:

The Finish synchpoint will be used before generating test results and also is added to the Master and to the Slave units.

When we run the network suite using several jobs (the Main routine of the Master project calls the UsingMultipleJob routine), we need not to synchronize the projects before creating files. The Master and Slave computers start this operation simultaneously because their tasks belong to one job. To synchronize changing the text file with several computers, we use the Jobs.WaitForState method. We call this method from the UsingMultipleJob routine before calling the ChangeFile routine:

Using this method, we delay the distributed test’s script execution until the ChangeFile job is running. After that, all test projects will add a line to the text file, each waiting for their turn. To share access to this file, a critical section is used.

In order for the projects to change the text file in the correct order, we use the SharedFile critical section in the ChangeFile routine of the Common unit. The ChangeFile routine also includes the operation that changes the TotalRowCount network suite variables. This variable returns the number of lines added to the shared text file and can be accessed from several computers at the same time, and this may cause an unexpected value. So, changes in the values of the variable should also be performed within the critical section. Thus, the SharedFile critical section will contain a code snippet that changes the shared text file that resides on the master computer and increases the TotalRowCount variable:

Once a project enters this critical section and changes the text file and modifies the TotalRowCount variable, other projects participating in the distributed test cannot enter this section and must wait until that project exits this section.

The CreateFiles routine of the Common unit also includes a critical section. It is the ChangeVar critical section that contains the code for changing the TotalFilesCount variable:

Once a project enters this critical section and modifies the TotalFilesCount variable, other projects wait until that project exits the section. After that, they will use the new value of the variable.

To indicate the stages of the performed distributed test during the test run, we have created the OnNetSuiteStateChange event using the Event Control editor:

To handle this event, we add the OnNetSuiteStateChange event handler to the Master unit. This handler occurs each time the network suite state changes and posts a message about the new state to the test log:

In our distributed test, we also use the WaitForState method:

This method delays script execution until the network suite is running. If the network suite is not running for a specified time period, the script halts the network suite's execution.

Conclusion

TestComplete allows you to create distributed tests and provides several features for synchronizing the execution of different projects participating in the distributed test. By synchronizing your distributed tests, you manage how your tests react in certain situations when testing a client-server application. If you are interested in using distributed testing with TestComplete or would like to see what other features TestComplete has, download and try it today.