Introduction to Data-Driven Testing in TestComplete

Author: AutomatedQA Corp.
Last updated: April 14, 2009
Applied to: TestComplete 7

Introduction

In addition to TestComplete’s support for automation of unit, regression, distributed and other automated testing types, it offers support for traditional record-and-playback-style GUI and functional testing. While certainly not a comprehensive method of automated testing on its own, recordable automated tests represent an excellent starting place for creating more robust and complete test projects. One of the complaints leveled against traditional playback-type automated testing is that it is inflexible and that the automated test cases are not easily extended. To answer this argument, the Data-Driven Testing (DDT) methodology was born. Although this technique does not originate with TestComplete, it is a fairly straightforward concept and is easily implemented within TestComplete’s automated tests.

The simplest explanation of Data-Driven Testing is this: data that is external to your functional tests is loaded and used to extend your automated test cases. One of the best examples is that of a customer order form. To use Data-Driven Testing in this scenario, you might record a single automated test, entering values into the various fields. After you have recorded the automated test, it only contains values that you entered during the recording. Most likely, data you have specified does not cause errors in the application, but other data may cause them. To be sure that the application works as expected you can alter the test to accept variables, entering those variables into the data fields. Thus, data-driven testing allows you to run this test each time you want to add an order record, passing in a new set of data each time.

It is convenient to keep data for automated tests in special storages that support sequential access to a set of data, for example, Excel sheets, database tables, arrays, and so on. Often data is stored in a text file and are separated by commas. This lets you modify them easily. If you need to add more data, you simply modify the file in any text editor (in case of hard-coded values, you should modify both data and code). Each string in the file holds values to be assigned or checked. Normally, each string holds values for one automated test. However, you are free to use several strings for one automated test. In this case, you may need to add a field that will specify which values are input, and which are verifying values.

Data-driven testing let you create automated test projects that can be infinitely extended by simply adding new lines of text to a text file or a spreadsheet. For example, data is read from an external source and fed line-by-line into the functional test until there is no more external data. In this way, new automated test cases that are added to the external data simply extend the loop for each new line of data.

When a storage holds both input and verifying data, a data-driven test includes the following operations performed in a loop:

• Retrieving input data from storage
• Entering data in an application form
• Verifying the results
• Continuing with the next set of input data

Later, we will illustrate how to perform these steps with an example.

TestComplete's Support for Data-Driven Testing

TestComplete includes special built-in routines and program objects that provide extended support for data-driven testing. They are DDT, GetListLength, GetListItem, CallMethod, GetPropertyValue and SetPropertyValue. The DDT object provides access to drivers that let you easily extract data from Excel spreadsheets, database tables and CSV files. The GetListLength and GetListItem methods are used to process a list of strings. The other three methods, CallMethod, GetPropertyValue and SetPropertyValue, let you modify the methods sequence and properties to be called by a automated test without having to modify the automated test.

To simplify access to data stored in Excel sheets, ADO databases or CSV files, TestComplete includes special driver objects that hide the actual data storage format and let you access data using a unified table-like interface. As we have said, the DDT object is used to work with the drivers. Its methods let you obtain driver objects for Excel sheets, ADO tables and CSV files: ADODriver, CSVDriver and ExcelDriver.

Once you obtain a driver object, you can use its methods and properties to iterate through records of the driver table, retrieve them and use in your automated tests.

The Data-Driven Testing Sample Project

Let’s see how to perform data-driven testing with TestComplete. The following sample will illustrate how to add new records from the external source to the sample application using special program objects. For this purpose, we will use the Orders sample application that is shipped with TestComplete. The application displays a list of orders and lets users create, modify and delete orders. You can find the application in the following folder: <TestComplete 7 Samples>\Open Apps\OrdersDemo\C#\bin\Debug\Orders.exe.

We will use the Excel sheet as data storage. You can find it in the following folder: <TestComplete 7 Samples>\Scripts\DDT\TestBook.xlsx. Each string of this table contains the values for one order, whose input will be simulated. Note that the data contains not only input values, but also the verifying values. You can see them in the last three columns of the table. One of these values will be used as a baseline copy for comparison with a calculated value.

The purpose of our data-drive test is to read data from the data storage, simulate data input into the form’s edit boxes and check whether the application calculates values for the Total edit box correctly.

Creating a New Project

At first, we will create a new TestComplete project. To do this:

• If TestComplete is not already running, launch it now. Once it is running, select New | New Project from the TestComplete File menu. This will call the Create New Project dialog. This dialog may function in two modes: Simple and Classic. We will use the Simple mode, which is enabled by default. In the dialog, specify the project name, location and VBScript scripting language. You can also switch to the Classic mode using the Classic button. It let you create a new project with the opportunity to specify additional project settings.
• Click Create to close the dialog.
• Right-click the TestedApps node in the Project Explorer panel.
• Choose Add | New Item from the context menu. The standard Open File dialog will appear.
• Select the Orders application executable in the Add Tested Application dialog and click Open. You can find the executable in the following folder: <TestComplete 7 Samples>\Open Apps\OrdersDemo\C#\bin\Debug\Orders.exe.

Recording an Automated Test

In the following steps, we will record a keyword test (keyword-driven test). The test should perform actions like opening the tested application, opening the Order form, populating the form, creating checkpoints, closing the form and closing the application.

To record the test:

1. Select Record Keyword Test from TestComplete’s Test Engine toolbar, or choose Test | Record | Record Keyword Test from TestComplete’s main menu, or select Record New Test from the toolbar of the KeywordTest Editor.
2. Add the "Executing the Orders application" comment to a test. To do this:
   • Click Add Comment to Test on the Recording toolbar. This will invoke the Add Comment to Test dialog:

     

   • Enter "Executing the Orders application" and click Add.
3. Click Run Tested Application on the Recording toolbar.
4. Add the "Opening the Order form" comment to a test as described above.
5. Choose Orders | New order from the main menu of the Orders application.
6. Add the "Populating the Order form" comment to a test.
7. Enter the following values into the edit boxes of the Order form:
   

   Field	Value
   Product	FamilyAlbum (select it from the drop-down list)
   Quantity	20
   Date	05/06/2005
   Customer Name	John Smith Jr
   Street	12, Orange Blvd
   City	Grovetown, CA
   State	US
   Zip	111155
   Card	Select MasterCard.
   Card No	555777555888
   Expiration Date	05/06/2005

8. Add the "Creating the checkpoint" comment to a test.
9. Perform the following actions to create a checkpoint for the Total value:
   • Select Create Property Checkpoint from the Recording toolbar.
   This calls the Create Property Checkpoint wizard.
   • On the Select Object page of the wizard, specify the Total edit box of the Order form. After specifying this edit box, TestComplete displays its name in the Object field of the page.
   • Click Next to proceed with the checkpoint creation.
   • On the Select Property page, choose the wText property from the list of the object’s properties and click Next.
   • On the Comparison Parameters page, make sure that the Equal condition is selected.

     

   • Click Finish to exit the wizard and add the property checkpoint to a test.
10. Add the "Closing the Order form" comment to a test.
11. Click OK on the Order window to close the Order form and add new record to the
Orders application.
12. Add the "Closing the Orders application" comment to a test.
13. Close the Orders application.
14. Click No on the Confirmation window to close the application without changes.
15. Click Stop on the Recording toolbar.

The recorded test looks like this:

Modifying the Data-Driven Test and Assigning Input Values

Now we will modify the recorded data-driven test to add a new order and check the order information generated by the application using the data storage. The modified test will run until a DDT driver is at the end of the data storage.

To access the data storage in the test we will use the Driver variable. You can create this variable on the Variables edit page of the KeywordTest Editor. To display the page, select Variables at the bottom of the KeywordTest Editor. To add the variable to the test, right-click somewhere within the Variables page and choose New Item from the context menu. Then you should specify the variable’s name (Driver) and type (Object).

To assign values to the Driver variable, we will add the Set Variable Value operation to the test after the Run TestedApp operation. To do this:

• Drag the Set Variable Value operation from the Statements category of the Operations panel to the test area. After the operation is dragged, the Set Variable Value wizard will appear.
• On the Select Variable page of the wizard, choose the Driver variable which value will be set with the operation and click Next .
• On the Set New Value page, specify the following parameters:
  • In the Mode combo box, select Code Expression.
  • In the Value box, specify: DDT.ExcelDriver("../TestBook.xlsx", "TestSheet", true) .

  Note that the specified expression must be in the project’s script language.

To provide multiple test iterations, we need to create a loop. To determine the end of the driver’s table, use the EOF property: right after loop’s creation the driver is on the first row of the table:

• Add the While Loop operation to the test. After the operation is dragged, the While Loop dialog will appear.
• In the Value 1 column, click the ellipsis button. The Edit Value dialog will appear.
• In the dialog, specify the following parameters:
  • In the Mode combo box, select Code Expression.
  • In the Value box, specify: KeywordTests.KeywordTest1.Variables.Driver.EOF().
• Click OK to close the Edit Value dialog.
• In the Condition column, select equal to from the drop-down list.
• In the Value 2 column, click the ellipsis button. The Edit Value dialog will appear.
• In the dialog, specify the following parameters:
  • In the Mode combo box, select Constant.
  • In the Type combo box, select Boolean.
  • In the Value combo box, select False.
• Click OK to close the Edit Value dialog.
• Click OK to save the changes and to close the While Loop dialog.

Now we should move the operations that open the Order form, populate the form, create checkpoints, and close the form to the loop body. These operations are situated under the “Opening the Order form”, “Populating the Order form”, “Creating the checkpoint” and “Closing the Order form” comments in the recorded test. The following image shows the operations that should be moved to the loop body:

Note that included among the operations that populate the Order form are the operations that call the Drag and Click methods. We don't need to use these operations to successfully run the test and you can just delete them from the test.

To populate the Order form with values retrieved from the data storage you should modify the operations situated under the "“Populate the order form” comment in the test. We will change the values specified, while recording the data-driven test, to calls of the Driver.Value property. This property returns a column value by column index or column name. To do this:

• Select the desired operation in the test area.
• Select theValue cell and then either click within the cell or press F2. This will activate the in-place editor.
• Click the ellipsis button of the editor. The Operation Parameters dialog will appear.
• In the dialog, click the ellipsis button in the Value column. The Edit Parameter dialog will appear.
• In the Edit Parameter dialog, specify the following parameters:
  • In the Mode combo box, select Code Expression.
  • In the Value box, specify the Driver.Value property with the appropriate index provided below.
• Repeat these steps for all of the operations that populate the Order form.

You can find the values and the indexes in the following table:

In addition, we should replace the On-Screen Action operation that checks the MasterCard button with the Run Code Snippet operation. To do this:

• Delete the On-Screen Action operation that checks the MasterCard button from the test.
• Drag the Run Code Snippet operation from the Test Actions category of the Operations panel to the test. The operation should be inserted to the position where the deleted operation was placed.
• After the operation is dragged, the Set Variable Value wizard will appear.
• In the dialog, specify the following code line:

  Aliases.Orders.OrderForm.Group.WinFormsObject(KeywordTests.KeywordTest1.Variables.Driver.Value(8)).ClickButton()

  Thus, we specify the name of the option button by the Driver.Value(8) property.

To compare the output value of the Total field with values retrieved from the data storage we should also modify the If…Then operation. We will change the condition specified, while recording the test, to the call of the Driver.Value(13) property. To do this:

• Click the ellipsis button in the Value column of the If…Then operation in the KeywordTest Editor. The Edit Condition dialog will appear.
• In the dialog, click the ellipsis button in the Value2 column. The Edit Value dialog will appear.
• In the Edit Value dialog, specify the following parameters:
  • In the Mode combo box, select Code Expression.
  • In the Value box, specify: KeywordTests.KeywordTest1.Variables.Driver.Value(13)

To proceed to the next row of the driver’s table, add the Run Code Snippet operation that calls the DDTDriver.Next() method before the end of the loop.

Thus the data-driven test will look like this:

You can find the complete sample project here: DDT Sample Project.

Executing a Data-Driven Test and Checking Results

Now we will run the data-driven test. As you can see, TestComplete opens the tested application and adds new records until there is no more data in the TestBook sheet. After the data-driven test is over, you can find the test results in the Test Log.

As you can see, the data-driven test run was successful.

If the output value had not matched the appropriate verifying value, the "The property value does not equal the template value" error messages would have been posted into the log. This indicates that the application works incorrectly. In this case, the values should be revised. You can include any information you need in error messages. For instance, you can specify the values of all input parameters that were used when the error occurred.

Conclusion

This paper should serve as an introduction to data-driven testing and TestComplete’s support for it. We hope this information will help you create your data-driven tests and make them more flexible and powerful. If you are interested in trying TestComplete for free, download it now and try it today.