Author: AutomatedQA Corp.
Last updated: April 7, 2009
Applied to: TestComplete 7
Introduction
Microsoft Visual Studio Team System includes tools and technologies for all members of software development teams: project architects, developers, managers and, of course, software testers and Quality Assurance personnel. It helps team members communicate throughout the software development process.
Team Foundation Build, which is one of Microsoft Visual Studio Team System tools, enables the development team to create software builds regularly and manage this process. It is important that software testers can run their automated tests as part of team builds, so it is possible to test their applications automatically after the build is over.
TestComplete Enterprise edition includes Microsoft Visual Studio Team System and Microsoft Visual Studio Team Foundation Integration packages (both for Visual Studio 2005 and for Visual Studio 2008). These packages extend Visual Studio Team System test projects with the TestComplete automated test type. After you add a TestComplete automated test to a Visual Studio test project, you can create team builds that will run TestComplete tests after the application’s build is over, you can also publish automated test results, create work items for them in Team System’s issue-tracking database and generate reports for the items.
These packages also enable you to use Visual Studio Team System unit tests as part of your TestComplete projects.
In this article we will provide step-by-step instructions of how to use TestComplete automated tests as part of team builds and consider the following points needed for creating and configuring team builds, integrating TestComplete tests with them and automate the testing of your applications:
In our explanation, we assume that the test project is created and configured on one computer, but will be run on the computer that will perform the build. Furthermore, we will only provide explanations for Visual Studio 2008 Team System.
Creating and Configuring Automated Test Projects
Once you have a TestComplete project (or project suite), you can integrate it in Visual Studio test projects for the further use in team builds. Typically, TestComplete projects are created and modified at a workstation, while Visual Studio builds are run on the build server. If your build uses TestComplete projects that reside on another workstation, make sure the build server has access to these TestComplete projects. In order words, share the folder that contains the TestComplete projects for network access. Furthermore, make sure that the TestComplete projects can access all needed applications on the build server, TestComplete or TestExecute is installed on the build server and the server has all needed third-party libraries installed.
Once your TestComplete project is ready, you can integrate it into Visual Studio tests and builds. To create a new test project in Visual Studio 2008 Team System, do the following operations:
Select File | New | Project from Visual Studio’s main menu.
In the ensuing dialog select one of test project types and specify the project template (for instance, Visual C# Test Project).
Specify the project name, location and the solution name. Then, click OK.
So, we have just created a new test project. Visual Studio will open and display its contents in the Solution Explorer.
Since the Microsoft Visual Studio 2008 Team System Integration package is installed with TestComplete, you can add TestComplete tests to your Visual Studio test project. If you use TestComplete 7, this automated test type is called TestComplete 7 Test. To add the TestComplete test to your automated test project, do the following:
Right-click the test project in Visual Studio’s Solution Explorer and select Add | New Test from the context menu.
In the ensuing Add New Test dialog select the TestComplete 7 Test in the Templates box:
Specify the desired test name and the test project. Then press OK.
After adding the TestComplete automated test to your Visual Studio test project, you can modify its properties using a special test editor in Visual Studio. To view this editor, double-click the desired TestComplete test item in Solution Explorer or select Open from its context menu:
In this editor, you should click the ellipsis button and specify the TestComplete project or project suite to be executed during the build testing. The editor will display a tree of test items defined in the project suite or project you have specified. You can select the test items that you want executed during the build testing and clear check boxes for the items that you do not want to include in the automated testing process. Also, in the Execute With section you can specify what automated testing tool should be used for executing TestComplete tests – TestComplete or TestExecute.
Note that if your TestComplete project or project suite is not located on the server computer, where the build will be run, you should specify the path to the project in the TestComplete test editor (see the image above) through My Network Places, i.e. you should specify the project path that points to the shared project folder and include the client computer name.
You have now integrated a TestComplete project with a Visual Studio test project. Now you need to configure the Visual Studio test project so the specified TestComplete automated tests are executed by Visual Studio during the build testing process. Do the following:
In Visual Studio 2008 Team System, switch to the Test List Editor panel. To display it, select Test | Windows | Test List Editor from the main menu.
On the left of the Test List Editor panel, right-click the
List of Tests node and choose
New Test List from the context menu. This will call the Create New Test List dialog:
In the dialog, specify the test list name, description and click OK.
Select the All Loaded Tests node in the Test List Editor panel. A list of all available tests in the test project will be displayed on the right section of the panel.
Drag the desired TestComplete test from the right of Test List Editor to the test tree on the left and drop the test on the created test list.
Check the TestComplete test to enable it.
Save the changes made to the test project.
We have finished creating and configuring the test project. Before you create a build and integrate the automated test project with it, it is recommended that you run the project to make sure that it runs properly.
To use your automated test project in team build, you should also implement some more preparatory operations. Since Visual Studio builds operate with files taken from the source control, you need to add the created Visual Studio test project to Team System’s source control. To add the project to the source control system, do the following:
Set Visual Studio Team Foundation Server as the current source control plug-in for Visual Studio:
Select Tools | Options from Visual Studio’s main menu.
In the ensuing Options dialog, activate the Source Control | Plug-in Selection settings group and select the Visual Studio Team Foundation Server in the Current source control plug-in box.
Press OK to save changes and close the dialog.
Right-click your solution in the Solution Explorer and select Add Solution to Source Control from the context menu. This will call the following dialog:
In the dialog, you can specify the source control project and folder where the test will be added. Select the team project that the automated tests are intended for.
Click OK to add the test project to a source control. Visual Studio will copy the project files to Team System’s source control.
After Visual Studio adds the solution to the source control, right-click the solution node in the Solution Explorer and select Check In from the context menu. This will call the Check In dialog:
In the dialog press Check In. Visual Studio will check in the solution files to the source control system.
You may also associate your automated test with a work item in the Team System’s database. This database holds info about the development process: executed tests, reported bugs, assigned tasks and so on. For instance, you may associate an automated test with a bug stored in the database. The test execution will indicate whether the bug is resolved or not.
To associate an automated test with a work item, do the following:
Check out your test project from the source control.
In Visual Studio 2008 Team System, open the Test List Editor panel.
Right-click the desired TestComplete automated test in the Test List Editor panel and select Properties from the context menu.
Switch to the ensuing Properties panel and click the ellipsis button of the Associated Work Items property. This will call the Associated Work Items dialog that lists all work items associated with your automated test.
To associate a new work item with your automated test, click Add. Visual Studio will bring up the Work Items Picker dialog where you can select the work item for association.
You can search for the desired items by using a query, or searching by the work item’s title or identifier.
Select the desired item(s) in the Work Items Picker dialog and click OK. Identifiers of the selected items will be displayed in the Associated Work Items dialog.
Click OK to close the Associated Work Items dialog and associate the items with the test.
Save changes made to the test and check in the modified test project to Team System’s source control.
The creation and configuration of the automated test project is now finished. The next step is to create and configure a team build project.
Creating and Configuring Team Build Projects
Now that you have created and configured an automated test project for a team build. This is a good time to create a team build and integrate the automated test project into it. To create a build in Visual Studio Team System 2008, do the following:
Share a folder on the server’s hard drive. This folder will store build results.
Open the Team Explorer panel in Visual Studio. If it is hidden, select View | Team Explorer from Visual Studio’s main menu to display the panel.
In Team Explorer, right-click the <your_team_project> \Builds node and select New Build Definition from the context menu.
The
Build Definition dialog displays:
The dialog tabs that require your input are marked with a warning icon.
On the General tab of the dialog, specify the build name and description.
On the Workspace tab, specify the source control folder for the team project that you are creating the new build definitions for and a local folder on the build agent.
On the
Project File tab, you can browse to an existing TFSBuild.proj project file or launch the
MSBuild Project File Creation Wizard to create a new TFSBuild.proj file.
To create a new project file, click Create and the MSBuild Project File Creation Wizard displays. In the wizard, do the following:
On the Selection page, choose the solutions that belong to the team project that you would like to build with the created team build. After selecting the desired solutions, click Next to continue.
On the Configurations page of the wizard, specify the build’s configuration and platform. Click Next to go to the last page of the wizard.
On the Options page, enable the Run test check box. Then, in the Test metadata file box, specify the desired test project metadata file (.vsmdi file) that has been added to the team project. In the Test list to run box, check which automated tests from the specified test project you would like to execute after the build is over. Then click Finish to create a build project file. These are the main actions you have to perform for integrating automated tests in a team build.
In the Build Definition dialog, switch to the Build Defaults tab. On this tab you can choose an existing build agent from the Build agent drop-down list, or create a new agent via the New button. It calls the Build Agent Properties dialog where you enter the Display name, Description and Computer name boxes and then press OK to create a build agent and close the dialog.
Also, you need to fill in the Builds will be staged to the following share field with a UNC path to a folder where the built binaries and log files will be stored.
Finally, click the OK button in the Build Definition dialog to create your build definition. It will be displayed under the Builds node of your team project in the Team Explorer panel.
Now that you have created a new build definition, you can now start the team build. The test project will be executed at the end of the build run. Visual Studio will publish the test results automatically after the automated test has finished.
Running the Build and Publishing Automated Test Results
To start the build in Visual Studio Team System 2008, do the following:
Switch to the Team Explorer panel.
Right-click the desired build definition and select Queue New Build from the context menu. This will invoke the Queue Build dialog:
In the dialog, specify the build definition, build agent and the folder, in which the files and modules will reside after the build is over.
Press Queue. Visual Studio will add the build request to the build queue and will display the Build Explorer that informs you about the build progress:
After the build is finished, Visual Studio will automatically run the specified automated tests and then publish the automated test results into the team project’s issue-tracking database.
Note that Visual Studio does not add automated test results that were generated during the build run, to the Test Results panel. To view these results:
Expand the Result details node in the build report and click the link under Test Run. Visual Studio will display the Browse for Folder dialog.
In the dialog, specify the folder that contains the files that include the resulting data and click OK. The result files will be placed into the specified folder and an informative row for the automated test results is added to the Test Results panel.
Right-click the added row in the Test Results panel and select View Test Results Details from the context menu, or double-click this row in the Test Results panel. Visual Studio displays an editor window with detailed automated test results (see the image below).
If you run a test project apart from a build, Visual Studio automatically adds the results to the Test Results panel, but it does not publish them. To publish these results into the team project’s issue-tracking database:
Switch to the Test Results panel. If it is hidden, select Test | Windows | Test Results from Visual Studio’s main menu to show the panel.
Select the desired test results in the panel.
Click Publish on the panel toolbar.
Note that you can view the results of TestComplete’s automated test execution directly within Visual Studio. You can easily view the test log by double-clicking the row for the desired TestComplete automated test results in the Test Results panel or select View Test Results Details from its context menu. Visual Studio will display them in the test log window which is similar to TestComplete’s test log:
The Microsoft Visual Studio Team System Integration package also provides the ability to publish TestComplete’s automated test results into Team System’s issue-tracking database directly from TestComplete. You can perform your automated test in TestComplete and then create the appropriate issue in Team System’s database visually from the results shown in the TestComplete test log. Besides Team System, TestComplete can also submit test results to Bugzilla.
Team System’s database stores work items that have information about team projects. For instance, a database includes results of builds and test runs. The results generated by TestComplete’s automated tests do not differ from the results generated by other Visual Studio test items. So, you can create a report for these results the same way you create other reports in Visual Studio Team System. For more information on this, see Visual Studio documentation.
You have just learned how to create and configure TestComplete tests in Visual Studio Team System, how to create, configure your team builds and integrate automated test projects with them, how to perform a team build process with automated testing of a resulting application and how to view test results and publish them into your Team System’s issue-tracking database. We hope this information will be useful and help you to create and execute your team build projects making them more powerful with TestComplete’s automated tests. If you are new to TestComplete, download now and try TestComplete for free.