Handling Unexpected Windows in Automated Tests

Author: AutomatedQA Corp.
Last updated: April 14, 2009
Applied to: TestComplete 7

Introduction

One of the biggest complaints against automated testing tools is the consequences of an unhandled unexpected window. An unexpected window is a window that appears during the playback of an automated test where that window was not visible during the recording of the test. When recording an automated test, you implicitly define the expected behavior of the application under test. During the automated test’s execution, unexpected situations can occur, like a modal window that displays with a message about an assertion or an error from the OS, or another application window may overlap the point where a mouse click should be simulated.

If your automated testing tool does not address the issue of unexpected windows, then all of the criticism is warranted. TestComplete, however, can handle unexpected windows flawlessly. This paper will examine the various options that TestComplete provides for handling unexpected windows while automated testing.

The important thing to remember is that not all windows are truly unexpected. If your application contains any calls which generate message dialogs or other windows, your automated test project should contain code or operations that handles any of these windows. If you know that they might appear, then you can handle them before they do.

TestComplete provides tools for handling both windows that you know might appear and windows that are completely unexpected.

Handling Unexpected Windows in Automated Tests

First, let’s define our goal. While it may be that there are other goals, we can probably agree that our primary desire when an unexpected window appears is to exit the window gracefully and to continue our automated test execution. If we have a large automated test project which executes every night at 1:00am, and which takes two hours to run, it’s not in anyone’s best interest to discover one morning that the entire project failed two minutes into the run because of an unhandled popup window.

Keep in mind that TestComplete only checks for unexpected windows when an automated test is performing an action on a window (Click, DblClick, Activate, and others). It does not check for unexpected windows when the automated test is accessing an object’s properties or when it calls a method like WaitWindow.

The window that impedes the running of the automated test can be modal or non-modal. A non-modal window is not an issue, since the operating system neither requires non-modal windows to have an input focus nor be closed before working with other windows. There are three ways to bypass these windows during automated test playback.

Most of the time, even if the unexpected window is floating on top of the tested window, the tested events will still be applied to the correct window; however, if you are concerned that the overlapping modeless window might obscure mouse events intended for the tested window, you can enable the Ignore overlapping window option. To view or modify this option, open the Properties page of the project editor and then choose Playback in the list on the left of the page.

Playback Settings

The other possibility for an unexpected window is to ignore it. Thus, TestComplete will close all the unexpected windows and continue the test run. To do this, you should enable the Ignore Unexpected Window option. The other options in the On unexpected window section will be ignored.

The third possibility is more complex. When a modal unexpected window appears during an automated test run, TestComplete follows a preset sequence of actions. By customizing these various actions, you can tweak how TestComplete behaves in this situation. Keep in mind that, by definition, at the time that you are writing your automated tests, you have no idea what sort of window may appear unexpectedly. If our goal is to recover gracefully from any and all unexpected windows, we can’t just apply one solution and expect superior results. Here is the preset sequence that TestComplete follows:

Automated test execution is delayed until expiration of the Auto-wait timeout interval that is set on the Playback options of your project.
If the unexpected window is still open, TestComplete generates the OnUnexpectedWindow event.
If the window is not closed by the OnUnexpectedWindow event handler, TestComplete posts an error message to the log along with an image of the unexpected window and then acts according to the Playback settings.
If Stop on error is checked, then the run ends at that point.
Likewise, if Stop execution is checked in the On unexpected window section. (The difference is that Stop on error stops on any error, while Stop execution stops specifically on unexpected windows.)
Otherwise, any of the four following actions is taken, depending on whether it is checked or not in the On unexpected window section of the Playback settings. That is, zero, one, two or three actions may occur. All are ways of getting the window to close. When some of the Closing Action options are enabled, they will be performed in the following order:

Press Esc
Press Enter
Send WM_CLOSE
Click on focused control

If the window is still not closed (which may occur if you do not have a closing-action checked, or if the closing action(s) failed), the run stops.

The Sample Automated Test Project

To best understand the features available for handling unexpected windows, you should see them in action. The following sample will illustrate how to handle an unexpected window with TestComplete according to the options that are specified by the project’s Playback settings. To do this, we will use the Generating Unexpected Windows sample application and create a new TestComplete project for automated testing of this application. Generating Unexpected Windows is a simple application especially created to generate unexpected windows. If the Work with exception check box is enabled, the message about an unhandled exception will appear when Enter is pressed. This message is an unexpected window that will be handled by TestComplete.

Creating the TestComplete project

In the following steps, we will create a new TestComplete project. To do this:

If TestComplete is not already running, launch it now. Once it is running, select New | New Project from the TestComplete File menu. This will call the Create New Project dialog where you should specify the project name, location and scripting language.
Click Create to close the dialog, as no further modifications are required.
Right-click the TestedApps node in the Project Explorer panel.
Choose Add | New Item from the context menu. The standard Open File dialog will appear.
Specify GeneratingUnexpectedWindows.exe and then click Open.

Recording the Automated Test

Now we will record the automated test using TestComplete’s keyword testing (keyword-driven testing) feature. The test will launch the sample application, type text into the edit box, switch to the next line, type text again and close the application. Perform the following actions to record the keyword test:

Select Record Keyword Test from TestComplete’s Test Engine toolbar.
Click Run Tested Application on the Recording toolbar.
Type “This is the first line.” into the edit box of the application.
Press the Enter key to switch to the next line.
Type “This is the second line.” into the edit box.
Close the tested application.
Click No on the appeared window to close the application without changes.
Click Stop on the Recording toolbar.

The recorded automated test looks like this:

Recoderded Test

Select File | Save All from TestComplete’s main menu to save the changes and run the automated test project. As you can see, TestComplete launched the application, typed two lines of text and closed the application, thus the test run was successful.

Modifying the Automated Test

Now we will see how TestComplete behaves if the unexpected window appears. To do this, add the On-Screen Action operation that will enable the Work with exception check box to the automated test after the Run TestedApp operation:

Add the On-Screen Action operation to the test. After the operation is dragged, the On-Screen Action wizard will appear.
On the Select Object page of this wizard, select the Work with exception check box on the screen with the Finder Tool. Click Next to switch to the next page.
On the Specify Method or Property page, select the wState property and click Next.
On the Operation Parameters page, click the ellipsis button in the Value column. The Edit Parameter dialog will appear.
In the dialog, select Code Expression in the Mode box and specify cbChecked in the Value box.
Click OK to save the changes.
Click Finish to exit the wizard.

Now the automated test will look like this:

Automated Test

Thus, during playback the Work with exception check box will be enabled and an exception message box will appear when Enter is pressed. This message will be handled as an unexpected window by TestComplete according to the special options. We should specify them in the Playback settings:

Open the Properties page of the project editor and choose Playback in the list on the left of the page.
Make sure the Stop on error check box is disabled in the Runtime section of the Playback settings.
Select the Press Esc check box and clear the other check boxes in the On unexpected window section.
Select File | Save All from TestComplete’s main menu to save the changes.

Executing that Automated Test and Checking its Results

Now we will run the automated test and see how TestComplete handles an unexpected window. At playback, after typing the first line, the unexpected window appears. TestComplete waits as specified by the Auto-wait timeout option in the Playback settings, closes this window and continues the automated test run.

After the test is over, we can analyze the automated test results displayed in the Test Log:

Test Log

When an unexpected window appears, TestComplete saves its image and posts an error message to the test log. You can also see the event message about closing the window by pressing Esc as we specified in the Playback settings. TestComplete generates these messages automatically.

Using the OnUnexpectedWindow Event Handler in Automated Tests

To handle unexpected windows, TestComplete includes the special OnUnexpectedWindow event. This event occurs during the automated test’s execution when TestComplete encounters a window that was not active during the automated test recording. The event gives us the most flexibility when dealing with unexpected windows and performing specific actions, like saving the whole screen as an image or comparing this window to any windows that you knew might appear in the application.

Let’s see how to handle an unexpected window using the OnUnexpectedWindow event handler.

In order to process the OnUnexpectedWindow event, the TestComplete project must include an Event control that contains this event. Creation of this control requires that your project contains the Events project item. By default, it is added. If for some reason you deleted it, you should include this item to your project using the Create Project Item dialog. To do this --

Right-click the project node in the Project Explorer panel and choose Add | New Item from the context menu. This will call the Create Project Item dialog.
In the dialog, select the Events project item, specify its name and location in the appropriate edit boxes and press OK to save the changes.
The Events project item will appear under the project node in the Project Explorer.

By default, after its creation the Events project item includes one Event control, GeneralEvents. This control contains a number of TestComplete events for which you can create event handlers and perform desired actions. The OnUnexpectedWindow event is included in GeneralEvents. If for some reason your project does not hold a control containing the OnUnexpectedWindow event, you should add the event to an existing Event control or create a new control and add the event to it.

Once you have an Event control containing the OnUnexpectedWindow event, you can create an event handler routine:

Right-click the Event control in the Project Explorer panel and select Edit from the context menu. TestComplete will show the Event Control editor in the Workspace panel.
The editor contains two lists: Available Events and Events to Handle. The Available Event displays the events of the object that is specified in the Source object box. These events can be added to the Event control. The Events to Handle list displays the events that were added to the control. The OnUnexpectedWindow event is added to this list by default.
Select the OnUnexpectedWindow event in the Events to Handle list. Activate the Event Handler cell and click the New button that is displayed within this cell. TestComplete will show the New Routine dialog.
In the dialog, specify the name of your event handling routine (GeneralEvents_OnUnexpectedWindow) and the unit that will hold this routine (Unit1); then click OK.

TestComplete will create an event handler and display it in the Code Editor.

Now we can switch to the Code Editor and write code of the event handling routine. It includes operations that will post messages about the unexpected window into the test log and close it.
VBScript:
```
	Sub GeneralEvents_OnUnexpectedWindow(Sender, Window, LogParams)
	  LogParams.Locked = True    
 
       ' Post an error message about unexpected windows to the test log
	  Call Log.Error("Unexpected window")                             
 
	  ' Post an image of the whole screen to the test log 
	  Call Log.Picture(Sys.Desktop.Picture, "Unexpected window", "This image was added to TestLog from the 
	  GeneralEvents_OnUnexpectedWindow procedure")
 
	  ' Close the unexpected window
	  Call Window.Close
	End Sub
```
JScript:

function GeneralEvents_OnUnexpectedWindow(Sender, Window, LogParams)
{
LogParams.Locked = true;

// Post an error message about unexpected windows to the test log
Log.Error("Unexpected window")

// Post an image of the whole screen to the test log
Log.Picture(Sys.Desktop.Picture(), "Unexpected window", "This image was added to TestLog from the
GeneralEvents_OnUnexpectedWindow procedure")

// Close the unexpected window
Window.Close();
}

DelphiScript:

procedure GeneralEvents_OnUnexpectedWindow(Sender, Window, LogParams);
begin
LogParams.Locked := true;

// Post an error message about unexpected windows to the test log
Log.Error('Unexpected window');

// Post an image of the whole screen to the test log
Log.Picture(Sys.Desktop.Picture(), 'Unexpected window', 'This image was added to TestLog from the
GeneralEvents_OnUnexpectedWindow procedure');

// Close the unexpected window
Window.Close();
end;

C++Script, C#Script:

function GeneralEvents_OnUnexpectedWindow(Sender, Window, LogParams)
{
LogParams["Locked"] = true;

// Post an error message about unexpected windows to the test log
Log["Error"]("Unexpected window")

// Post an image of the whole screen to the test log
Log["Picture"](Sys.Desktop.Picture(), "Unexpected window", "This image was added to TestLog from the
GeneralEvents_OnUnexpectedWindow procedure")

// Close the unexpected window
Window["Close"]();
}

Run the automated test again and analyze the results in the Test Log.

Test Log

The test log includes some messages about an unexpected window. These messages are generated from within the OnUnexpectedWindow event handler during the test execution when TestComplete encounters a window that was not active during the automated test run. The first message is a notification that an unexpected window appeared during the automated test, the second message contains a screenshot of the whole screen. This image is shown in the Picture panel of the log page. The next message includes information about closing the window. As you can see, after handling unexpected window the TestComplete continue the test run.

Conclusion

In this article, we demonstrated how to handle unexpected windows in automated tests created with TestComplete. We hope this information will help you create your automated tests and make them more flexible and powerful. If you haven’t tried TestComplete, download and try it today.