Cross-Browser Testing With TestComplete

Author: AutomatedQA Corp.
Last updated: December 9, 2009
Applied to: TestComplete 7

Introduction

The goal of cross-browser testing is to make sure that your web site or web application behaves correctly in any browser. This is sometimes very challenging and can cause many QA departments a lot of headache. Performing cross-browser verifications manually is time consuming, since most of the browser versions cannot co-exist with each other.

To automate the visual layout of cross-browser testing it is better to apply specialized web-services like Browsershots, BrowserCam, Litmus and others. These services generate a series of screenshots taken in various browsers and operating systems, so that you can easily find what is wrong or if everything displays correctly.

These tools are alright for testing the visual appearance. But often the program logic depends on the browser and OS version more so than the visual appearance. What should we use to test how the web site, server and application behave and react to user actions in different browsers? Not to mention, do all of this automatically.

TestComplete can automatically perform specific actions over a web site, server app and/or web application and log the results of your web testing. You can even launch your TestComplete web tests under various operating systems and with various web browser versions.

TestComplete 7.5 works with Windows 2000 all the way up to Windows 7 (including 64-bit editions) and supports Internet Explorer ver. 5 – 8; Mozilla Firefox ver. 1.5.0.1 - 3.5 and browsers based on the Microsoft WebBrowser control. So, you can test a wide variety of configurations and combinations using both test labs with a lot of physical computers or virtual test labs using virtual machines like Virtual PC, Virtual Server, VMWare Workstation, VMWare Server or any other similar tools. Also, to automate the virtual machine management you can use Automated Build Studio.

Creating a web test that will work in any supported browser is not as easy as it sounds. The user interface and inner object hierarchy of different browser versions is not always the same, let alone the structure of browsers is always different depending on the vendor. For example, a test recorded with Internet Explorer 6 will most likely not work right out of the box in Mozilla Firefox 2.

However, as hard as this may sound, it’s still possible, but we need to take the browser differences into account when creating web tests.

The purpose of this article is to provide a general concept of creating cross-browser tests in TestComplete and to provide you with solutions to some of the most common problems. To illustrate, with real-world code snippets, we created a sample project that will work with TestComplete’s Web Test Sample (<TestComplete Samples>\Scripts\WebTest\) in a cross-browser test.

Organizing a Cross-Browser Testing Project

One of the most fundamental rules of QA is to “plan your test thoroughly”. So, before we start coding let’s clarify what we are going to do and how we will do it.

The TestComplete project should:

1. Get a list of currently available browsers.
2. Launch the first browser.
3. Prepare for the web test.
4. Perform web test or tests:
   • Open the web page for testing.
   • Perform the web testing actions.
   • Log the web testing results.
5. Close the browser.
6. Launch another browser from the list.
7. Repeat steps 3-6 for all browsers.

Getting a List of Browsers for Cross-Browser Testing

Many techniques can be used to retrieve a list of browsers for your web testing – we can read data from the registry, or from INI, XML, Text or Binary files.

In the sample project we’ll use an XML file (Browsers.xml) with the following structure:

The structure is recognized by TestComplete’s Storages and FileSection objects. As you can see each of the browsers is described by its name and path to its executable.

Launching and Closing a Browser for Cross-Browser Testing

There are several ways to start a browser as shown in the Launching Browsers help topic in TestComplete. The most common and practical approach for cross-browser testing is to use the TestedApps collection. To add an application at run-time we’ll call the TestedApps.Add() method, it returns the index of a newly added browser application which later can be launched with the TestedApp.Run() method. To close the browser, we’ll use the TestedApp.Close() method.

Preparing the Web Test for Cross-Browser Testing

This step includes items that you may need to perform in order to prepare the currently selected browser, TestComplete or the environment for cross-browser testing. Quite often, you don't need any special configurations, and this step may be skipped. In other cases, you may need to – configure the browser settings, assign project or project suite variables, load the appropriate name mapping scheme and so on.

In the sample project we'll use this stage to initialize helper variables and select the desired name mapping scheme.

Performing a Cross-Browser Testing Web Test

Your cross-browser test should take into account the specific technology of the browsers, so that it can be executed in any browser. To do this, the Scripting and Name Mapping project items are needed in TestComplete.

In our web testing scripts the browser specifics will be handled by branching the code with if or switch/case constructs. That is, different code fragments will be executed depending on the currently selected browser. The Cross-Browser Testing via Scripts section describes how to organize the branch and access the page elements from different browsers. The XBrowsing (Script) project in the sample TestComplete project suite demonstrates this approach.

The Name Mapping feature can be applied to address the page elements shown in different browsers by the same mapped name. The Cross-Browser Testing via Scripts and Name Mapping section describes how to create browser-independent tests combining scripts and name mapping. The XBrowsing (Script and Name Mapping) project of the supplied sample TestComplete project suite illustrates this concept.

Cross-Browser Testing via Scripts

The code samples are in JScript, but the tests can be implemented in any other scripting language supported by TestComplete.

Taking into account the information provided in the previous section, we can create the main procedure:

Getting References to a Web Page in Cross-Browser Testing

TestComplete adds a special Page object to each of the supported browsers. This object is a wrapper for the browser control that displays web pages. However, the Page object is typically deep inside the object hierarchy, which varies a lot from browser to browser and from one version to another. For instance, the hierarchy of Internet Explorer 7 and 8 includes windows that correspond to the browser’s tabs (these windows are absent in the hierarchy of Internet Explorer 5 and 6). The window hierarchy of Firefox 2 and 3 also contains one extra window that is absent in the hierarchy of Firefox 1.5. To learn more about the specifics of handling pages opened in tabs, see the Web Testing and Tabbed Browsers technical article.

Tip: Since version 6.0, TestComplete has the “Make Page object a child of the browser process” option that simplifies accessing the Page object from Internet Explorer and Firefox. When the option is enabled, the object is duplicated and becomes a child of the iexplore or firefox process.

To obtain the Page object from various browsers, you can use the routine given below. It has two input parameters: the first one is the Process object that corresponds to the browser; the second one is a string that defines the URL of the sought-for page. If the second parameter is an empty string, the routine searches for an arbitrary Page object, otherwise, it searches for the Page object with the specified URL. On success, it returns the Page object, on failure, it returns Null.

The Page object has methods and properties that are common for all onscreen objects and adds a number of specific methods and properties that let you navigate through web pages in the browser, search for elements of a web page and perform other operations. That is, by addressing the Page object you can avoid problems that might appear when you are working with the user interface.

Here is an example that illustrates this: The location and program hierarchy of the Address edit box is different for each of the browsers. So, opening a page by entering the desired URL via the user interface is browser-specific. The Page.ToUrl method allows you to open the specified address in every supported browser.

There is another feature of the Page object, which is even more important. The object acts as a parent for the rest of the elements of a web page (see the next section), and it also exposes the members of the underlying rendering engine.

Depending on the browser family, the rendering engine is either Microsoft’s WebBrowser or Mozilla’s Window.

Microsoft Engine in Object Browser

Mozlla Engine in Object Browser

The engines have their own set of methods and properties, so we should determine which engine is currently used before referring to its members. This is pretty simple – different versions of Firefox use Mozilla’s engine, all other supported browsers use Microsoft’s engine. So, all we have to do is check whether the browser is Mozilla Firefox:

To make the test reproducible on either of the browser families, we need to organize a minimum of two code branches – one for the Microsoft engine and one for Mozilla. Below are some typical examples:

• Assume you need to navigate to one of the previous URLs stored in the history list. If you record a click over the Back button, this will be a browser-specific action (since all browsers have different GUIs), and you will need separate instructions for each of the browsers. However, if you refer to the engine’s methods, then only two code branches will be needed.
  
  
  …

  if (isMozilla(PageObj))

  PageObj.back();

  else PageObj.GoBack();

  …
• If a page contains active content the Internet Explorer browsers may display notification message that ask the user if the content is permitted, whereas, the Mozilla browsers do not notify the user in the same situation. To playback the test on any of the browsers, we should handle the notifications in IE. So, the code below may be required in this case:
  
  
  if (!isMozilla(page)) AllowBlockedContent(browser)

Accessing Elements of a Web Page for Cross-Browser Testing

All page elements are shown as child nodes of the Page object. To represent the hierarchy of web page elements, TestComplete uses one of four different object models: DOM, Tag, Tree or Hybrid. The chosen web tree object model defines how an element is addressed during test recording and playback. For instance, the same button will be named:

• …Page(“NNN”).document.all.Item(“btnGo”) in the DOM model,
• …Page("NNN").INPUT.Item("btnGo") in the Tag model,
• …Page("NNN").Form("f").Table(0).Cell(0, 1).SubmitButton("btnGo") in the Tree and Hybrid models.

Therefore, a test should be played back using the same web tree model it was recorded with. At design time, the web tree model is set via the project’s Tree model property; at run-time it can be checked and set via the Options.Web.TreeModel property.

If the desired element can be uniquely identified by one or more of its properties (Name, ID, tagName, text, innerHTML and others), you can avoid discrepancy in the object model names by using the Page.NativeWebObject.Find method. This method searches the web page for the element that contains the property value of the given object and returns the found element on success. So, for instance, the code myGoBtn=Page("NNN").NativeWebObject.Find("id", "btnGo") will work fine regardless of the chosen web tree model.

In your cross-browser tests you can use whatever approach you like. However, if the test refers to a lot of elements, then multiple calls to Page.NativeWebObject.Find may slowdown the cross-browser test, therefore in this case it is recommended to set the web tree model and later on address the elements using the model notation.

After you have found the needed element, you can refer to its members. In TestComplete, scripting objects that correspond to elements of a web page contain the following members:

• Methods and properties defined by the Document Object Model.
• Methods and properties defined by the HTML 4 standard.
• Methods and properties provided by TestComplete, namely:
  • Methods and properties common for all tested objects.
  • Visual objects that contain methods, properties and actions common for onscreen objects.
  • Objects that correspond to combo box, list box and check box elements and include control-specific methods, properties and actions.
  • Additional methods and properties provided by the selected web tree model. For instance, the Tree model adds the RowIndex and ColumnIndex properties to the Cell objects (these objects correspond to the TD and TH elements).
• Methods and properties specific to the browser family:
  • Microsoft Internet Explorer or the WebBrowser control
    The list of Microsoft browser-specific element members is provided by: HTML and DHTML Reference on Microsoft Developer’s Network.
  • Mozilla Firefox
    For information on methods and properties provided by Mozilla Firefox, see Gecko DOM Reference on Mozilla Developer Center.

If you want to create a reliable test, you should use browser-specific methods and properties as little as possible. If calls to browser-specific members are inevitable, you should use all possible alternatives, so, the if and switch/case constructs are also welcomed here.

To differentiate the browsers you can use the helper project variable that combines the process name and its major version number.

Cross-Browser Testing via Scripts - Sample

Now, let’s see how to create a cross-browser test using TestComplete’s Web Test Sample. The routines described here are listed in the XBrowsing (Script) project.

The web test routine in our cross-browser test retrieves an arbitrary Page object, navigates to the Web Test Sample page (<TestComplete Samples>\Scripts\WebTest\MainPage.htm), switches to the “Tree” web model, and then simulates actions over different types of web controls.

Below are several notes regarding the code snippet above:

1. As you can see, a major part of the code is common for all browsers. To differentiate between the different browsers, branching is applied.

   Pay special attention to the fragment where the scripting object that corresponds to the modal dialog is obtained. It takes into account the following browser specifics: in Firefox the modal dialogs have the “JavaScript Application” caption, while in Internet Explorer they’re entitled as “Explorer User Prompt”. Due to Microsoft’s Loosely Coupled IE (LCIE) concept, the modal dialogs in Internet Explorer 8 are invoked in the helper tab process. Hence, we need to search for the dialog object within the entire Sys object rather than only within the browser process.

2. When the routine is executed under Internet Explorer, it additionally calls the AllowBlockedContent routine that permits the ActiveX controls to access the computer.

   The routine’s code is the following:

   01.//Enables the active content which is blocked by default

   02.function AllowBlockedContent(browser)

   03.{

   04. var InfoBar, ConfirmDlg;

   05. // Search for the Internet Explorer Information bar

   06. // (It always belongs to the master instance of IE)

   07. InfoBar=browser.FindChild("WndCaption","To help protect your security*",15,true);

   08. if (InfoBar.Exists)

   09. if (InfoBar.Visible)

   10. {

   11. InfoBar.Click();

   12. InfoBar.PopupMenu.Click("Allow Blocked Content...");

   13. // Search for the confirmation dialog

   14. // (It belongs to the IE instance displaying the blocked page)

   15. if (browser.FileVersionInfo.MajorPart==8)

   16. // In IE8 it is the child tab-process

   17. ConfirmDlg=Sys.FindChild("WndCaption","Security Warning",3);

   18. else

   19. // In IE7 it is the same process instance

   20. ConfirmDlg=browser.FindChild("WndCaption","Security Warning",5);

   21. ConfirmDlg.Window("Button", "&Yes").ClickButton();

   22. }

   23.}
3. To verify links, the VerifyLink routine is applied. The routine simulates a click over the link and then checks the opened page. When you follow an invalid link, the browsers display different types of error messages. The routine analyzes the class names of the page elements and if they contain the word “error”, then it considers that link as invalid.

   The routine has three input parameters: browser - the Process object that corresponds to the current browser; parentPage - the Page object that contains the link to be verified; linkElement - the object that corresponds to the <A> element in HTML.

   01.function VerifyLink(browser, parentPage, linkElement)

   02.{

   03. var newPage, newUrl, originalURL;

   04. //Get the initial and desired URLs

   05. originalURL=parentPage.URL;

   06. newUrl=linkElement.href;

   07. //Follow the link

   08. linkElement.Click();

   09. newPage=browser.FindChild("Name","Page(" + newUrl + ")",15);

   10. newPage.Wait();

   11. //Search for the objects which contain "error" in their name

   12. // In Firefox these are "errorTryAgain" "erroeContainer"

   13. // In Internet Explorer these are "ErrorInAddress", "errorDescr" "errorExpl"

   14. err=newPage.FindChild("Name","*error*",10);

   15. if (err.Exists)

   16. Log.Warning("Link " + newUrl + " does not exist",err.FullName);

   17. else Log.Message("Link " + newUrl + " - success");

   18. //Revert to initial URL

   19. newPage.ToURL(originalURL);

   20.}

Cross-Browser Testing via Scripts and Name Mapping

The name mapping feature assigns a custom name to the objects, and uses the mapped name in your cross-browser tests instead of the real name. By creating name mapping aliases, you can “mask” the actual hierarchy of program objects. In addition to giving a custom name, you also need to provide a set of properties and their values to be applied as recognition attributes (otherwise TestComplete won’t find the mapped object). There can be multiple mapping configurations which define the same mapped names and aliases but differ in the values of recognition attributes.

We can apply multiple mapping configurations to create cross-browser tests. If we properly define “dedicated” mapping configurations, then the same mapped name may be used to refer to web page elements in different browsers. This way, the resulting script will be uniformed, but for each browser the appropriate name mapping configuration should be applied. This way we can replace most of the code branching in scripts with name mapping.

The tricky part of this approach consists in defining the appropriate configurations. Unfortunately, the automatic name mapping (introduced in TestComplete7), won’t be helpful in this case, and we need to define the name mapping configurations manually. This means that we need to map not only the desired object but all of its parent objects as well. Typically, the object hierarchy that includes web page elements looks like:

1. The Sys object.
2. The browser process object.
3. A number of window objects. The last window contains the web page.
4. The Page object.
5. The document object (for the DOM and Hybrid tree model).
6. The all collection or tag-named collections (for DOM and Tag tree models).
7. Web page elements.

Mapping Page Elements

While creating the name mapping scheme, we need to comply with the recommendations given below to make the cross-browser test:

• To map the Process object, use the Index and FileVersionInfo properties in addition to the ProcessName property that is suggested by default. This helps take into account (by creating a separate configuration) the diversity between different browser versions. For example, how Internet Explorer 8 uses multiple executables for each tab that is displayed.
• When mapping the Page object use its URL as a recognition attribute. If the page is formed dynamically, it may contain a query string (parameters passed after the question mark in the URL). To make the mapped settings independent of the parameter values, you can use the asterisk (*) wildcard in place of the query values -- Page("http://www.example.com/index.asp?act=*&sid=*"), or in place of the entire query string -- Page("http://www.example.com/index.asp*"). To make the mapped page accessible on any level of the hierarchy, enable the “Find this node on any level of NameMapping tree” property in the Object Name Mapping Dialog. Thus the test would not depend on the current value of the project’s “Make Page object a child of the browser process” option.
• When mapping objects that correspond to web page elements, avoid using browser-specific properties as recognition attributes, use only those properties that are available in every browser. To find common properties open the same tested page in different browsers and use the Object Browser to find the desired element and compare the properties.

Cross-Browser Testing via Scripts and Name Mapping - Sample

Currently, three different name mapping configurations are enough to reflect the browser diversities: one for all Firefoxes, one for IE 7 and earlier, and one for IE 8. The name mapping configurations used here, as well as the listed routines can be found in the XBrowsing (Script and Name Mapping) project of the supplied TestComplete project suite.

Name Mapping Configurations

Having defined these configurations, you should load the appropriate configuration before starting a cross-browser test. The PrepareForBrowser routine will do this:

When the name mapping configuration is loaded, we can address the web page and its elements by their mapped names ignoring the actual browser object hierarchy.

Conclusion

By following the above-mentioned recommendations, you can create a test that will work in any supported browser and make your automated web tests into cross-browser tests. Of course, it’s impossible to go over all possible scenarios with cross-browser testing, so there will be some trial and error work in your test development.

Related Links

Useful recommendations on running multiple versions of multiple browsers on the same computer:

• http://www.thesitewizard.com/webdesign/multiplebrowsers.shtml

Reviews of services for checking the site visual appearance:

• http://designm.ag/resources/cross-browser-testing/
• http://freelancefolder.com/7-fresh-and-simple-ways-to-test-cross-browser-compatibility/