Case Study: Quantum System Improves Product Releases with TestComplete

Quantum System's Quantum Insight application is a highly configurable and extendable adapter between the legacy Mail Application Programming Interface (MAPI) and modern .NET based interfaces. Using Quantum Insight, the end user can get the benefit of accessing different business data sources and applications from within MS Outlook without switching between applications. Quantum System needed a testing solution to test this highly configurable application so they turned to TestComplete.

Defining a Need for Test Automation

"When we began designing our tests, we realized that our Outlook add-in would not have much of a chance to succeed unless we could assure that Outlook’s standard functionality would never be affected, and of course our functionality should be tested before each release," said Juan Gabardini, from Quantum System.

"Besides that, we expect to have customers running many combinations of Windows and Outlook versions and languages."

"Taking this into account, we needed a cost–effective testing strategy that allowed us to respond quickly to market demands, so automating test cases with TestComplete had a strong appeal."

Choosing TestComplete

Juan explained that they selected TestComplete based on the following steps:

• Defined a set of characteristics that we considered important. We ended up with a list of around 20.
• Researched existing tools, via Internet and word of mouth.
• Pre-selected tools that could be evaluated because they had local representatives, trial versions, online support or technical information available online.
• We evaluated the characteristics of 4 pre-selected tools (TestComplete and 3 world class tools) which provided technical information, local representatives (all but TestComplete have at least one) or trial versions.
• Comparing features between products seemed be of little help to select one, because they seem similar in many tools or because extensive tool use is necessary for a precise evaluation. In the end, 4 features were decisive: Outlook GUI support, cost, technical support and how easy they are to learn and use. We discarded 2 tools because they didn’t have Outlook GUI support and the local representatives were not proactive enough to help us in our pilot.
• Finally, we developed a pilot, implementing two test cases (one simple and one complex) with TestComplete (implemented by us) and the remaining tool (implemented by the local representative).

"We selected TestComplete because it has better support for Outlook's GUI, even if the pilot was implemented by us with only online support and documentation. TestComplete is the easiest way to automate Outlook tests. Other tools require most of the test scripts to be done in ‘advanced mode’, and using workarounds because they don’t recognize Outlook's controls."

"Another important reason was the cost; besides being more affordable in per seat prices, TestComplete is the only one that has runtime licenses. This is important to us, because some development is made by outside developers, and we want them to execute the test cases."

Improved Time Savings and Time to Market

"We have over 400 automated TestComplete scripts, which allow us to execute a regression test of internal version in less than an hour," said Juan.

"Our biggest gain is in our products time to market. We can now accept a new version in two days for internal use and four days for delivering to customers."

"Before using TestComplete, internal versions were barely tested, and our development teams either spent a lot of time troubleshooting bug related problems or they had to wait until the product was tested, affecting our time to market."

Helpful TestComplete Qualities

"Outlook control and .NET control recognition help us to focus on testing our functionality. For instance, we were able to compare Outlook's behavior with and without our product, generating test cases automatically. This was a huge help, since we want to make sure that Outlook’s functionality behaves as expected, but we don’t want to spend our test group time writing test cases for Outlook’s standard functionality," said Juan.

Conclusion

TestComplete has allowed Quantum System to implement their test cases and effectively test their Quantum Insight application for Outlook, improving their quality and product's time to market.