Overview
Corel Corporation provides easy-to-use personal
productivity software that helps millions of users worldwide run their businesses
and extend their creativity at work and at home. With the flexibility of TestComplete's
Connected Application feature, Corel's Chris Manley was able to design a fully featured
automation solution.
TestComplete Addressed Corel's Needs
Corel has plenty of experience with automated testing and has continued, over the
years, to employ automation as an effective tool while efficiently assuring the
quality of their products. However, the Corel Test Automation Initiative project
was conceived to address several outstanding needs.
"First, we needed to address the problem of the waxing and waning interest and commitment
to automation that stems from an individualistic approach lacking in standardization"
says Dave Meisenheimer, Automation Initiative - Project Leader at Corel. "Over the
years, individuals have been highly motivated to use various automation strategies
to become more efficient in their daily tasks. However, while an individualistic
approach is quite effective in some ways, we found that maintaining and building
on individual successes was difficult as people changed roles within the company.
We wanted to reduce the amount of time we spent re-inventing the wheel."
"Secondly, we needed to build a system that was accessible to multiple groups and
levels within the company. For instance, reporting results needed to be centralized
and standardized so that the value of automation was more consistently obvious,
and interest and commitment to automation could be sustained."
For example, Corel wanted:
"Our development groups to be hooked on results as feedback for their success/failure on a build by build basis."
"Our managers to be hooked on results as a means of making decisions regarding software releases."
"Our QA testers to be hooked on results for the purpose of easing their testing load."
"With everyone more focused on automation and relying heavily on results, we felt
that automation could contribute more effectively to our overall testing efficiency.
Past experience had shown that both an individualistic approach and, alternatively,
a specialized automation team approach had caused communication gaps that prevented
automation from reducing duplication of effort. With the Corel Test Automation Initiative,
everyone is more aware of the automation test coverage and efficiency gains are
highly tangible and significant."
Thirdly, and pivotally, Corel needed to be more efficient. The clear and precise
mandate from their management was that automation was to make them more efficient
and enable them to release SKUs with greater speed and sustained confidence. "TestComplete’s
connected application feature and the power of C# is what has permitted us to really
nail this requirement. We’ve been able to create a multi-tier framework that is
robust, maintainable and flexible to the demands of our highly customizable software.
Language versions, various product SKUs, and a highly customizable UI are no longer
problems for us as our framework and TestComplete allows us the flexibility to accommodate
the running of these test cases in all these different environments without any
awareness required on the part of the test cases. The test cases are written once,
on our base SKU, and the framework handles all filtering. We can now budget fewer
manual testing resources toward acceptance testing of our SKU releases. These releases
receive better test coverage and benefit from a quicker turn around of test results."
Shopping for a Testing Solution
Corel evaluated the tools available from all of the major vendors and found TestComplete
to meet all of their criteria. "TestComplete excelled, particularly with respect
to the following criteria:
Able to recognize and deal with most of the custom controls within our applications
Highly customizable with respect to how tests are written, stored, and run, and
with respect to how results are reported Unicode-enabled Restricted users Deals well with web controls, Win32, Mfc, MSAA interface, and custom controls
Corel Saves 5-7 Days Every Release
"We now save approximately 5-7 days of manual testing on the release of every major
SKU. Additionally, we are performing more testing with automation than we would
have with a manual test effort and we can now release with greater confidence. Likewise,
while we would not run every test daily through a release cycle under a manual test
regimen, automation affords us the opportunity to do so without cost - as a result,
we run our automated tests daily, and find defects immediately and earlier in the
cycle, resulting in a more efficient triage, debugging, and defect resolution by
our dev team."
By automating the most commonly executed manual test cases, Corel's manual testers
are now free to do more “use-case based” testing that emulates how their products
are exercised by real users.
TestComplete is Affordable and Flexibile
"In addition to the affordability, the greatest feature in our opinion was the connected
application feature. This feature is what provides us with the flexibility to be
extremely creative and to have the freedom to do pretty much anything we’re asked
to do with the system we’ve created. With this feature, we really aren’t limited
by the tool in any way."
Corel on AutomatedQA's Support
"As part of our due diligence we spoke directly with the folks at AutomatedQA before
purchasing the product. They were willing and eager to answer our questions and
even helped us get over a few hurdles during our evaluation period. That level of
commitment to our success with the tools has continued as we’ve built up our solution."
The Corel Test Automation Initiative
The Corel Test Automation Initiative is designed to unify processes for test automation
across teams and products. An integrated system for test case creation and management
has been created to enforce a standardized approach and reduce maintenance costs.
TestComplete and TestExecute are used to aid in the testing of five of Corel's applications
to date. From the WordPerfect Office suite, Corel has automated their application
acceptance tests for WordPerfect, QuattroPro, and Presentations. They are currently
in the process of automating acceptance tests for CorelDRAW and Corel PHOTO-PAINT
applications in their CorelDRAW Graphics Suite.
"We have used automation tools in various ways in the past. Strategies have varied
from individual testers maintaining their own ‘record and playback’ tests to having
a full automation team focused on creating automated test cases. Our current project
attempts to address a number of the short-comings of past initiatives."
For example, automation assets owned by individuals tended to be lost or poorly
maintained as personnel shifted roles within the company. "At one time, a dedicated
team of automation specialists tended to contribute less to overall efficiency than
we would have liked since it was sometimes difficult to manage separate teams in
such a way as to reduce duplication of effort. Our current project attempts to overcome
these problems by centralizing our automation efforts in such a way as to unify
the approach across teams, but simultaneously maintains control of the effort in
the hands of the QA and development teams that require the testing."
Corel's automation strategy, and tactics for executing the strategy, are vetted
by an Automation Work Group that is comprised of QA testers and software developers
from various application groups, and two dedicated automation engineers. Automation
engineer and project architect, Christopher Manley has designed a tiered framework
that abstracts and centralizes much of the maintenance costs. "We have evolved a
base framework that is common for all of our application testing. The base framework
is written in C# and references TestComplete’s libraries such that nearly all of
the interaction with the application under test is proxied via TestComplete’s connected
application features. We are very excited about the ability to structure our framework
this way, as it provides us with ultimate control over everything from how we setup
a test run, to how we interact with the application and how we log results."
Corel's Testing Framework
Overlaid on Corel's base framework is a keyword framework that describes all of
the UI elements in their application (every button, combo box, etc.) from TestComplete’s
perspective. "We have developed a fairly sophisticated relationship between our
keyword framework and base framework such that our various versions, languages,
and customized SKUs of our products are easily supported and maintained. This way
test cases do not have requirements to accommodate such complexities as it is all
handled in our framework. We’ve already reaped great rewards for this abstraction
as a significant update to the UI of our WordPerfect Office Suite was easily handled
with centralized maintenance of our base framework and keyword framework with relatively
little impact on our test cases."
"One of the biggest advantages of the keyword framework is the ease with which non-programmers
can be trained to write and maintain test cases. Our automation engineers have been
able to focus mainly on extending the framework while QA testers take on responsibility
for creating their own test cases. The keyword framework is essentially a language
for writing test cases where each line of the test case corresponds to a particular
user action in the application UI. An object model was designed, such that our keyword
language corresponds 1:1 with application UI, making it very easy for test case
writers to compose each line of their test case. Figure 1a and Figure 1b show how
the application UI is tightly coupled with the keyword language using a few lines
of code from a short test case that would launch WordPerfect’s “Open File” dialog,
point to a file and click the open button in the dialog."
Figure 1a: Code snippet - UI-based object model provides a keyword language that
makes test case creation and maintenance easy.
Figure 1b: UI-based object model provides a keyword language that makes test case
creation and maintenance easy.
Corel Creates Their Own Environment
"We’ve also created our own test case editing environment (fig. 2) so that test
case writers have the luxury of intellisense (fig. 3 and 4) to guide them in their
choices as they build each line of their test code. This frees the non-programmer
from remembering thousands of obscure function names and procedures. None of this
is possible without TestComplete’s connected application feature."
Figure 2: Test case editor that supports keyword language with intellisense. In
addition to easing the process of test case creation/maintenance we standardize
the approach across applications and development teams.
Figure 3: The test case writer is guided by a UI-based object model and intellisense.
Figure 4: The test case writer is guided by a UI-based object model and intellisense.
Corel's Conclusion
"TestComplete is in a league of its own. To say it has high value would be unfair,
as it would imply a compromise that doesn't exist. It is both the best tool for
our purposes and, by far, the lowest priced among the half-dozen or so tools that
we evaluated."
About Corel Corporation
Corel Corporation provides easy-to-use personal
productivity software that helps millions of users worldwide run their businesses
and extend their creativity at work and at home. With its emphasis on customer-driven
product development and affordable pricing, Corel addresses the specific and evolving
needs of value-oriented small businesses and individuals seeking proven alternatives
to more costly and inflexible software products.
As one of the top ten packaged software brands in the world, Corel offers a broad
range of feature-rich software addressing four key market segments: PC graphics,
office productivity, digital image editing and natural-media painting and illustration.
Corel’s award-winning product brands all share important attributes: ease of use,
affordability, reliability and loyal, enduring customer bases.