Automated Software
Testing
by Darrel A. Raynor, PMP, CCP DARaynor@DataAnalysis.com
(C) Copyright 1999 Data Analysis & Results, Inc.
Question
"Is automated testing software a viable option for today's
multi-platform software development projects?"
Problem
Several of my clients wonder about available tools and processes for
automating part of their software testing. Automating your testing can
provide worthwhile benefits such as repeatability and time savings
when done right. From a project perspective, there are four main areas
to consider: deciding to automate your testing, installation,
training, and scheduling tasks.
Automating your software testing usually does not provide
time-saving benefits for a development project. Any time saved is
offset by the install, training, and initial test case development. As
the system moves into system test however, with appropriate use of
automated testing tools and strategies, significant time can be saved.
Higher quality can also be a result, since less time is spent tracking
down test environmental variables, odd situations, and rewriting
poorly written test cases. The big time
savings can occur during the maintenance and enhancement of your
system.
Project Management should be used to plan the implementation of
testing tools. Without such, most automated test implementations I
know of have failed. The complex nature of automated software testing
(AST), including culture change, role change, and process change
should be managed like any
other key project. A recent, and the best book I have found on the
subject (see below) says, "The text provides the project
manager with guidelines concerning the goals and objectives of the
testing effort and the decision about whether to automate testing. It
also offers guidance on introducing automated testing on a project and
outlines the processes for performing test planning, design,
development, execution, and evaluation." 1 Project goals should
be well spelled out. Using AST need not have negative effects if
planned for and staffed early in the project. Adding an experienced
test analyst to the project in the early stages allows time to best
use your chosen AST. The learning curve can be climbed during
non-stress times. If you leave the AST work until the normal test
phase of a project, I can guarantee a rough ride, which could lead to
project, or at least AST, failure. When I start a project or take over
a failing project, I find having a Technical Writer and Test Analyst
available indispensable in staying on plan.
Suggestions
1. Read this book (see below). Sample Parts include: 1
"Introduction of Automated Testing to a Project", and
"Test Planning and Preparation". Chapters include,
"Test Team management", Test Analysis and Design", and
"Test Program Review and Assessment." The book has many WBS
and outlines to
help in planning and execution. These are precisely the topics I am
concerned with for a project. Technical information about any test
tool is readily available from vendors and third part sources.
2. Think about the structure you would like for your Test Team and
their tasks. Pushing to fully understand objectives is the key to best
using AST. To best understand everything involved with your use of
AST, break all the tasks down into work packages, just as you do for
your 'normal' development
tasks. Estimate, execute, and re-estimate the work packages to hone in
on accurate assessments of your Team's skills and the level of effort
required.
3. Document your steps including the planning, control, and execution
so that you and others may follow your lead. Especially document the
tactics you tried that did not work, so that others will be spared
that particular rough road. AST can add much to your project's
quality, and save time of future projects.
4. Push for AST when you have one or more of the following:
Multi-platform releases, many defects or enhancements that need to be
corrected in the next year, code that is 'common' to several products
that has to be tested for and with each product release, repetitive
testing such as for a financial system undergoing constant revision
via maintenance.
Resources
Book - 1 See Dustin, Rashka, and Paul 1999. Automated Software
Testing. Addison Wesley for the best treatise I have seen. Includes a
CD-ROM of documents and graphics.
Web - See http://www.csst-technologies.com/genericDeveloping_Automated_Test_Script.htm
l for a simple guideline on writing test scripts. There is a wealth of
knowledge on AST on the web, any number of search criteria will bring
it up for you.
Organizations - Quality Assurance Institute http://qaiusa.com
puts on a yearly testing symposium. Many AST vendors also have user
groups and conferences. The Software Engineering Institute (SEI) at
Carnegie Mellon University also has papers and other resources
available on AST through the
theory of testing.
-------------------
Darrel A. Raynor is a Project Manager and Project Turn-around
Consultant. He consults, speaks, and writes on using technology in
business. He is a Project Management Professional (PMP), a Certified
Computer Professional (CCP), and has an MBA. Darrel's passion is
building and leading software application development project teams,
specializing in project planning, Year 2000, requirements, and project
execution. Mr. Raynor's core beliefs
include full disclosure, adherence to professional society codes of
ethics, open communication and teaching. Mentoring project leads and
managers is Mr. Raynor's delightful specialty. He founded Data
Analysis & Results, Inc. in 1985. Contact him at 972-935-9525 or DARaynor@DataAnalysis.com.
Trade
names, Trade, and Service marks are the property of their respective
owners.