Agile QA Practices: A Better Way to QA

The past several years the Internet has been transformed by a slew of new web based tools and applications. A key success factor in all of these applications is adherence to a development methodology in which product teams release fewer features at a time, but release them at a greater frequency. Adherents to this methodology call this process "iteration."

This methodology allows teams to:

  • respond more quickly to customer feedback
  • adapt the product based upon actual usage
  • become more agile in the marketplace

However, this methodology requires a discipline that is difficult for many to obtain and maintain. That is because it is a radical departure for most software development companies who have deeply ingrained habits and cultures built up around more rigid and waterfall like approaches to software development.

For those that have made the transition to this Agile Development model, what is most intriguing is that while engineering teams and product management have undergone a tremendous process revolution the process of testing software has remained relatively unchanged. And for those companies, I ask you, how agile can your process be when chances are your QA engineers are still testing software the same way they did five years ago and ten years ago?

This is the primary challenge I encountered during Six Apart’s (now a Test Run customer) migration to an Agile Development methodology: no matter how many advances our project managers and software engineers would make in their processes, our QA team was still relying on Excel to manage their test planning process. Excel is a great tool don’t get me wrong, but in this instance Excel made it difficult:

  • to gain transparency into what they were testing
  • to contribute and collaborate in a meaningful and measurable way
  • to audit QA’s progress from release to release

This ultimately resulted in what QA was trying to avoid: more bugs.

On numerous occasions we suffered a regression in overall quality because a QA engineer failed to copy a test case they added in one version of a test plan to another version of the test plan located in a different spreadsheet. Before implementing our Agile development process their process and tools worked just fine. However, their process had failed to keep up with the quickened pace of release cycles. In the past we rarely tested multiple releases concurrently; therefore there as rarely a need to manage the merging and consolidation of test case data throughout a release.

What also made the transition difficult was a turn over in QA staff. As we shuffled resources we lost a lot of knowledge that Excel made difficult to capture. So as new staff members tried to pick up where others had left off, details were missed and bugs surfaced. It took months for those engineers to develop their own internalized knowledge base about our software and be able to operate it with the efficiency of previous team members.

These are some of the challenges that inspired the creation of Test Run. Unlike other test planning tools, Test Run is architected to be iterative, because I believe that test planning is a learning process, just as much as feature development is. In other words, a test case is something that should get better each time your execute it. This is something Test Run helps every QA team achieve in their process, which is something no other test case management tool helps you do. It accomplishes this by:

  • utilizing a single, but large repository of test cases to draw upon
  • maintaining a test case execution history for you automatically
  • allowing users to attach notes and file to a test case that travel with the test case no matter where or when it is executed
  • using tags as a means of categorization as opposed to complex and rigid database schemas

In my many years devoted to optimizing the QA process through useful and meaningful software I have learning one thing over and over: your software development process is only as agile as your least agile stage of development. Therefore, if you really want to capitalize on the benefits of the Agile Development Methodology, it is critical for you to look outside the context of software engineering alone and look at your whole process. Ask yourself:

"If my software engineers are exploring more efficient ways of designing and building applications, shouldn’t my QA engineers be exploring more efficient ways of testing those applications?"

Yes. Of course. A million times, yes!

Check out Test Run today!

Reprinted from the Test Run Blog.

Recommended Entries

4 Comments

Great to see Test Run officially out of Beta. Well done!

I wonder if they have actually tested it...

In about 10 minutes I ran into four defects and two of them prevented me from test case creation and test plan execution.

I am sorry you have run into problems. We are currently in the process of running a live beta of Test Run 1.1, which may be related to the problems you are seeing.

Are you using Internet Explorer? I am in the process of phasing out support for IE6.

I did use Internet Explorer, but v7 (7.0.5730.13)

I have a question. Any plans on having integration with JIRA?

Thanks, Yaroslav

Leave a comment

what will you say?


Recent Comments

  • I did use Internet Explorer, but v7 (7.0.5730.13) I have a question. Any plans on having integration with JIRA? Thanks, Yaroslav ...

  • I am sorry you have run into problems. We are currently in the process of running a live beta of Test Run 1.1, which may be related to the problems you are seeing. Are you using Internet Explorer? I am in the process of...

  • I wonder if they have actually tested it... In about 10 minutes I ran into four defects and two of them prevented me from test case creation and test plan execution. ...

  • Great to see Test Run officially out of Beta. Well done! ...

Close