PM Articles > Alan S. Koch > What Is Agile Testing?

What Is Agile Testing?

Part 1 of a Series

by Alan S. Koch, PMP

This column is the first in a series.

Part 1 What Is Agile Testing?

Part 2 Agile Testing: No Mini-Waterfalls!

Part 3 Agile Testing - Pair Testing

Part 4 Agile Testing - Technical Excellence

How does testing fit into the Agile methods? What do testers do on Agile projects? Can an Agile approach remedy traditional testing-related issues? What can I do to make Agile testing work better?

Welcome to our Agile Testing series, where we will address these questions and many more so you can make the best use of testing and testers on your Agile projects. In this first installment, we will take a general look at precisely what the Agile methods say about testing. With that background in place, the following installments will delve into how it all plays out on actual Agile projects, and how we can avoid the ills that many organizations have suffered on their way to effective Agile testing.

The Agile Manifesto and Testing

The words "test" and "quality" do not show up in the Agile Manifesto or the 12 Agile Principles. But these documents contain many statements that imply the need for quality and testing.

The Agile Manifesto contains a statement about the value of "Working Software." Although the word "working" does not necessarily imply high quality, it should be obvious that working software cannot be valuable unless it achieves appropriate levels of quality, which implies testing.

Half of the 12 Agile Principles are more explicit about quality that clearly cannot be achieved without testing.

  1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. "Satisfying the customer" with "valuable software" can only be done if that software is high quality. If we are to ensure that our software is valuable and will satisfy the customer then we must test it.
  2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. The only way that "change" can be to "the customer's competitive advantage" is if it results in software that is more valuable than it would have been. (Refer to the comment about "valuable software" in #1, above.)
  3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Again, the phrase "working software" implies much more than merely working. (Refer to the comment about "working software" in Agile Manifesto, above.)
  4. Working software is the primary measure of progress. In order to represent progress, software must be better that merely "working." Some necessary level of quality is implied which requires testing.
  5. Continuous attention to technical excellence and good design enhances agility. The phrase "technical excellence" implies far more than mere good quality. Testing is one of many activities that are necessary to achieve excellence.
  6. Simplicity -- the art of maximizing the amount of work not done -- is essential. This Lean principle implies many things for software projects. Effective testing is one of those things because poor quality causes developers more work (in the form of unnecessary rework) rather than less.

Agile teams must treat testing as an important activity in their projects.

So we can see that Agile teams, if they are to embrace the Agile Manifesto and 12 Agile Principles, must treat testing as an important activity in their projects.

The Agile Methods and Testing

Each of the Agile methods contains different practices and achieves the values and principles of the Agile Manifesto in different ways. Let's look at the two most referenced of those methods -- Scrum and Extreme Programming (XP).

Scrum (the most widely embraced Agile method) does not directly address the topic of testing because it does not discuss technical activities. But it implies a lot of testing.

  • Acceptance Criteria are recommended for each User Story. The mere existence of such criteria implies that they are the basis for testing.
  • Product Owner Acceptance of each User Story is required before it can be considered to be complete. Although this acceptance may be based solely on the end-of-Sprint demo, it is often that case that Acceptance testing comes into play.
  • Each Sprint must deliver software that is production-quality. Achieving this level of quality implies a variety of quality-related activities, and testing will always be among them.
  • Stabilization Sprints are used when a product is so large and complex that the amount of regression testing required to achieve production-quality is not feasible within each sprint. In Stabilization Sprints, the entire Agile team is focused on nothing but testing (and defect fixing as needed) immediately prior to a release. (Note that when a project includes Stabilization Sprints, the team still strives to achieve production-quality software in each Sprint, even if there is insufficient time for complete testing.)

Extreme Programming (XP) (less widely adopted than Scrum) defines technical practices that are implemented by many Agile teams.

  • Test Driven Development (TDD) (also called "Test First") makes testing an integral part of the development process. In TDD, the developer creates all necessary tests before he/she begins coding a User Story, then continually runs those tests to measure progress, adds more tests if he/she writes code for which there is no test, and is done with that User Story only when all of the tests pass. (We will go deeper into TDD and its variants in a later installment in this series.)
  • Continuous Integration (and its cousin, Daily Integration) means producing a new build every time a programmer completes a work on a User Story. These builds include 100% regression and integration testing to ensure the quality of the evolving product.
  • Automated testing is expected because TDD and Continuous Integration are infeasible without automation. The TDD and integration tests (above) are automated, and those scripts are added to a growing library of automated regression tests. Each developer's work is not considered to be complete until he/she has done 100% regression testing using the library of automated regression tests.

In addition to the Scrum and XP practices described above, there are other widely accepted Agile practices that include testing.

  • Definition of "Done" results when the members of an Agile team agree with each other on the standards they will use to determine when work on a User Story can be called "Done." Every team that defines "Done" includes different things in that definition, but the common theme is that these definitions tend to focus on quality-related activities. It is not uncommon for a team's definition of "Done" to include explicit testing and test automation activities.
  • Continuous Delivery is the natural extension from Continuous Integration (described in XP, above). This involves actually pushing each successful new build into production as soon as it passes the included tests. Continuous Delivery raises the bar on testing because if a build is not production-quality, serious repercussions will be felt!

Agile Testers

While the Agile methods have much to say about testing, it is notable that they do not even mention testers. Most of the Agile methods do not discuss any technical specialty (architect, designer, coder, tester, etc.), and Extreme Programming goes to the extreme of advocating for egoless teams in which all team members are empowered and expected to do any activity that is needed for the team to successfully satisfy the customer.

Testing is integral to most of what Agile teams do

Such an egalitarian team structure is rare in practice; most Agile teams are actually teams of specialists. Nonetheless, from our discussion of the Agile practices and the values and principles they derive from (above), it should be clear that testing is integral to most of what Agile teams do. It is also notable that the whole Agile approach is predicated on open and continuous collaboration among all of the team members.

So when an Agile project includes testers, the implications are as different from traditional team structures as the Agile process is different from a waterfall process. Over the past five years or so, the Agile community has been actively working to understand precisely how testers can provide the greatest value to Agile projects. This understanding is solidifying and includes these elements:

  • One Agile Team. A test team that operates in parallel with the Agile development team is clearly suboptimal. It is best it integrate developers and testers together into one Agile team.
  • Collaboration. Testers' unique skills and knowledge make them valuable partners in all of the collaboration that goes on in Agile projects. Testers must be involved from day one.
  • Developers & Product Owners must still test. Introduction of Testers into the Agile team does not change the need for testing to be part and parcel of what every other team member does. But it does allow for a tester to collaborate with those team members to make their testing even more effective.
  • Testers must test, too. In addition to their expanded roles (in the bullets just above), testers should also be testing; not replicating the developers' or Product Owner's testing, but adding depth and breadth to them so the team can produce an even higher-quality product.

As with everything that goes on in software development, testing is different on Agile projects from our traditional experiences. And with the Agile methods' collective silence on the role of testers, it is no wonder that so many organizations have stumbled.

In the remaining installments of this series, we will go into the details of how the general guidelines we looked at here play out on effective Agile projects. Stay tuned!

Not all comments are posted. Posted comments are subject to editing for clarity and length.

Post a comment

(Not displayed with comment.)

©Copyright 2000-2017 Emprend, Inc. All Rights Reserved.
About us   Site Map   View current sponsorship opportunities (PDF)
Contact us for more information or e-mail
Terms of Service and Privacy Policy

Stay Connected
Get our latest content delivered to your inbox, every other week. New case studies, articles, templates, online courses, and more. Check out our Newsletter Archive for past issues. Sign Up Now

Got a Question?
Drop us an email or call us toll free:
7am-5pm Pacific
Monday - Friday
We'd love to talk to you.

Learn more about ProjectConnections and who writes our content. Want to learn more? Compare our membership levels.