Pain? Test Automation w/ Incremental Development

#sw-test #sw-auto

I am a fan of test automation - when used correctly - with the right planning, motivation and expectations (see posts When to use Test Automation part 1 & part 2).

However, if you really want to put the concept (planning, motivation and expectations) to the test then try using it during an incremental development project.

This provides a whole set of additional factors to bear in mind - usually they're in the form of constraints and obstacles. Below is a snapshot of some thoughts in this area.

Typically, an incremental development project will have several teams (combined design & test) that will be working on separate packages of work (ideally with limited inter-dependancies). The aim is to have discrete packages that continually add to the base application/system and do so in a way to continually increase the functionality of the system/application in a controlled manner.

There are several different falavours of incremental development techniques with agile techniques being the latest umbrella grouping.

Problem?

Ok, so what's the issue with test automation and incremental development compared to more traditional techniques?

The decision to go for test automation should be based on a grouping of factors (pros & cons, benefits and costs) as part of a business case for the go decision.

When planning the automation side then factors to bear in mind:

• The turn-around time for the development package

Is it 2-4 weeks or 2-3 months?

• Complexity of the development package.

Typically, a study/analysis should highlight this.

Vital that the automation experience is considered.

Automation issues should be flagged at the same time

as other design issues.

• Have the automation experts ring-fenced to consider impacts across the board - not just for one team.

Benefits are:

Architecture of the automation framework is maintained.

Improvements can continuosly be fed into the pipeline.

• Interdependancies with other teams
  

There will be some - most of the time.

This becomes a planning issue with the project leaders,

technical coordinators and automation experts.

The goal is to find the best fit for the different efforts

and delivery dates.

Benefits?

This all seems like a lot of work, so what's the pay-back?

Well, the goals of any incremental development effort holds true: smaller increments are delivered and verified to build a system in a more controlled way (more checkpoints along the way and ensuring that the system is not broken.)

This gives the result that the automation scripting (or code) is built up in increments with underlying changes to the framework incorporated. The regression testing performed ensures that the deliveries enhance the total test base - ie not breaking any legacy (or if it is broken it's then known and documented.)

Summary

Plan, plan and plan again. Usually plans become out-of-date quickly - so keeping them updated and communicated is important.

Communication. Get the right people involved at the right time. The automation effort should be driven/coordinated by the expert/s - these are involved at the early stages - so that issues affecting the test/automation effort are identified early.

Automation competence. Keep it and ring-fence it. During incremental development this is a competence that may need spreading over several teams so it's important not to dillute the work with non-automation related test work.