Automated testing – the holy grail?

One of the most interesting debates that continues to rear it’s head is around automated testing, and people’s view as to what it is.

It ranges from seeing it as a panacea for all of the testing shortcomings, removing the need for testers as the automated tests do their jobs, through to automated tests being an optional extra if time permits.

I sit somewhere to the centre, not of the view that automated tests can replace people, but seeing the value that having a good set of repeatable automated tests can bring.

So, unpacking that a little, why do I think automated tests are a good thing?

  • Firstly, the concept of using a tool to help us to do our jobs easier has been around since our ancestors started to make flint tools for digging. We advance little without a tool of some sort, but we need to use the right tool, at the right time, for the right reason, otherwise it’s a waste of time.
  • Secondly, who wants to repeat the same tests manually over and over again? Been there, done that, and it’s not fun.
  • Thirdly, any time that you can save by manually repeating the same thing, means more time to think about and cover different test scenarios.
  • Fourthly, its an opportunity to learn a new skill – writing code to exercise code! For many people, it’s actually a fun challenge, but also gives us an insight into the mind of a developer. And that can only be an advantage.

The problems we have though is that some people (normally budget holders, normally outside of technology) believe that adopting automated tests means that they can dispense with testers. No! That is not the case. You cannot automate what you haven’t defined. Someone needs to use analytical thinking skills to determine what the test cases are, and to then test those cases.

The automated test tool is just that – a means to an end. There are skills that are needed to define the tests, and skills needed to write the code to make those tests run. Someone with a background in testing is still needed – whether you call them Quality Engineers, Developers in Test or Testers (I think another blog post on role names is pending!).

Used appropriately, automated tests complement other testing techniques, such as manual exploratory testing. There are different tools for functional UI testing, API testing, non-functional load and performance testing, so don’t be tempted to think that one tool is all that is needed. Selecting the right tool for the job is important.

There is a time element here – investment is needed in the right framework that is sustainable, and familiar to a number of team members, not just one person. It takes time to write automated tests, time to check failed tests, time to fix failed tests, time to maintain the framework and improve it as new ideas and techniques are discovered.

Automated tests cost time to create and maintain – but they save manual effort in execution. Used properly, automated tests can give confidence that:

  • A build has not broken fundamental use cases.
  • Functions built in previous sprints have not been broken (regression tests).
  • The performance of the application meets or exceeds expectations.
  • The application can handle the anticipated load.

I think it’s important to set some expectations here:

  • Automation itself is not the holy grail.
  • It is not a magic wand that gets rid of every problem.
  • It is also not an excuse to remove testing as a function from an organisation.
  • Automated tests cannot stop part way through and go off script, but a manual tester can decide to press the F5 or back button and see what happens. Exploratory testing!
  • Automated tests are a way of executing the same tests over and over again. That’s it!
  • Automated tests complement manual exploratory tests.
  • Automated tests are a tool to help the team.

Remember the saying ‘A fool with a tool is still a fool’.

So use the tools wisely.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.