My hope for Testing in 2018…

It’s the 1st of January 2018, and at 3pm the rain and grey skies have cleared, and a little blue sky and a few rays of sunshine appear. It’s that little ray of hope in an otherwise grey day that helps make me think of the future, and to wonder where we as an industry will be at the end of the year.

What will be have learned? What will we be doing differently? What new skills and approaches will we have adopted? How will our jobs have evolved?

I have one overriding hope for the testing industry this year, and that is to finally put aside the obsession with just one aspect of the testing craft – ‘Automation’.

There have been so many debates and I think to be honest it’s time to move on, as it is an unnecessary distraction from other things that we should be discussing.

I am going to quote James Bach here (see Testing vs Checking), and the White Paper that you can link to from that page:
“The trouble with “test automation” starts with the words themselves. Testing is a part of the creative and critical work that happens in the design studio, but “automation” encourages people to think of mechanizable assembly-line work done on the factory floor.”

Testing is a craft. It is something that requires thought. It requires a skill to be able to identify what needs to be tested and how to go about testing.

Automation is just the ‘how’, which is fine, but with the focus very much on the ‘how’, we seem to have overlooked the importance of the ‘what’. 

Various comments on LinkedIn by other testing professionals have suggested that this demeans the craft – and I have to agree. Anyone who is not a tester/has no testing background, maybe in a senior management position with budgetary control, may well look at the testing activities, and assume it’s basically writing code to perform tests. This does not help us to showcase the thought processes that we have to go through to identify what needs to be tested – using risk based approaches, exploratory testing, story walk-through’s and our own experiences in general to work out how to try to break something that hasn’t yet been built.

Test automation looks great on paper – who doesn’t want to save time, and get rid of the boring repetitive work. It’s an easy sell. And in theory if we can automate a bunch of repeatable tests, then we have time to spend elsewhere. However this is not always the case. Because we only ever discuss automated tests, senior management can lack visibility of the other types of tests that need to be factored in, not to mention that if you leave an automation pack untouched for any length of time, it will need some work to get it running as there are bound to have been changes to the application in the mean-time.

Lets assume we are testing a new web page. The testers do some manual tests and then start to write automated tests to cover the scenarios. Unless they know otherwise, a team can then assume that the job is done – we have repeatable tests so lets move on. But the automation that is so often talked about covers just ONE PART of the testing needed – Regression.

So – what about performance and load testing for example? Where do they fit in? Another tool is needed to create load tests, but there is also the critical thinking needed to establish what the acceptable performance benchmarks are for 1 user, 10 , 100, 1000 etc. And then the understanding needed as to how to scale up the load tests – do they all repeat the same scenarios, or do we try to mimic user behaviour? The running is the last element of a long thought-driven process.

And I haven’t really covered the benefits of exploratory testing. I’ve raised this point in a previous post – automated tests cannot stop part way through and do something different. Not yet anyway – maybe that’s something that machine learning will introduce! But for now, automated tests will just keep doing the same thing over and over again – checking.

This is not testing.

I’ll repeat myself here – testing is the thinking, the investigating, the risk assessment, the planning of what we need to do, looking for things that have been missed by whoever created the requirement – something that they had never considered could happen. After that, it becomes the ‘how’ – what is the best way to perform the tests – as a person using a keyboard to navigate our way round a web application or by writing automated tests to do that for us in a repeatable way.

My wish for 2018 is that we stop making it seem as though testing is all about the automation. It is not. We are far more than writers of testing code, so let’s showcase what we do that adds real value to our organisations.

We are the critical thinkers – let’s be proud of that.

Happy New Year!