Manual Testing is dead – long live Manual Testing!

This posting is a little later than planned (by about a week), as I had intended writing it after attending the National Software Testing Conference in London, where I was fortunate to speak as well as attend some great talks. I came away with 4 blog ideas, and this is the first one of them.

The demise of manual testing is being discussed in blogs, magazines, conferences, meetup’s etc. If you look at job ad’s, you’d think that manual testing has already died a death and been buried! They all mention ‘Automation tester’ – like its the ONLY thing that testers need to do. So, it was refreshing to attend sessions where people took a different view of things.

I’ve mentioned before about the need for testers to be able to do manual exploratory testing, and it was great to hear Ingo Philipp from Tricentis discuss this in a conference setting. As an industry we need to push testers back towards performing manual exploratory testing, to be complemented by automated regression testing, otherwise we are going to start missing defects due to deficiencies in the overall coverage.

Think about it. An automated test is only as good as the person who wrote it, and only as up to date as when it was last maintained. An automated test cannot make allowances for something that has changed. It cannot stop part-way through and think ‘I wonder what happens if I click this button rather than following the process flow’. It cannot look at the number of steps and highlight that the application sucks from a usability perspective. It cannot point out that the colour scheme is unreadable, or that the company logo is the wrong colour/shape/size etc. It can only run the steps it has been coded to do and validate against what is has been told to check for. So a test may pass, as the application displays what was expected, but what if additional text is present that shouldn’t be there? The test would pass, and unless anyone manually tested that screen, it would go undetected.

I’m not saying that automated tests are unimportant, far from it, but they have their place within a tester’s toolset and are not the only tool available to a tester. Of course you need to have automated tests in place to be able to follow a Continuous integration process, but automated tests cannot cover every possible scenario.

There is another aspect to this as well. Most applications that we develop are going to be used by human beings, so why do we insist on believing that it is best tested by code, without any human test covereage as well? Automated tests should cover the repetitive tests, the load and performance tests, but a person, a tester, needs to look at the application and think about the different paths that the end user could take.

The fun is in the thinking. The benefits are derived from the thinking. A tester’s brain is needed to assess the tests needed, and I despair when I read of really good manual testers with many years experience who feel that they have to leave the field of testing as they do not have automation experience. I sympathise as I came from a manual background. Coding holds no interest for me – if it did, I would have become a developer. So, my career has been based on testing software and working out how best to do so.

We need to stop this freefall ride into automation oblivion, and look at hiring and supporting multi-skilled testers. If a tester cannot write automated code, does it really matter? Better to have a tester who can look at a requirement and work out what needs testing, than a tester who can code but has no clue how to test something!
Developers can write code, so why not pair a develope with a tester to write the automated tests that a tester defines. If a tester wants to write code, and has the time to do so, then that’s great – but we should not be penalising people for knowing how to assess a requriement and define the tests needed (i.e. the core elements of the job), just because they do not have an additional skill in coding automation tests.

I do feel that there are a number of us trying to push back the tide a little to show people the benefits of doing both manual and automated testing, but the more that speakers such as Ingo and myself are getting out there and promoting the benefits of exploratory testing, the better. We need to stop this damaging trend, and ensure that we retain the best skilled testers before they feel undervalued and move on.

Now, who’s with me on this?

Maintenance

Blogging is something that I do sporadically – there are people who seem to have an awful lot of time during the working day to post updates, but I am too busy to do that, and in the evenings I want to unwind and let my brain relax a bit. I find that during the weekend where I am winding down a little is when some of my better ideas pop into my mind.

Like this morning. I went out into the garden to put the Guinea Pig into the outdoor run, where I noticed just how quickly the grass had grown since it was cut only 8 days ago. Looking around, things are growing at a fast pace – shrubs, grass, weeds and the vegetables in the greenhouse. In fact the grass verges were a metre high before the council cut them last week!

I sat with a cup of tea and started thinking about the maintenance jobs I need to do later on today – onions and lettuces to plant out, weeds to pull, some shrubs to trim a little. You may wonder where I am going with this, and what it could possibly have to do with testing, but here’s the thing – it’s all about maintenance. Oddly as I was thinking about gardening, the idea for this blog came to me.

As testers, we need to perform maintenance around our testing artifacts in the same way that gardeners need to do in their gardens.

We have many test artifacts – in my organisation there are testing notes for each application, so that if a tester is new to that team, they will have reference notes about the application, what it does, where environments are, the test data needed, where the automation pack is etc, and of course we will have our test automation suites as well.

Having this information to hand is a good thing – but only if it is maintained and kept up to date. Applications are ever changing and we should ensure that we perform maintenance as part of our core testing approach, rather than leaving it.

The grass outside our houses and on the green up the road was left to grow to a metre tall, and as a consequence children couldn’t kick footballs around until it was cut by the council. It is the same with testing – things become unusable. If we do not maintain our automated tests, they either do not run at all, or a number of them fail, giving a false indication to the development team as to the state of the build. It then become necessary to waste time investigating why some failed, and determining if they are genuine failures or not. And if links are not updated, or test data suites are ignored, they become useless and the time originally invested becomes worthless.

Planning to include maintenance as part of our testing day jobs is a must – but it is a challenge. We haven’t got there yet with all teams, due to pressures for delivery, so this is something that I also need to work on with my team. It isn’t easy, as we have to persuade others that the time spent doing the maintenance is valuable and gives us some pay-back.

I’d be interested in hearing from anyone who has managed to pull this off, as I would be willing to bet it may only be a reality for a minority of testers.

My first challenge today is getting the garden in order – who knows, I might have some more bright ideas whilst pulling out some weeds later (it is very therapeutic!!).