My hope for Testing in 2018…

It’s the 1st of January 2018, and at 3pm the rain and grey skies have cleared, and a little blue sky and a few rays of sunshine appear. It’s that little ray of hope in an otherwise grey day that helps make me think of the future, and to wonder where we as an industry will be at the end of the year.

What will be have learned? What will we be doing differently? What new skills and approaches will we have adopted? How will our jobs have evolved?

I have one overriding hope for the testing industry this year, and that is to finally put aside the obsession with just one aspect of the testing craft – ‘Automation’.

There have been so many debates and I think to be honest it’s time to move on, as it is an unnecessary distraction from other things that we should be discussing.

I am going to quote James Bach here (see Testing vs Checking), and the White Paper that you can link to from that page:
“The trouble with “test automation” starts with the words themselves. Testing is a part of the creative and critical work that happens in the design studio, but “automation” encourages people to think of mechanizable assembly-line work done on the factory floor.”

Testing is a craft. It is something that requires thought. It requires a skill to be able to identify what needs to be tested and how to go about testing.

Automation is just the ‘how’, which is fine, but with the focus very much on the ‘how’, we seem to have overlooked the importance of the ‘what’. 

Various comments on LinkedIn by other testing professionals have suggested that this demeans the craft – and I have to agree. Anyone who is not a tester/has no testing background, maybe in a senior management position with budgetary control, may well look at the testing activities, and assume it’s basically writing code to perform tests. This does not help us to showcase the thought processes that we have to go through to identify what needs to be tested – using risk based approaches, exploratory testing, story walk-through’s and our own experiences in general to work out how to try to break something that hasn’t yet been built.

Test automation looks great on paper – who doesn’t want to save time, and get rid of the boring repetitive work. It’s an easy sell. And in theory if we can automate a bunch of repeatable tests, then we have time to spend elsewhere. However this is not always the case. Because we only ever discuss automated tests, senior management can lack visibility of the other types of tests that need to be factored in, not to mention that if you leave an automation pack untouched for any length of time, it will need some work to get it running as there are bound to have been changes to the application in the mean-time.

Lets assume we are testing a new web page. The testers do some manual tests and then start to write automated tests to cover the scenarios. Unless they know otherwise, a team can then assume that the job is done – we have repeatable tests so lets move on. But the automation that is so often talked about covers just ONE PART of the testing needed – Regression.

So – what about performance and load testing for example? Where do they fit in? Another tool is needed to create load tests, but there is also the critical thinking needed to establish what the acceptable performance benchmarks are for 1 user, 10 , 100, 1000 etc. And then the understanding needed as to how to scale up the load tests – do they all repeat the same scenarios, or do we try to mimic user behaviour? The running is the last element of a long thought-driven process.

And I haven’t really covered the benefits of exploratory testing. I’ve raised this point in a previous post – automated tests cannot stop part way through and do something different. Not yet anyway – maybe that’s something that machine learning will introduce! But for now, automated tests will just keep doing the same thing over and over again – checking.

This is not testing.

I’ll repeat myself here – testing is the thinking, the investigating, the risk assessment, the planning of what we need to do, looking for things that have been missed by whoever created the requirement – something that they had never considered could happen. After that, it becomes the ‘how’ – what is the best way to perform the tests – as a person using a keyboard to navigate our way round a web application or by writing automated tests to do that for us in a repeatable way.

My wish for 2018 is that we stop making it seem as though testing is all about the automation. It is not. We are far more than writers of testing code, so let’s showcase what we do that adds real value to our organisations.

We are the critical thinkers – let’s be proud of that.

Happy New Year!



Its all about people.

This might sound really obvious, but not much can get done without human involvement somewhere along the line. Therefore people are important, and that is the same whatever industry you happen to be in.

Working in technology, we can be forgiven for believing that technology is king, and people are somewhat incidental. We are focussed on delivering changes to products and using technology to do so. Code is written, tested and released in regular cycles, to deliver benefits to an end-user – a person. But what about the people who are not those referred to in a User Story? Those actually involved in gathering requirements, writing the code, testing it, releasing it? Are they not important too?

Yes, I believe they are, and I also believe that too few companies genuinely believe this to be true. Thankfully the company I work for is very people focussed and personal and professional development is not just something we pay lip service to. Failure to invest in people just means that they will leave. A new study on how millennials see the world indicates that they are happy to just jump ship on regular intervals to get ahead. That contrasts with my belief in loyalty and that moving too often looks as though you lack staying power, but there is a generation gap here, so maybe the difference in outlook is not so surprising! In any case, it’s not just about Generation X, Y and millennials, but people in general. We all need to feel valued, that we are doing something worthwhile and appreciated, stretched so that we have learning opportunities, and trusted to do a good job. No-one wants to feel bored and undervalued, whatever year they were born in.

The problem is that managing people is hard work. Everyone is an individual, with different motivators and needs, and a good manager has to keep track of each person and deal with them in the way that works for them as individuals. When I first started managing, I believed in treating people the same – that’s fair isn’t it? And treating people how I would like to be treated. Both admirable ideas, but fundamentally flawed. Firstly I assumed everyone was motivated by the same thing, and that isn’t the case. And secondly, my preferences are not the same as others. So by trying to do the right thing, I missed out on looking at people as individuals.

Roll forward a number of years and as I matured into my role, attended training courses and benefitted from coaching by my line manager, I came to realise and appreciate the differences between each of my team members. I manage 9 testers now, all unique in their own ways, and I absolutely celebrate those differences. I love being able to find out what motivates someone, and give them opportunities in those areas. It’s great to see how they respond, and the passion with which they do their work when truly trusted and motivated. There is so much to be gained as their manager, as I get to celebrate their successes with them, and see them fired up and ready to tackle new things.

Whilst writing this, it strikes me just how much of a mind-shift I have had to make over the past 5 years, but it has been totally worth it. I would like to think that I am a good manager – not perfect, and still with a lot to learn – but no longer taking a lazy approach to managing people, and instead considering them as individuals, and treating them as such. I mentioned earlier about treating people fairly, and I can still do that by giving them all different opportunities, and not leaving anyone out. It isn’t about treating everyone as though they were clones of each other, and it isn’t about assuming that everyone is motivated by the same thing or has the same dreams and aspirations.

Yes, it takes effort and time to get to know every individual, but then if it isn’t about the people, what’s the point?

The danger of words

Ok, so here I am back on the theme of recruitment. Why? because I am trying to fill two roles in my team, and it has been a struggle.

There is a fundamental problem with CV’s. There are many different ways in which a CV can be laid out, many different words used to describe experiences, but no hard and fast industry rules or best practices governing them.

Every CV is a personal statement, therefore should reflect the individuality of the author. Layouts do not need to be the same, although the type of useful information should be uniform. A good example is where candidates will list a job role, and the first bullet point isnt about their experience or responsibility – its about the company or product, lifted from a web page somewhere. I’m interested to a degree what type of application they work on, but I dont need a company history! Or there are many fonts used (refer to previous posts for that one), or messy tables that make it hard to read.

So, what should a candidate cover?

Thats easy – it’s the skills, direct experience and responsibilities that they are doing/have done in each role – e.g.

  • Writing a Test Strategy for Project X.
  • Working as part of an Agile scrum team, attending Planning, Retrospective and Demo meetings and Daily Stand-Ups.
  • Estimating user stories.
  • Writing tests based on User Story acceptance criteria.
  • Writing automation tests using xyz, in K* language.
  • Executing manual and automated tests, and raising defects in ppp.
  • Performing non-functional tests – Performance using nnn, load using zzz.

And so on.

One of the biggest issue however is how candidates justify the use of certain words, expecting those reading the CV’s to know what they mean.

I’m referring to ‘Experience in‘ and ‘Knowledge of. Both in my opinion are dangerous!

I spoke to a candidate on the phone who had used both of the above all over the CV, including for a tool that I needed the candidate to have used, in order to fit into the team and hit the ground running. Unfortunately we don’t have time to train someone from scratch due to looming deadlines, so I had reviewed a batch of CV’s to narrow down to 3 candidates. This candidate gave me a run through of the skills and tools that they used, but missed out this one. I picked up on this and asked about it – and the reply was that they were aware of the tool but had never used it. I asked why it was on the CV – to which the response was “That’s why I used the phrase Knowledge of”. Wow! Priceless answer……except for the fact that I am in no way a psychic, therefore have no idea what the candidate was thinking when writing the CV! The candidate proceeded to argue with me (never a good idea in a phone interview) until I pointed out that to include it was dishonest and that only skills or tools that they have practical experience should be covered. I also asked how I was supposed to know the difference between their statements! Needless to say the call was ended shortly afterwards.

I then experienced the flip side of this in the next phone interview where someone HAD used the tool on a personal project at home, but decided not to list it. It’s interesting how people’s minds think. My view is that even if you have used something for a short while, or as a home project, it is fine to list it within its context. This shows that although it is not a skill that would be useful if the role requires someone experienced, it could be if there was time to train someone up with basic knowledge already, and it shows initiative and a willingness to learn.

Another issue is repetition. Do candidates realise how many people search through CV’s using the Find function in Word?. If I need a set of technical skills, that’s what I’ll do to see if they are listed in the body of the CV, and not just in the agency cover sheet (believe me, it happens far too often). The problem is that candidates will have performed a number of similar roles, and copied bits and put them under each role. Why? I understand that a candidate will have estimated, tested, attended Agile meetings etc in 3 companies, but surely there is a way of rewording a little in each one – to not do so looks very lazy. If a candidate can’t be bothered to show that each job differed a bit, then a) why move in the first place, and b) does that show a lazy attitude to work, or are they trying to reuse phrases and show me something? Its hard to know, but the decision to phone or face to face interview hangs off the information that is presented in the CV. As a manager, I cant afford to waste time finding out.

Not everyone will agree with me, but I am speaking for myself as a recruiting manager. I want the basics. I have a number of CV’s to review and not a lot of time to do so. Poor layout, waffle or repetition is wearing, annoying and wastes my time, and will not get the candidate through the door.

The best CV’s cut the waffle, tell me what I need to know, don’t contain lies, and are not craftily worded to try to get the CV past me to try to gain an interview. Lies or discrepancies will come out in an interview where everyone will have just wasted a lot of valuable effort, and the candidate will not get the job anyway, so why bother?

My advice to anyone writing a CV is this – read it as someone who has only 5 minutes to screen it. In an ideal world every manager would have longer but that doesn’t happen. What are the standout items that you want them to read? Do you have to put all those bullet points in from a job you did 3, 4, 5 years ago? The most important roles are the most recent 1-2 years, so focus on those. Remove any duplication – the fact that you have used Selenium Webdriver doesn’t need repeating 14 times! And use words carefully! Words can be dangerous – they can mislead and they can give a false or negative impression of you if not used with care. What does ‘Experience in’ really mean. Contextualize it. Help the person reviewing your CV to find the real you. And get someone else to proof read it.

The fact that I mention that I am recruiting is NOT an invitation for recruitment agencies or candidates to make direct contact. You can however find details of roles that we have open in my organisation here: – direct candidate applications only.

Thanks for reading.

Oh – before I forget, the above ‘Curriculum Vitae’ image is from a site who are a group who have set up a website aimed at helping people on a local estate in the north of England to write good CV’s and help them find work. What a great thing to do – well done guys!

Challenges when recruiting testers – number 8

Recruiting challenge number 8 is…..late cancellations or not turning up for interviews.

Most of what I have blogged about up until now has covered CV’s, but moving on from that there is the challenge of getting someone actually in to see me.

So many times within the last 15 years that I have been involved in interviewing, I have spent time preparing for either a phone or face to face interview, only to find that candidates cancel at very short notice (1 minute beforehand was the record!), or do not turn up at all and do not tell me. I find this very rude to be honest, and extremely unprofessional. It is basic good manners to tell someone in good time if you are unable to make an interview!

There have been some bizarre excuses (when candidates have bothered to make them), including one from a lady who got offered a job 2 hours before a face to face interview, so did not turn up. When we phoned her, she was surprised! Was I supposed to be able to guess that she would have been offered a job? 

As a manager, recruiting takes so much time, so this is really unhelpful and I could be spending the time on other things, rather than preparing for a candidate, and sitting around waiting for them.

I treat candidates with respect – they deserve my time in preparing for the interview given that they have taken time off work to attend in person or over the phone. All I ask in return is that candidates show me respect, and turn up if arranged, or cancel with a decent amount of notice!

Challenges when recruiting testers – number 7

Continuing the theme of recruitment challenges, and number 7 is unrealistic salary expectations.

This one may well generate comments but I will explain what I mean.

A Tester needs to understand the value of their skills in the marketplace based upon their skills and experience. Unfortunately many feel the salaries they can ask for are far higher than should be the case, and this causes a number of issues:

1. Distortion in the market. As soon as one company pays 20% more for a role, then that starts to create an expectation that everyone else can afford to do so and follow on. This is not healthy.

2. The effect it has on existing employees. If you have someone who has been promoted to a Senior tester and they move to the next salary band, they are generally happy with this. But if you then have someone more junior who you have to pay more than the senior person, then it can cause resentment if the senior person becomes aware of it. And as a manager you are in a difficult position as you want to treat people fairly.

3. The effect on everyone else if a company decides they cannot justify paying people more than the role is actually worth. The work needs to be done, but there are less people to do it.

Of course it’s this ‘market’ that we all blame (its the same for house prices) – like a mythical entity that has it’s own brain and behaviour. But that is not true – the market can be sensibly ordered if people decide to be realistic about what they will pay for a role, and for candidates to be sensible about what we can and should ask for based on their skillsets.

We all want to earn more, but greed for the sake of it is not a good thing. Good candidates are missing out on opportunities to get in at ground level in organisations for the sake of asking too much money. And money is not the sole factor when looking at a job – there are many other things to consider:

Holidays, charity days, flexible working, pension scheme, office environment, free food and other perks, some or all fares paid, training opportunities etc. An overall package is based on salary plus the value of all the extras.

There are times when candidates need to consider taking a job that may not pay exactly what they think they are worth, as they have a chance to learn new skills, prove their worth and see their salary increase based on merit, and that has to be more satisfying.

Why would any employer want to overpay someone when they don’t know how they will actually perform? It makes no sense.

So for me, the best candidates are those who have a true sense of their value, and are not asking too much or too little…..

This is the flip side of asking too much – asking for too little. It can be an issue especially for a senior role, as it may make the recruiter think a candidate is not confident or experienced enough for the role. It could be that the candidate hasn’t done their homework and assessed their true value. There is no point undervaluing their worth either.

My advice: be realistic, assess the role, the salary offered plus extras, the training, the job growth opportunities and also assess your worth. Then you can go in confident that you can justify the salary you would like.

Challenges when recruiting testers – number 6

Moving on to number 6 on my list of recruitment challenges is………….arrogance.

This might seem a strange thing to add, so bear with me whilst I explain.

A problem I find with candidates is to do with attitude, and it can come across just as much on the phone as it does in a face to face interview.

There is a fine line between explaining what someone has been doing in various projects and the contribution they made, and coming across as though they personally ran the whole test team and came up with every innovation ever implemented! A number of candidates I have spoken to think very highly of their abilities – which in itself not an issue, but if they can’t back up their opinion with cold hard facts, then perhaps this opinion is somewhat misplaced.

Luckily I do not come across too many candidates who speak ill of their current or previous employers (it’s a piece of advice that you can find within 5 seconds of searching for Interview Tips on the internet!), but I really do think that having a more humble attitude during interviews would not harm candidates who are over confident.

A side effect of this sort of attitude is the impression that I (as an interviewer) am lucky that they should deign to come to the interview, and quite frankly the questions are boring and beneath them. Some candidates are very easy to read – it’s all in the body language. 

Of course there is always the extreme example – there was a candidate who after every question I asked him, said ‘That’s a good question’, and then replied in a patronising way. That’s not the attitude I need in my team!

My advice to candidates is this: do not to make the interviewer feel like they are pulling teeth because you don’t give enough detail in your answers, and do not make it sound like you can single handedly save the testing world. Be realistic about what you can do/have done, have a little humility – there are times in business when it is needed, and if you can show it in an interview, it shows a depth of character to the interviewer and is a point in your favour.

Challenges when recruiting testers – number 5

Continuing the series, and number 5 on my hit-list of recruitment challenges is using different fonts and sizes on a CV.

Let me clarify – it is absolutely fine to have a header in one font and the paragraph or bullet points in another, but the issue is where there is a mixture of fonts and sizes within one paragraph or section.

Don’t get me wrong – I do not specifically look for these sorts of things, they just jump out at me. It could be due to my many years of testing, looking for things that are out of sync, but from a hiring manager’s perspective it can be a good thing to spot…..

An example is where I have put out a job advert where we are looking for a specific skill, and I’ll use Selenium again. If I see that a CV has ‘Selenium’ added to a list of tools that the candidate professes to have experience of, and it is in a different font to the other tools, or the same font but a different size, it looks as though the candidate has added it to the CV just before sending off to apply for the role.

Again, it seems a bit picky, but when you add it to spelling mistakes and vague language, you can start to build up a picture of a candidate. If someone has amended their CV to better fit the job specification, that in itself is not a problem (as long as they are being truthful about their experiences, and not just adding bits to look good), as they could be doing housekeeping and removing irrelevant areas.

But it does lead to wondering whether they do have Selenium experience or are they just trying to get through the first screening by adding it in? Do I want to spend time on the phone asking questions only to find they don’t have this and I have wasted my time? Candidates can miss out on interviews just through carelessness.

Once again, attention to detail is expected of a tester, so font types, sizes and alignment should be spot on!