A Change Is As Good As A Rest

Since mid-February I have been working as an interim Project Management role on a GDPR project, which I have thoroughly enjoyed, and I am now in my last week of the mopping up activities. This had followed a discussion around my previous role being at risk of redundancy (which I shared in a post here).

Knowing that this had an end date, I’ve been looking on and off for my next opportunity/adventure, and had been looking at roles within Test Management or Head of Testing. The problem is that a lot of the roles I saw were not a great fit. I was looking for something where I could make a real difference, coaching a team, improving the testing process, making things better, but job specs were highlighting experience with writing a Test Strategy, and implementing automation, and none stood out as asking for anything different. It seemed that there was already a plan – they just needed someone to come in and work to it. There was little mention of all the other areas of testing that need to be covered – Exploratory, Performance, Load, Security, UAT, and hardly anything about developing people.

I’ve also been reading posts on Twitter, LinkedIn, Slack and other forums on testing approaches, and find I am out of step with what seems to be a popular opinion. There seems to be a simplistic view that testing is all about automation. Even those who should know better – large companies with reputable names who purport to have experience with testing – are still perpetuating the myth that you can replace all manual tests with automated ones. Well good luck with that!

I delivered a talk in May at the National Software Testing Conference on talking about testing, not just automation, and that testing is bigger than just automation. It seemed to strike a chord with a few of the people there, so I know that I am not alone in this, but I still feel that I am in a minority. Do you ever get to the point where you feel you are trying to hold back the tide?

It actually worries me that as an industry, we are allowing ourselves and others to see testing as just automation. And it saddens me that the hard work that many of us have put in over the years to elevate testing and ensure that test roles are seen on an equal footing with other tech roles is being eroded. A testing role is not a pseudo Developer role, but that is how we are being seen more and more. We do a lot of analytical work, but are we getting the credit for it?

I was starting to wonder how long it would take to find the right role for me within Testing – if indeed it existed! Should I stay within testing? Was it right for me? Am I out of step with everyone else? Where would I go? What kind of organisation? Where will I find a good match with the values I hold? So many things to consider.

Then, purely by chance, I found myself having a conversation about how my experience would be a good fit in a Project Manager role! I had considered other avenues from time to time (a few years ago I spoke about transferable skills) but had thought that my experience would preclude me from moving. But the great thing about working in a larger organisation is that as a known quantity, my overall experience and achievements are taken into account, whereas looking for a different role externally would be hard to break into. Whilst there are some gaps, I will undertake training to plug them, but as a PM with experience of testing, and good organisational skills, I should be in a great position to influence how we deliver a quality product.

I started to think about the things that drive me to come to work each day:

  • Doing a job that I enjoy
  • Doing a job that is interesting and challenges me (I am not good when I am bored)
  • Making a difference by doing what I do
  • Getting satisfaction from delivering something
  • Working with good people who have the best intentions

The GDPR project has given me all of the above, and I have really enjoyed it (people give me funny looks when I say that!), therefore continuing on project work makes a lot of sense, especially given the concerns I have around finding a Test Manager role where I would be a good fit.

So – the decision has been made, and in one week, I embark in a new chapter in my career as a Project Manager.

What about testing? That’ll still be a part of my life (it’s been 30 years and I still find website errors without meaning to), it’s instinctive, but I just wont be involved on a daily basis. It certainly wasn’t the move that I had anticipated, but things happen for a reason, and I am looking forward to a new and exciting chapter in my career where I can make a difference.


GDPR – the countdown

With only 3 weeks and 3 days to go to the GDPR deadline, I’ve noticed more and more posts on LinkedIn from people claiming that if you store this particular data, you’ll be in breach of the regulation and be fined. Of course they are probably hoping that you’ll bite, and contact them so they can make some money. I cant help but respond to say where they are wrong, as I really do not want to see people taken for a ride.

The reality is that if you have not started now, then you have more than likely left things too late to become fully compliant by the 25th May – but all is not lost. There are plenty of sites that can offer proper advice and guidance, rather than scaremongering and half truths. I found the https://ico.org.uk/ site to be a great resource, and I certainly referred to this when I started my PM role working on GDPR back in mid February.

I find this to be an interesting piece of legislation – it makes good sense and really is something that we should have already been doing. Even now we are still seeing breach notifications (Twitter being the latest one), so the principles of managing data that we collect as organisations, is a sound one. With the rights attributed to us as individuals/citizens of the EU, we also have a corresponding list of responsibilities as citizens of the EU who gather data on other citizens.

The principle of Privacy by Design elevates Security to the forefront, and forces organisations to do more than pay lip service to securing the data that they collect about us. What’s not to like? I think we have all had enough of seeing data breach notifications.

The principle of having a legal basis to collect and use data ensures that the data collected is justified legally, is only to be used for that purpose, only the amount of data needed is collected and stored, and the data is only to be retained for the period of time that it is needed.

Organisations also now need to ensure that there are ways to amend, delete or export data when requested by individuals, using the right to have data corrected, right to be erased, right to see the data collected about them, or the right to port the data to another organisation in a flat file format (e.g. CSV).

This legislation has created a lot of work across the EU, however the end result has got to be positive for everyone. Organisations have less data to store – so there are cost savings to be made. And it will help employees adopt good data management habits, not storing everything in forgotten folders for years ‘in case we need it’. It would be interesting to see just how much storage is saved overall!

A lot of the work we have had to do is in mapping datasets. What do we have, where is it stored, why is it needed, who has access, is it secured, what types of personal data are stored, how long is it retained etc. Even if you are starting late, I would recommend looking at the data you currently have, log what you have and why – the legal basis.

Look at your sales and marketing areas – how will you ensure that you have a legal basis to contact prospective customers or to send out marketing communications?.

Clean up old files wherever they are stored – Emails, Folders, Sharepoint sites etc.

And look at the IT infrastructure. Security in transit and at rest, how much data is stored, how long is it needed for, what data can be cleared out, could data be anonymised, do you have an updated privacy policy? Personal data comes in many forms, including user ID’s. If you have log files with ID’s, do you need them, and if so how long for?

Sales teams will look at utilisation in order to spot upsell opportunities – but do you need names or email addresses, or are the raw figures helpful for trend analysis? Removing certain personal data from a file is as valid as removing the file itself.

All of these are steps that can be taken to ensure compliance.

Whilst the legislation comes into effect on 25th May, I would like to think that any inspectors will take a more lenient view on an organisation who can demonstrate progress in becoming compliant, rather than having ignored things and made no steps at all. That is my own view though, not an official one!

But that is not the end of the story. GDPR only starts on the 25th May – there needs to be a change in how data is managed from that date in order to remain compliant, and be able to prove compliance.

This work is not going to go away, and it will have a long lasting effect on how we all do our day jobs in the future.

All change!

Well, following the success of my January 1st posting, things on the blog front have been very quiet – but that doesn’t reflect my life in any way at all.

In mid January my role was put at risk of redundancy, meaning a 30 day consultation period that concluded last week. If I am honest, it wasn’t something that totally surprised me, as the tech organisation has been changing over the past 12 months, and the role I had didn’t really fit within the new framework.

So how did I feel? It was very odd, as I didn’t know how I should feel. What should I do? How should I be thinking? Where do I start?
There is no template, and not having been through this before, I used common sense to be honest. I decided to tackle this as a positive thing. My children are 18 & 21, my wife works, so its not as though I am the only breadwinner, and its not the end of the world. I have also worked full time without a break in employment since I left school over 30 years ago, so actually, I felt that I could do with one.

It also struck me that each job change has been to try to move onwards and upwards – and now the cycle has broken. It is really liberating to break out of that process! I had no idea how freeing it would feel.

I work for a great company who have really supported me through the process, putting me in touch with an organisation who support people in their job and career journey after redundancy.

My first task, literally 2 days after I was told, was to set about writing a list of things that I wanted to do:

  • Update my LinkedIn profile
  • Take advantage of the help offered by the company I was referred to for career guidance, and sign up for sessions/webinars etc
  • Update my CV
  • Speak to contacts
  • Order business cards to give out at conferences/meetups etc
  • Look into contracting (I have only ever been a permie)
  • Sort out my shed
  • Clean the greenhouse
  • Fix the shower grouting
  • Plan my radio shows in advance

And so on. It was a random mix of things that I wanted to do, including jobs I had put off and not done, so the idea was to keep busy.

I kept a list of what I did each day – job alerts, agency contacts, applications, webinars attended, shed tidying etc so I could see the journey, and it helped. Being productive and organised is important to me, and I made sure to get up around 7.30 each day so as not to get into the habit of sleeping in late!

So – why update now? Well – things don’t always go as planned. In my mind, I was all set to leave on the 14th Feb, I was fine with that, and had made plans for the 15th & 16th so I would not stop work and suddenly feel a bit adrift. And then the week before, I had a conversation which changed things.

I was asked to stay on until the end of June as an Interim Project Manager in the team I was with up until January 2017, as my domain knowledge would be beneficial. I said yes, and start on Monday 19th. I am looking forward to this – not only does it buy me some more time, but with stakeholder management, delivery and workshops a part of the role, these are things that will help boost my skills. It’s not strictly a Testing role, but there is a lot of crossover, as an increasing number of Test Manager/Head of Test roles are looking for these things, and I have the chance to be doing them now. It’s a win-win.

I feel very fortunate to have been asked, and glad that I approached the whole redundancy process with a positive mindset. Although the redundancy is deferred, I can pick up the search again in a month or two when I am ready, so nothing I have done or learned is wasted. If I hadn’t been through the process, my CV would not have received it’s overhaul, so at the very least, that’s been worthwhile!

Every experience is an opportunity to learn, and we all have the choice as to how we deal with whatever life throws at us. I can empathise first-hand with others who have gone through the same process rather than offering platitudes, with no idea of how that person may have felt.

It’ll be interesting to post again in 6 months and see where life has taken me. Who knows!

My hope for Testing in 2018…

It’s the 1st of January 2018, and at 3pm the rain and grey skies have cleared, and a little blue sky and a few rays of sunshine appear. It’s that little ray of hope in an otherwise grey day that helps make me think of the future, and to wonder where we as an industry will be at the end of the year.

What will be have learned? What will we be doing differently? What new skills and approaches will we have adopted? How will our jobs have evolved?

I have one overriding hope for the testing industry this year, and that is to finally put aside the obsession with just one aspect of the testing craft – ‘Automation’.

There have been so many debates and I think to be honest it’s time to move on, as it is an unnecessary distraction from other things that we should be discussing.

I am going to quote James Bach here (see Testing vs Checking), and the White Paper that you can link to from that page:
“The trouble with “test automation” starts with the words themselves. Testing is a part of the creative and critical work that happens in the design studio, but “automation” encourages people to think of mechanizable assembly-line work done on the factory floor.”

Testing is a craft. It is something that requires thought. It requires a skill to be able to identify what needs to be tested and how to go about testing.

Automation is just the ‘how’, which is fine, but with the focus very much on the ‘how’, we seem to have overlooked the importance of the ‘what’. 

Various comments on LinkedIn by other testing professionals have suggested that this demeans the craft – and I have to agree. Anyone who is not a tester/has no testing background, maybe in a senior management position with budgetary control, may well look at the testing activities, and assume it’s basically writing code to perform tests. This does not help us to showcase the thought processes that we have to go through to identify what needs to be tested – using risk based approaches, exploratory testing, story walk-through’s and our own experiences in general to work out how to try to break something that hasn’t yet been built.

Test automation looks great on paper – who doesn’t want to save time, and get rid of the boring repetitive work. It’s an easy sell. And in theory if we can automate a bunch of repeatable tests, then we have time to spend elsewhere. However this is not always the case. Because we only ever discuss automated tests, senior management can lack visibility of the other types of tests that need to be factored in, not to mention that if you leave an automation pack untouched for any length of time, it will need some work to get it running as there are bound to have been changes to the application in the mean-time.

Lets assume we are testing a new web page. The testers do some manual tests and then start to write automated tests to cover the scenarios. Unless they know otherwise, a team can then assume that the job is done – we have repeatable tests so lets move on. But the automation that is so often talked about covers just ONE PART of the testing needed – Regression.

So – what about performance and load testing for example? Where do they fit in? Another tool is needed to create load tests, but there is also the critical thinking needed to establish what the acceptable performance benchmarks are for 1 user, 10 , 100, 1000 etc. And then the understanding needed as to how to scale up the load tests – do they all repeat the same scenarios, or do we try to mimic user behaviour? The running is the last element of a long thought-driven process.

And I haven’t really covered the benefits of exploratory testing. I’ve raised this point in a previous post – automated tests cannot stop part way through and do something different. Not yet anyway – maybe that’s something that machine learning will introduce! But for now, automated tests will just keep doing the same thing over and over again – checking.

This is not testing.

I’ll repeat myself here – testing is the thinking, the investigating, the risk assessment, the planning of what we need to do, looking for things that have been missed by whoever created the requirement – something that they had never considered could happen. After that, it becomes the ‘how’ – what is the best way to perform the tests – as a person using a keyboard to navigate our way round a web application or by writing automated tests to do that for us in a repeatable way.

My wish for 2018 is that we stop making it seem as though testing is all about the automation. It is not. We are far more than writers of testing code, so let’s showcase what we do that adds real value to our organisations.

We are the critical thinkers – let’s be proud of that.

Happy New Year!


TestExpo – a reflection

I was invited to speak at the Unicom TestExpo conference on Tuesday 31st October at a hotel in Heathrow. I’ve spoken at a number of different events, and I have worked with Unicom before, so knew what to expect.

The day started with everyone together, before breaking into three streams after lunch – Agile, Test and DevOps. It’s a little odd when the big room is half empty as people have moved away to other areas, but it seemed to work ok somehow, although I noticed that people tended to gather at either end of the room, and the middle was a bit empty.

There were some good talks – obviously there were sales based ones, as is to be expected, but I sat with Mark Winteringham (Software Testing Clinic) and got to see his talk on the Automated Acceptance Testing Paradox, followed by Mark Fewster discussing whether Equivalence Partitioning and Boundary Value Analysis were old hat or not. I have to say that I derived more value from these talks than the sales ones, but that is to be expected to a degree, unless you attend an event specifically to find a tool to solve a problem.

My talk was the last of the day, at 5pm, and I was aware that I was the last person standing between the attendees and the drinks reception – no pressure then! My talk was around the Core Competencies of a Good Tester, something I want to expand on in the future (maybe in a small book at some point). I was aware that whilst we had 2 screens showing the slides, I was in the middle, so I needed to walk up and down the stage to make sure that the people at each end of the room felt included, so apologies to anyone who thought I had issues standing still – it was deliberate!

Without giving too much away, I delivered the talk, asked questions to get responses (to keep people engaged), got a few laughs too, which is always a good sign, and at the end had 4 questions from audience members. Now this can seem scary – you don’t know what someone will ask, and you are on the spot, but thankfully I felt that the answers I gave were of some benefit.

It did make me think though in terms of how I see the value in what I deliver, given the preparation time and a day out of the office. One lady particularly had a question, which continued as a conversation with Mark Winteringham and myself afterwards, as she was struggling to make her voice heard as a tester in her team. It reminded me why I set up a global QA Chapter in my organisation, so that testers who felt isolated could talk to others for help. She was so relieved to find that this was not about her – it can be a common theme, and it was at that point that I realised that the value was right there in front of me. Even if just one person got something from my talk, then it was worth it. If I have helped just one person to feel less alone, and to give some guidance and support, then it was worth it.

After the event, and next day too, I received some positive feedback as to how useful the session was, and I want to thank everyone who responded. Gaining feedback helps me to become a better speaker, and to remind me why I am doing so.

This will spur me on to do further talks, and look for ways to give something back to others working within technology. So, if you are reading this, and are stuck with something then reach out, either to me, or to a testing community. There are many out there.

As I said in a previous post – the testing community is such a friendly and helpful one, and I am glad to be apart of it.


Automated testing – the holy grail?

One of the most interesting debates that continues to rear it’s head is around automated testing, and people’s view as to what it is.

It ranges from seeing it as a panacea for all of the testing shortcomings, removing the need for testers as the automated tests do their jobs, through to automated tests being an optional extra if time permits.

I sit somewhere to the centre, not of the view that automated tests can replace people, but seeing the value that having a good set of repeatable automated tests can bring.

So, unpacking that a little, why do I think automated tests are a good thing?

  • Firstly, the concept of using a tool to help us to do our jobs easier has been around since our ancestors started to make flint tools for digging. We advance little without a tool of some sort, but we need to use the right tool, at the right time, for the right reason, otherwise it’s a waste of time.
  • Secondly, who wants to repeat the same tests manually over and over again? Been there, done that, and it’s not fun.
  • Thirdly, any time that you can save by manually repeating the same thing, means more time to think about and cover different test scenarios.
  • Fourthly, its an opportunity to learn a new skill – writing code to exercise code! For many people, it’s actually a fun challenge, but also gives us an insight into the mind of a developer. And that can only be an advantage.

The problems we have though is that some people (normally budget holders, normally outside of technology) believe that adopting automated tests means that they can dispense with testers. No! That is not the case. You cannot automate what you haven’t defined. Someone needs to use analytical thinking skills to determine what the test cases are, and to then test those cases.

The automated test tool is just that – a means to an end. There are skills that are needed to define the tests, and skills needed to write the code to make those tests run. Someone with a background in testing is still needed – whether you call them Quality Engineers, Developers in Test or Testers (I think another blog post on role names is pending!).

Used appropriately, automated tests complement other testing techniques, such as manual exploratory testing. There are different tools for functional UI testing, API testing, non-functional load and performance testing, so don’t be tempted to think that one tool is all that is needed. Selecting the right tool for the job is important.

There is a time element here – investment is needed in the right framework that is sustainable, and familiar to a number of team members, not just one person. It takes time to write automated tests, time to check failed tests, time to fix failed tests, time to maintain the framework and improve it as new ideas and techniques are discovered.

Automated tests cost time to create and maintain – but they save manual effort in execution. Used properly, automated tests can give confidence that:

  • A build has not broken fundamental use cases.
  • Functions built in previous sprints have not been broken (regression tests).
  • The performance of the application meets or exceeds expectations.
  • The application can handle the anticipated load.

I think it’s important to set some expectations here:

  • Automation itself is not the holy grail.
  • It is not a magic wand that gets rid of every problem.
  • It is also not an excuse to remove testing as a function from an organisation.
  • Automated tests cannot stop part way through and go off script, but a manual tester can decide to press the F5 or back button and see what happens. Exploratory testing!
  • Automated tests are a way of executing the same tests over and over again. That’s it!
  • Automated tests complement manual exploratory tests.
  • Automated tests are a tool to help the team.

Remember the saying ‘A fool with a tool is still a fool’.

So use the tools wisely.


Mind The Gap

Testing is all about the gaps.

Ask anyone what a software tester’s job is, and the answer will more than likely be along the lines of ‘to see if a bunch of code does what it is supposed to do or not’.

Ok, I’ve simplified it, but you get the idea. The perception is that it’s basically exercising software to see if it behaves as it should do, in accordance with the requirements.

But testing is more than that. Or it ought to be! It’s about finding the gaps between what a product owner/stakeholder asked for, and what was delivered – from a software behaviour perspective (at functional and non-functional levels) and also from a customer behaviour and usability perspective.

There may be gaps in the requirements that no-one has considered, gaps in the process that the end user will follow, or gaps in the understanding of the data flow between applications. A tester is in a great position to apply critical thinking skills to not just assess the requirements in terms of what is stated, but to identify what is missing, ask the awkward questions and feed that information back into the team.

I wonder though if we push this enough within the industry. So much emphasis is on the ‘automation’ side of things that we lose sight of the other areas where testers can (and should) add value. Automated tests are necessary and valuable, as we cannot manually regression test everything, and I am not saying that we should do so. But these are essentially ‘dumb’ tests, just repeating what they have been written to do. An individual with great critical thinking and analytical skills can be far more productive spending their time looking for gaps than writing automated tests, yet we seem to value writing automated tests over critical thinking. (See what I did there – good use of the Agile Manifesto syntax).

The writing of automated test scenarios is different to the actual automated tests themselves. The scenarios need to be thought out and established before they can be coded, and you need a particular skill to do that (what I term the testers mindset). The coding of the tests also requires a particular skill in developing code. Developers are great at writing code – it’s their bread and butter, so would it not make sense for testers to focus on the scenarios and ask the developers to code the tests?

I appreciate that there are testers who enjoy coding, so this wouldn’t suit everyone, but if I had to make a choice due to resource constraints, I would rather a tester focus on what needs to be tested, and let someone else code how it is done. After all – in order to automate a scenario, you must first define it.

I expect there will be many who disagree with me, and I welcome other opinions on this. As I state on the home page, this is just my ramblings based on my perception of the industry as it stands, and I could be mistaken, so if you know or feel differently, please do let me know.