Re-defining QA in a Continuous Deployment Environment

More and more, we work in places where continuous development is championed, MVPs are common, pre-release testing is limited, and the pressure is on to constantly redefine features in new and better ways. Updates happen all the time, without notice, and the way things work is shifting under our feet more often than not. This kind of  fast-paced, constant change moves the web forward and keeps services relevant. It also challenges the status quo of traditional processes for quality assurance.

If deployment is continuous, testing must also be continuous.

  • Ask questions
  • Test continuously
  • Observe first, then listen

These things are inextricable. If you don’t listen carefully, you probably won’t know what’s coming. If you’re not testing, you’re probably not listening. And it all culminates in good communication. What people sometimes forget is that good communication is a give and take—you have to listen as well as you speak. Really great communication is just as much about your ability to interpret and clarify possibly bad communication as it is for you to express your own ideas with clarity. If you can translate what people say they want into what they actually need, you will go far.

  • Keep watch
  • React fast
  • Timing is everything

If you are listening and asking good questions, it’s very likely you are well-informed about what’s launching next—even in a continuous deployment world. That will let you react fast. It’s important to react fast because you take advantage of the close attention developers pay just before and after they ship code. Reporting top issues in that space is so much more effective than reporting them after developers have moved on to their next project. If you’re going to react, react fast.

QA doesn’t have to be so nitpicky. Software development is full of hard choices about priorities. You have to say no to some good projects, and possibly even some painful bugs, if you’re going to meet goals and make deadlines. Because of that, identifying the biggest pain points fast is way more important than identifying all of them. If you’re listening carefully, the most important problems have a way of showing up pretty quickly. Look for high impact, highly visible problems first.

In the QA world, people tend to put a lot of pressure on themselves to find every single problem before code walks out the door, and that’s part of why the process tends to feel disruptive. The pressure to want to keep testing and block shipping is high. Fight it. If you can react fast, turn tests around fast, and hit that open window of awareness where the code is fresh in everyone’s mind, QA can become less disruptive and more effective at the same time.

The goal is not to get the most number of bugs fixed—it is to figure out the biggest pain points for the highest number of people and get those in front of developers in the least disruptive way possible. The number of bugs fixed matters a lot less than the number of people you help.