DevOps is radically changing the culture, tools and processes of most teams. Those that don’t embrace this change are in danger of extinction. However, in order to deploy more frequently with better quality, teams must embrace continuous testing. But aren’t testers part of Waterfall and two-year release cycles? Doesn’t the new world of frequent, rapid deployments preclude testing?
Before looking at how the role of testing can impact DevOps, let’s consider just how fast DevOps is moving.
How many deployments did you or your company do last year? Perhaps you’re doubling the rate of deployments – and you feel pretty good. How much faster are you able to deploy a change of code to production? Twice as fast? Three times as fast?
Unfortunately, linear growth just isn’t good enough anymore. Let’s take a look at graph depicting the result of the Puppet Labs State of DevOps Reports from 2014, 2015 and 2016, showing the performance of high-performing teams vs low-performing peers in terms of lead time (the time from code committed to running in production):
What’s really staggering is seeing the trend line for lead time: high-performing teams were deploying 200 times faster in 2015 and are now deploying 2,555 times faster – and they’re doing so with three times lower the failure rates.
So what has this got to do with testing? Everything! Continuous Integration and Continuous Deployment cannot work effectively if your team isn’t embracing Continuous Testing. After all, more frequent deployments mean that you have to test more frequently! Your team has to change the way it views testing – and testers themselves need to change and embrace this new move.
What Continuous Testing Means to Developers
The days of long, siloed test cycles are over. Developers (and Ops) – in fact, everyone involved in creating software or infrastructure for software – need to build testing into their day-to-day work. This is typically tough for developers – they often make the worst testers! However, if you’re going to deploy frequently and rapidly, you had better make sure you’ve got a fast way to validate the changes you’re making. That’s where test automation becomes critical. Without test automation, teams have no objective way to trust changes.
However, test automation is easier said than done. It’s surprising how few teams architect their code with testing in mind. If you’ve ever had to write unit tests for existing code, you’ll understand what I mean – coding with testing in mind tends to produce cleaner code that is less tightly coupled – all the things developers know they should be doing anyway. The short term pain of refactoring leads to a long term gain of being able to easily add to the automation suite. The key here is to be cognizant of testing throughout the development process – from requirements gathering and prioritizing all the way to build and release.
What Continuous Testing Means to Operations
Testing can also benefit Operations. Many ops teams have dozens or hundreds of scripts lying around that help them complete their day to day tasks. More and more teams are starting to source control these scripts. But Ops teams should be moving beyond just source control and get into testing so that they too have an objective means of measuring quality. There are even scripting test frameworks – for example, Pester can be used to unit test PowerShell scripts.
What Continuous Testing Means to Testers
Teams no longer have the luxury of weeks (or even days) to manually test deployments. This is a tough transition for most testers to make, since they’re going to have to embrace some form of coding in order to automate test suites.
It’s unfortunate that the word “DevOps” doesn’t evoke testing – “DevTestOps” would be a more accurate term, though it doesn’t roll of the tongue as easily! However, testers need to be included in the unprecedented levels of collaboration that DevOps is bringing between Developers and Operations. If you’re a tester and you don’t like the idea of coding, then perhaps your role is to “pair program” with a developer. Testers tend to think about quality in different ways from developers – that’s where the pairing of a Tester (who can architect test plans) and a Developer (that can code up test frameworks) can make a powerful combination.
Of course teaming up with Operations is inevitable for testers too – since testing is invariably going to involve test labs or test environments. Testers will need to learn how to work with Ops to specify dynamic environments and environment configurations using Infrastructure as Code tools such as Azure Resource Manager (ARM) templates or Chef or Puppet and so on. Even if you’re still using “static” test environments, testers can aid Operations in ensuring that the environments are conducive to good testing.
If your Developers and Ops Teams start to embrace Continuous Testing, and if your testers are willing to make the transition from traditional tester to “test automation engineer” or “test automation architect”, your team will increasingly trust the build and release automation pipeline. This positive feedback loop will put you onto the exponential curve and ensure that you keep pace with other high-performing peers.