The Three Deadly Words: Quick, Easy and Cheap
I know what you’re thinking. “He said ‘quick’ and ‘easy’ in the title – he must be lying.” I’ve said that myself from time to time while listening to, or reading some presentation about a new technology. Those two words together almost guarantee the beginning of a fairy tale. If the presenter or author added “cheap” or “inexpensive” to the list of adjectives, I knew he was lying. I avoided using inexpensive in the title as a matter of principle because of my personal bias in that regard. However, inexpensive is an adjective that can be used legitimately, but since you may still be thinking this akin to a bedtime story, I’ll roll with that theme and start with a tragic story.
A Tale of Tragedy
Most of us have experienced something similar. Development has finished creating some new feature or fixed a bug, or whatever. They have checked in the changes and built the project. All unit tests pass and it is ready for deployment to a QA environment so that the changes can be validated and regression tests run before being passed on to UAT. There is only one problem. There is no QA environment available. The current environment(s) are tied up with user acceptance testing, validating a previous build or just don’t exist. When we ask for a new environment we either get management telling us there is no money for a new environment or the glassy stare of an overburdened IT department telling us they can provide an environment – in about six weeks.
This is the story from a large number of my clients. Very often, the solution to this dilemma is that changes are pushed directly to production. After all, “It worked on my machine.” The results are fairly predictable; customer reported defects go up and customer satisfaction goes down. The development team spends more and more time fixing bugs, again deploying the fixes directly to production, which causes more undetected defects, which causes the developers to spend more time fixing bugs… and the cycle grows worse as we draw the noose tighter and tighter around our throats.
Of Heroes and Happy Endings
Let me tell a better story. It’s not a fairy tale. It’s a real story, based on real technology that is available today. This story has a happier ending. It’s a story about a superhero who comes to the rescue.
Once upon a time, there was a startup software company…
The company had developed an innovative software as a service solution. It was a huge success, but quality was an issue. With a lack of resources available a “best effort” approach to testing was adopted using developer workstations and a few local virtual and physical machines cobbled together to create an approximate testing environment. It worked for a while, but as the company gained more customers and the velocity of changes increased to meet the demands of new customers, bugs began appearing more frequently. It became more and more apparent that a “real” testing environment was needed, and not just one, but several. However, there were no funds or resources available. Customer satisfaction was beginning to slip. All seemed lost. All the hard work and innovation would be for naught. Where could they turn? <key dramatic music>
In less than a day, they were able to put together a testing environment that was much more matched to their production environment and were able to adequately test the solution for both accuracy and performance. All this was done at a small fraction of the cost of acquiring and outfitting their own on-premise environment. Initial cost estimates were set at about $700.00 a month per environment if the environments ran continuously. Shutting down the systems when not in use, which initially was planned for about 50% of the time, could significantly reduce those costs depending on other factors such as bandwidth usage, etc…
Since the business logic server was already virtualized, the VHD file was uploaded to Azure where it could be captured (cloned) quickly to create multiple environments. A web server and SQL database were quickly created and test data uploaded to the database.
The Azure environment was configured with a virtual network linked to their on-premise network and test agents were installed and configured using Microsoft Test Manager’s Lab Management feature. Now that the test environment was connected to their on-premise TFS server, they could perform both manual and automated tests and analyze the results to identify areas of development focus. Test data, including IntelliTrace information could be gathered from the Azure environment and used to help developers track down the root cause of defects and performance bottlenecks. Code updates could be seamlessly deployed to the environment and additional tests run.
The resource dilemma was solved. QA could continue to test and developers continued to create new amazing features that more and more customers found invaluable.
A Happy Ending
And they lived happily ever after? That remains to be seen, but the stress and frustration around testing environments has been removed, if not drastically reduced. Sustainable development and testing efforts have been achieved. Future plans to move their TFS implementation from on-premise to Visual Studio Online will reduce another area of maintenance overhead along with reducing the complexity of their testing solution. This is especially true since the company has its eyes on the global market and is opening development offices in Europe and Asia.
Yes, the future is indeed bright.