In the last post, we talked about PowerShell and DSC at a technical level. In this blog post, I wanted to cover the challenges and bottlenecks that I see within companies as the code enters the build, test and release process (BTR process). To be clear, I am not talking about the practices of how to perform build, test and release. What I am referring to is configuration drift, which is the change from a baseline to a new configuration state.
For example, configuration drift, new system configuration changes, and resource impacts are a few of the challenges Configuration Management and Release Management face on a day-to-day basis. While I would like to say that PowerShell and DSC solves everything (“hammer and everything is a nail” syndrome), a better approach is to focus on the challenges and bottlenecks that PowerShell, DSC and Azure do address. This includes how to get the teams (development thru operations) involved. This is where PowerShell and DSC comes in.
What I will be covering in this post:
This post will be covering how PowerShell and DSC address a number of challenges that I have found with in the development lifecycles Configuration Management (CM) and Release Management (RM) practices.
What I will not be covered in this post:
I will not go into the anything too deep concerning Configuration Management (CM) and Release Management (RM) practices. These are large, complex topics that would take its own blog post
Goal of the post:
Understanding how PowerShell and DSC are a critical part of the DevOps world and how it improve the CM and RM practices.
As customer expectations of fast delivery and high quality are increasing, the challenge for companies and teams is how to develop, test and deploy on a “production-like” environment that does not require a big investment and resources to maintain it.
While reading “Windows PowerShell Desired State Configuration Revealed” – Chaganti, Ravikanth (2014-09-23), I came across an interesting fact that gets to the heart of the problem.
“Gartner publication on the top-seven considerations for configuration management predicted that through 2015, 80% of outages impacting mission-critical services will be caused by people and process issues, and more than 50% of those outages will be caused by change or configuration or release integration and hand-off issues.”
If you have been in the IT business for any length of time, you know that the above facts are true. Configuration drift, poor release practices and communication hand-off are the top bottlenecks in obtaining a Reliable, Repeatable and Reusable, software delivery pipeline.
The other factor at the core of operation and release management practices is the reduction of risk. Risk comes in the form of changes to the environment set or mix-match of environment states at the time of testing. While we may not be able to test on a true production-like system, one goal should be to test on systems that are required to have the same core entities while ensuring the reduction of configuration drift.
The whole process reminds me of atom particals. At the core is the production environment with all the development and system changes spinning around it (I know it is a stretch, but the visual is what I am going for). As in chemistry, if I remove, add, or change one of the elements of the atom, the state of the atom can become changed. Let us hope it is not in a form of a mushroom shape cloud. In addition, your releases should not feel like a nuclear experiment.
For our daily “real life” deployment lifecycle, new features and system changes are a standard occurrence. Detecting and tracking these changes in the entire software development pipeline (Dev thru Prod) may sounded simple enough, but in reality, easy it is not.
In the past, we used a CM DB to store the configuration changes and system states. One of the problem with that approach is the overhead required to develop and maintain the system. This is where DSC comes in. DSC can report on system state using logging and changes a system to a “Known State” as part of the development and release cycle. However, the DSC cannot do monitoring in the classical or operational sense. For example, DSC is only aware of the identifiable entities that are in the scripts and not system changes outside of the script.
This leads to a different way of thinking about Configuration Management. Does the CM team stay in the “Monitor and React” mode or does the CM team take a proactive mode to update and verify system changes using DSC and a software delivery pipeline? The advantage of using the DSC for CM is that the system changes are now tested at the same time as the new application features. Yes, this means that the operation changes will also need to go through the same delivery cycle as all other changes. No more cowboy changes in production.
Using the DSC resources module to manage identifiable entities that can range from a simple file to a role or feature within the Operating System improve the development cycle by creating a configuration baseline that most companies do not have.
There are many times when I am on site with a customer and ask for the build agent configuration and the identifiable entities for the build agents are unknown. If they had DSC scripts in source control as part of the deployment cycle, spinning up a new build agent with all of the roles, features, permission and application would be a non-event. The same applies with an environment set for an application. The build out for a configuration baseline is one of the problems DSC solves.
Configuration drift has plagued the development, testing and release cycle for many years. I am a little bias on this topic since I have experienced the impact and have been in the hot seat when a production release goes bad due to simple framework version, a missing role, or feature setting that was not translated to the other teams.
DSC treats the configuration as code and as part of the overall development cycle. It’s not just an “Ops thing” anymore. Everyone has some “skin in the game”. The big shift I see is the operation team integration with the development cycle. DSC helps bridge the gaps. We should be treating our configuration as code.