At Northwest Cadence, we have created a work environment that emphasizes excellence, integrity, and out-of-the-box thinking. Our customers have high expectations (rightfully so) and we wouldn’t have it any other way! Our team of outstanding people consistently rise to the occasion. They think smart, work hard, forget the box and have fun exceeding expectations, both inside and out of our company walls.

The ALM Consultant will provide project support, various deliverables, and quality solutions on Visual Studio (including Visual Studio Team System), software design, and Application Lifecycle Management. Engagements will vary and will involve providing expert training, consulting, mentoring, formulating technical strategies and policies and acting as a trusted advisor to customers and internal teams.

The ALM Consultant position requires up to 50% travel. This is a full time position and will be based in the Kirkland, Washington office.

I won’t regale you with stories of the parties or the in-hall discussions, but I do want to highlight three talks, and our role at TechEd.
With all the votes counted, Northwest Cadence did well.  In addition to speaking to great audiences, interacting with really smart people and meeting some new clients, our sessions apparently hit the mark and were rated highly by the attendees!  Three of our talks deserve special mention:

• Branching and Merging for Parallel Development, a talk by Jeff Levinson on the benefits and pitfalls or various branching strategies took 3rd place in the Developer Tools, Languages and Frameworks track.
• Practical Regulatory Compliance and Risk Management, my talk on using Team System to help software development teams achieve regulatory compliance (as well as manage risk) took 2nd place in the Developer Practices track.
• Metrics That Matter: Using Team System for Process Improvement, my talk on metrics took 4th place in the Developer Practices track.

Finally, we spoke at the preconference.  (Can I say “finally”, when it preceded all the other talks?)  Of the 15 preconferences, our talk titled Improve Your Software Development: Real World Solutions with Team System 2008 ranked 2nd just behind the excellent talk on SharePoint planning and governance.

For anyone interested in hearing or seeing these sessions, shoot me an email or leave a comment!  I’d love to share them with you!  We’re representing several of these sessions at our TechTalk: VSTS Firestarter event on the Microsoft Campus this Thursday, and we’ll be recording them.

A special thanks to those customers who generously allowed us to use your stories, and especially to those who went so far as to provide us with their data warehouse and cube.  You guys rock!  Thank you!

Click the image for a link to Tech Ed! And be sure to drop by and visit us at the Team System booth!

Northwest Cadence Further Distinguishes Itself by Earning a Microsoft Competency in Custom Development Solutions and Learning Solutions

Kirkland, WA, USA – 9 January, 2009

Northwest Cadence, www.nwcadence.com, today announced it has attained Gold Certified Partner status in the Microsoft Partner Program with a competency in Custom Development Solutions and Learning Solutions, recognizing Northwest Cadence’s expertise and impact in the technology marketplace. As a Gold Certified Partner, Northwest Cadence has demonstrated expertise with Microsoft technologies and a proven ability to meet customers’ needs. Microsoft Gold Certified Partners receive a rich set of benefits, including access, training and support, giving them a competitive advantage in the channel.

Northwest Cadence provides consulting, coaching, and training on Visual Studio Team System and Application Lifecycle Management. With careful attention to both the present and the future, Northwest Cadence provides sound solutions that can be implemented, metrics that can be measured, and knowledge-transfer to ensure sustainable success. “This Gold designation helps tell the story of who we are. We are extremely pleased to be recognized as a Microsoft Gold Certified Partner. Although this Microsoft designation is quite an achievement, we are especially proud because it represents who we strive to be, every day” said Lori Borg, President of Northwest Cadence. “At Northwest Cadence, our mission as a team is to listen and be responsive to our customers and attaining this Gold status will enable us to continue to better serve our VSTS community.”

“Customers are looking for partner companies that can bridge the gap between their business demands and technology capabilities,” said Allison Watson, corporate vice president of the Worldwide Partner Group at Microsoft Corp. “They need to trust in a company that can act as an expert adviser for their long-term strategic technology plans. Microsoft Gold Certified Partners, which have certified expertise and direct training and support from Microsoft, can build a positive customer experience with our technologies. Today, Microsoft recognizes Northwest Cadence as a new Gold Certified Partner for demonstrating its expertise in providing customer satisfaction using Microsoft products and technology.”

As one of the requirements for attaining Gold Certified Partner status, Northwest Cadence had to declare a Microsoft Competency. Microsoft Competencies are designed to help differentiate a partner’s capabilities with specific Microsoft technologies to customers looking for a particular type of solution. Each Competency has a unique set of requirements and benefits, formulated to accurately represent the specific skills and services that partners bring to the technology industry. Within select Competencies, there are Specializations that focus on specific solution areas that recognize deeper expertise within that Competency. Serving as a specialized path to earning those Competencies, Specializations give direct access to the tools and resources that support that specific area of focus.

The Custom Development Solutions Competency is designed for technology partners providing custom-built solutions for clients that require value-added capabilities to optimize business opportunities.

“Our developer partners enable us to deliver high-quality solutions and applications to our customers,” said Nick Abbott, group manager in the .NET Developer Product Marketing Group at Microsoft Corp. “As the demand for applications built on the Microsoft platform continues to grow, there are more opportunities for providers of custom-developed applications than ever. The Custom Development Solutions Competency provides partners with a way to showcase their expertise delivering custom-built solutions to customers, to enhance partners’ revenue opportunities and positioning them for growth.”

Microsoft partners with the Learning Solutions Competency specialize in delivering high-quality training and full-service learning solutions to help customers maximize their investment in Microsoft technologies. Earning this competency identifies partner members of Microsoft’s premier commercial training and delivery channel, making it easier for partners to expand their scope of services and market potential. Microsoft Gold Certified Partners with the Learning Solutions Competency deliver a comprehensive range of information technology and developer training services, including skills assessment, technical training, student mentoring, and Microsoft Certified Professional exam preparation.

The Microsoft Partner Program was launched in October 2003 and represents Microsoft’s ongoing commitment to the success of partners worldwide. The program offers a single, integrated partnering framework that recognizes partner expertise, rewards the total impact that partners have in the technology marketplace, and delivers more value to help partners’ businesses be successful.

Northwest Cadence specializes in Visual Studio Team System and Application Lifecycle Management. Deeply committed to the VSTS and ALM space, Northwest Cadence is committed to ensuring its clients have an exclusive advantage based on the company’s product expertise and ability to bring it all together. Northwest Cadence is involved in the current and future Microsoft product development of VSTS and employs a team of consultants and trainers who are designated as Microsoft VSTS MVP’s, VSTS Advisory Council Members, and Certified Trainers. Northwest Cadence is proud to be a part of Microsoft’s respected VSTS Inner Circle program.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

For more information, press only:

Lori Borg, Northwest Cadence, (206) 947-0967, Lori.Borg@nwcadence.com

Let’s start with functional bugs. These are the bugs that really shouldn’t be in applications. Now, I am a realist and I understand that it is impractical to try to eliminate all bugs (you can do it but at what cost?) in most applications. This does not hold true for life-critical applications in which there is no margin for error. However, how do you eliminate all of the functional bugs? Testers are not the way to eliminate bugs. If a bug gets to the testing stage you’ve already introduced waste. You want to ensure that before the software gets to the testers, that there are no bugs.

Note: In Lean theory, there would be no testers (or “inspectors”) because all they would be doing is finding a problem that shouldn’t have been introduced in the first place. But in software development I believe this is impractical in most cases.

Let me pose this question to you (this is more applicable if you are a developer but if you aren’t, put yourself in the developers’ shoes): How does a developer know when they are done writing code?

Seems like a simple question, right? I commonly get five answers when I ask this question:

1. You’re never done coding a requirement, it’s an ongoing, evolving process with no end in sight.
2. When my code does what the requirement says it should do.
3. You’re done when the analyst says you’re done.
4. You’re done when all of the unit tests pass.
5. You’re done on the day the code is due.

The fourth answer is usually from groups doing Test Driven Development or heavy unit testing in a true agile process such as Extreme Programming (XP). I’ll address questions about methodology later in this book.

Looking at answers 1, 2, 3 and 5 you can almost feel the depression. Certainly answer number 1 is depressing – essentially you never finish and you can never know when you’re finished. Answer 5 means that the developer feels that maybe they don’t get the time to finish the way they would like to – again, depressing. Answers 2 and 3 are essentially the same (you hope anyway) but there is a major problem. As I’ve noted already, requirements are subjective which means that while you think you’re done you probably aren’t done.

Test Cases before Code

A better answer is: “I’m done when all of the test cases for this requirement pass.” Now this presents some problems. First, most organizations do not create test cases at the same time as they create requirements. Second, those organizations that do create test cases at this stage do not create repeatable test cases (a.k.a. flimsy test cases). Finally, I have not worked with any organizations that require the users to sign off on test cases the same way they have to sign off on requirements. It kind of doesn’t make sense does it? To have customers sign off on documents which are mostly vague but not have the customers sign off on documents that provide definitive proof that a requirement has been met.

So then, the solution to this problem is simple, assign testers to work with analysts and write test cases for requirements before the requirements are handed to developers. Sounds simple, right? This is where a cost justification comes in. You need additional resources earlier in the process which will save costs later in the process. It’s hard to prove a negative to management and get the appropriate resources and funding.

Another issue with this simple solution is that depending on the type of methodology you are using, it may be difficult to break people of old habits. The manners in which requirements are documented make a huge difference in the ease of creating related test cases. For instance, the best format (let me qualify this – in my opinion) for requirements in relationship to test cases are use cases. A sample use case is shown in the table below.

While you can probably find lots of other situations, alternate paths and exception paths, this serves as a good example for entering items you may find at any retail store.

The reason why use cases are the best starting point for test cases is because they already describe the interaction between the user and the system so you know what the user is supposed to be providing and you know what information the system is supposed to be responding with.

If you are using a methodology such as Scrum which advocates user stories there may be some difficulties. Not because there is anything inherently wrong with user stories, but you can’t write code from a user story. User stories are designed to note a high level requirement and describe the users’ use of the functionality to provide perspective and context for the developers. What many forget to do is to provide more detailed requirements (which can be in the form of use cases) which can form the basis of test cases (if you are going to write use cases based off of user stories, then the description would be the user story). In other circumstances there is no formal method for writing requirements documents which means the structure of each document is entirely up to the analyst who writes the requirements.

Let’s look at a sample test case which tests the use case in the table above.

HOLD EVERYTHING! What’s wrong with this test case (besides that it isn’t complete)? This is the typical test case that I see when I work with customers all of the time. If you are a professional tester reading this you know the answer to this: This test case is not repeatable. Believe it or not, I have had to disabuse people of the notion that this type of test case is a good test case. But let’s see what the problem is here. In step 1 I log on as the user. Well, what user? It assumes that the logon was successful so that means I need to log on with a valid user. What user though? This is a test system and will hopefully not have production values in it. In step 3 I’m supposed to scan a barcode for the test. What barcode? Can I use the paperback book I’m reading at lunch for this purpose or maybe the barcode from the candy bar that I bought for lunch? Again, the system is supposed to display the appropriate information which assumes I’ve picked an item actually in the inventory. This makes for a poor testing and a poor coding experience. Let’s try this test case again.

Now this is almost a repeatable test case. What’s missing? There really is only one thing missing here (there are a few minor things but I’m sticking with the major issues) – the setup. How does the system know the tax amount is 8.25%? It can be different depending on where the machine is located. Test cases need to included necessary infrastructure setup and in this case the test would have to be run with different sales tax amounts (or no sales tax amount if we want to assume that maybe this is being used for internet sales in some cases).

In order to have a complete battery of tests this test case would have to be repeated with numerous different values. How many different scenarios do you test? What are the likelihood of the different scenarios occurring and what is the cost if they do occur and the software fails. In the above example, one may assume that you could simply ignore the situation in which the item isn’t found in inventory. After all, if the item is on the store shelf it had to have been received and if it was received it was scanned into inventory. Maybe the chance of this occurring is very, very low but what if it does occur? Without being able to manually enter the item you would not be able to sell the item to the customer. So the cost of this mistake is the amount of the item plus some of the stores reputation. This may be too high a price. It is critical to understand the importance of each of the scenarios when making the decision of what to test.

Well, why don’t we just test everything? The short answer is that it isn’t feasible (again, this does not apply to life-critical applications but it does apply to virtually everything else). The cost in time, resources and equipment to test everything has to be balanced against the ROI. I believe it is possible to make error free applications but at what cost? If it costs 10,000 to test every aspect of the alternate paths but those only become a reality once every 5,000 sales for low dollar amounts, does it provide a benefit? The answer is probably not. As with almost everything else, use the 80/20 rule. That is, 80% of the usage of your application is going to involve 20% of the code (we write an enormous amount of code to handle exception paths and alternate situations). Test that 20% of the code 100%. The other 80% of the code can be tested as problems are discovered or as time permits.

Now that you have a series of good test cases which concretely detail what the system must do, a developer can now answer the question, “When do you know you are done coding?” The answer is, “When the functional test cases I was provided pass.” This is the only right answer (but not usually the one the developers can use).

Roles & Responsibilities

The figure below shows who is responsible for transitioning items between states in a perfect world.

This figure very succinctly states the following:

1. A user, analyst, developer or tester can file a bug report.
2. A project manager must assign the bug report to someone to investigate. The person they will assign it to is an analyst.
3. An analyst can determine if the investigation is complete. They can also close a bug for the reasons noted in the previous section.
4. A project manager assigns the bug to a developer (or not if the bug is being fixed in a later release).
5. A developer will set the bug to Active when they start working on it.
6. A developer determines when the bug is fixed.
7. Depending on the release process, either a Release Manager or QA Manager will note when the bug is ready for testing (and assign it to a tester). This is because there may be a configuration management (i.e. branching structure) which the code fix needs to be promoted through so that the testers can actually get it in a good build.
8. A tester will note that the bug is being tested.
9. A tester will note that the bug fix has been verified. If the bug has not been fixed, a tester can also re-assign the bug to a developer.
10. A release manager will determine which release the bug fix will be deployed in and prepare the appropriate documentation.
11. Upon successful release, the project manager will close the bug.

Reports

So, you have this great bug tracking process now and you know what information you can get from each step. How do you get it easily? What reports are most important? Where do you cut down on waste?

Waste

Let’s start with looking at a report showing waste and determine how it might be eliminated. The next figure shows a chart of the average flow time for bugs in a system using the above process.

This chart is based on the following data:

Note that I have included only working hours – no weekends or after hours. While this would normally be an average of these values, I wanted to demonstrate how time is determined. Still, if this were an average, you would note that the average time to fix bugs and deploy them is 67.5 hours or almost three days on average. Where can you reduce the amount of time wasted?

Note

In Lean, batching is considered a bad process. The reason for this is that when you batch it means that someone is waiting to receive all of the times that make up that batch and therefore time is wasted. However, in software development, and specifically when dealing with change and release configuration management there really is no other way because fixes almost always must go in together. Promoting them one at a time would take more time than it would save and is also a major configuration headache.

Looking at the chart and table, the actual time spent working on the bug is 15.25 hours (the hours spent investigating, fixing and testing). The time wasted is 50.25 hours (the hours spent waiting while the bug was waiting to be investigated, between the investigation being completed and the bug being assigned, between the bug being assigned and the bug being worked, between the bug being fixed and it being tested and finally between the time the bug was tested until the time the bug was deployed). Again, going back to the principle of waste in Lean theory, the 50.25 hours is complete waste. Now having said that, let’s take a reality check since not all of Lean can apply to software development. Where can you reasonably shave hours?

The first thing to do is to understand why an item may be in a given state for an extended period of time. Let’s look at a hypothetical process and see what may happen in some of these states where waste is occurring.

The next post in this series will cover the hypothetical process. We’ll also start covering different scenarios and organization size when dealing with issues because this is NOT a one-size fits all process.

Fixed, unverified

This state indicates the bug has been fixed according to the developer.

This means that the developer has fixed the code, checked it in and associated it with the appropriate bug work item. They have run the test case provided by the analyst in the Under Investigation state. In addition, the time between the bug being in the Active state and the Fixed, unverified state is the time it took the developer to fix the bug. In other words, we now have an approximation of the developer effort to fix the bug. It doesn’t provide an exact time but it provides an overall throughput time.

Ready for Testing

This state indicates the bug has been assigned to a tester but the testing has not yet started.

There are a number of reasons for this state. The first is that your branching model is such that a tester does not have access immediately after the code has been finished (i.e. you are waiting to promote code to the QA branch). The second is that there is simply other work going on and that the tester just can’t get to it right now. The transition between Fixed, unverified and Ready for Testing should be made by a Test Manager to acknowledge that the bug has been assigned to a tester (or to the testing group in general) and that it is on the right code branch for testing. The transition to this state does not provide any other meaningful information.

In Testing

This state indicates that the bug is actively being tested.

The other important information provided by this state is the amount of time between the bug being ready for testing and when the testing started. This again shows waste.

Fixed, verified

This state indicates that the bug has been fixed according to independent testers.

The time between this state and the In Testing state provides the amount of time spent testing this bug and is considered work time.

Note: For those familiar with Lean theory, this work would be classified as waste. The reason is because work should never pass a station with defects – it means the process for building the code needs to be fixed. However, as I have mentioned before, I believe this is mostly impractical in software development.

Scheduled for Deployment

This state simply indicates that the fix is scheduled to be released.

An important piece of information to capture at this state is the release that the fix is being deployed in. This helps create a Release Notes document to provide the user explaining the changes in the upcoming release.

Closed

The bug is closed after the release that it is included in has been deployed.

The time between the bug report being Proposed and the bug being Closed provides the entire throughput for the bug being fixed. This information is a good metric of whether or not your process is improving (if reducing the amount of time to accept and fix bugs is one of your goals). While removing individual pieces of waste is important, you have to keep your eye on the overall goal.

A note about UAT

You may be wondering about the lack of inclusion of User Acceptance testing steps to track users’ acceptance of the bug. There are two reasons why this isn’t included. The first is, does it provide you any additional information? Sure, it lets you know the users accepted it, but you can do that as part of the QA process. Second, by doing UAT separately it means that you have to have separate UAT environments from QA which many organizations can’t afford. Is the cost worth it? This may be something you need to take into account as you think through your process.

This finished the process and description of each step in the process. The next post in this series will cover Roles & Responsibilities and Reports and how to track improvements in the process.

Let’s look at each of the states so we understand what the state represents. And if this looks like the start of a new methodology, well, it is and it isn’t. It is because I’m going to detail out of this series of posts how to handle numerous items and there is a lot of methodology and process displayed here. But it isn’t because much of this is just common sense. Also, note that you do not need to implement every step. This process that I’m describing here can be scaled based on your needs.

The states and their descriptions are these:

Proposed

This state indicates that a reported bug has been received. It does not indicate that any action has been taken with regard to working on the reported bug.

The key item to note is the reason which in this case tells us what group filed the bug. This is important because as has already been noted, it costs more to fix a bug the later it is found in the process. This allows you to note where the bugs are found and determine where certain weaknesses lie.

Under Investigation

This state provides status to the end user to indicate that the bug report is currently being triaged.

The state transition provides us one very useful piece of information – the time it takes from the bug being received until someone starts working on it. This is the first of the areas where we find waste.

This state requires some work though – no free rides here. Several things need to happen:

1. A duplicate bug must be checked for. If a duplicate is found, then this bug needs to be closed and linked to the original bug report.

2. Find the test case that relates to the functionality at issue.

3. The analyst needs to determine what the correct functionality is supposed to be. If it works correctly (as designed) then what the user is actually reporting is a change request and a change request needs to be created. This bug can be closed and linked to the change request.

4. The bug needs to be reproduced. If it can’t be then it is closed.

5. If the bug can be reproduced, then the test case needs to be modified (or created).

What does this information provide us?

If the bug is a duplicate it means at least one thing and maybe two. First it means that this bug occurs in a commonly used piece of code and is therefore a cause for concern. Secondly, depending on the lag time between the original bug report being filed and the duplicate bug being filed may indicate that the bugs aren’t being fixed in a reasonable period of time.

If you cannot find the associated test case(s) then there is another issue. First, why was no test case created? Was it because this is in a less frequently used part of the system? If so, that may be acceptable (maybe it isn’t part of the normal path and so minimal testing was done on it), but it means that this part of the system is being used and additional test cases may need to be created to proactively identify any additional, related bugs. If you do find the test case, what about the test case missed the bug? Is there a gap in the test case? Are all paths not tested? Can you go back and review the code coverage report to see if there is a path through the code that was missed? These questions and their answers will help you proactively identify other areas in this particular area of code so you can take mitigating steps.

What if the functionality is as designed? The user is then requesting a change to the functionality – maybe. They may not understand how the functionality is supposed to work. But what does this tell you? Were the users trained in the correct use of the application? Is there online help for the application and is it effective? If the users haven’t been trained and you receive a high number of these items then maybe it’s time for a training class. If they have been trained and there is online help, then maybe the help isn’t effective or effectively integrated into the application. This may indicate that changes in those areas are needed – but only if you know what to look for.

If the bug can’t be reproduced, it must be closed. However, it must not drop off of the radar. When duplicate bugs are found you need to remember to check the closed bugs as well. If you find a number of non-reproducible bugs being duplicated, it may be time to activate the bug and try to have the developers spend some time investigating the situation.

Finally, if the bug is validated, that isn’t good enough. There needs to be a test case which shows what the correct behavior of the system is supposed to be so that the developers know when they are done fixing the bug and they have a way to verify that fix. If there is an existing test case and it is incorrect, then you need to look back at the requirements to determine if the test case was incorrect or whether the requirements were incorrect and make the appropriate changes (each of these points to a different area that needs to be looked at for improvement).

Investigation Complete

This state indicates that the bug report has been verified and is now an actual bug. And again, it provides a solid indication to the end user of the status of their bug report.

The transition from Under Investigation to Investigation Complete provides the first piece of information on work performed in order to fix the bug (after all, you can’t fix the bug until you have verified it).

At this point the assignment of work can commence. However, there may be a high number of work items to assign so it may be up to a Change Control Board (CCB) to accept and validate work for a release. If that is the case then the list of Investigation Complete items will make up part of the agenda. This however can present problems. See the section entitled “Waste” later in this chapter for the problems related to this and some ideas on how to fix them.

Assigned

This state is an indicator that the bug has been assigned to a developer for work.

This is called a buffer state. The transition between Investigation Complete and Assigned is basically unimportant. What is important is that this transition says is that you are planning to fix the bug. Now, what it doesn’t tell you is what release you are planning on fixing it in. Using TFS you can simply re-assign the bug to an appropriate future iteration. The assigned state also tells users that no one has started working on the bug yet. Additionally this state provides feedback to the CCB about what work is scheduled to be done but hasn’t started yet so the CCB doesn’t accept more bugs than can be reasonably fixed.

Active

This state indicates that the bug is actually being fixed. No bug must ever move to this state without having an accompanying test case.

This state isn’t as important as the transition to this state. The time between the bug being assigned and the bug being active is lag time and is therefore waste.

One of the activities that should be performed during this phase is the creation of a unit test to verify the existence of the bug in code. This will help create a large library of unit tests which target previous bugs which makes regression tests easier to run.

In this part I’ve started to describe what happens in each state and what information you can gather from each state. In the next post I will round out the steps in this particular process and then I’ll move on to discussing the benefits of the metrics. Note that this process covers virtually every type of metric you may need (there are a few it doesn’t cover but I’ll talk about that in the next post). Note also that you can pick and choose which states you want. For example, if you’re reasonably confident that you don’t have bugs sitting around for a long period of time, then maybe the early buffer states aren’t needed. Or if you aren’t sure where the problem is occurring, use broad categories and introduce additional steps to pinpoint a problem as needed.

Stay tuned for the next post!

Any and all comments are appreciated – especially if you think I got something wrong.

This part lays the groundwork for practical process improvement.

At this point you’ve made one of two decisions – you are going to implement process for the first time (your current process is referred to as chaos) or you are going to improve your existing process. Process for process improvements sake is a poor reason to change or implement a process. So, what are you trying to improve (a.k.a. where are you now)? Before you know where you want to go you have to have some type of baseline to measure your progress against. Implementing a process is all about small, measured steps – try to do everything at once and you are guaranteed to fail.

You want to work in small iterations in order to improve your process – even if you are working on a formal (or waterfall) based project. Why small iterations? Did I just tell you to change to an agile development process? Well, no, not exactly. Short iterations allow for rapid feedback and the ability to measure and introduce change quickly and at well defined points. This does not mean you have to start using XP or Crystal or Scrum or any of a dozen other agile methodologies. And these methodologies don’t work particularly well in every situation – especially if you are working on a much more structured project such as a life-critical system.

The goal here is not so much to measure velocity (although that in itself is highly useful), unless that’s an area you want to improve on which I’ll talk about in a much later part of the series, as to be able to introduce a change and gauge its effectiveness.

The other thing that is critical is metrics. If you’re making changes because you think you need to make a change (aka “the gut feeling reason”) then you aren’t going to be able to prove whether the change helped you or not except annecdotally. While we as developers don’t mind this, management really does. They don’t want to plow money into process changes without knowing the benefit they are getting for their dime. So let’s give them what they want and also help ourselves in the process. Don’t make any changes without having a baseline – ever. In addition, when you are doing a retrospective on whether the change worked or not, do it by comparing metrics (as well as gut feeling) to determine the effectiveness.

Next, pick and choose what you want to change (aka get the low hanging fruit first). Choose items that are more simple to change and are easier to measure. In light of this, I’m going to start off this series with simple areas (or at least, in what our experiences with customers have taught us are simple areas) and move to more difficult areas later on.

In preparation for the next part of this series, the first area I’m going to walk you through improving is the bug handling process. This doesn’t cover how to reduce the number of bugs you get initially (well, it does in part but that isn’t the focus) but how you handle the bugs in your process. As I mentioned before, you need baseline metrics to determine the effectiveness of the changes. To begin with, spend the next week or two and measure how long it takes from when you receive the bug (the date and time – to the hour) to when the bug fix is released into production.

In the next part I’ll talk about how to reduce that time frame (obviously this fix will only be important if you feel that it takes a large period of time to implement the fix and that your customers are unhappy) and what you should be doing at each stage.

Branch by Quality

Figure 1 – Branch by Quality, basic pattern

This pattern is the ultimate in flexibility – but can also get very complex. It allows you to handle unique situations and respond to changes very quickly. It can be used for supporting multiple versions of an application or for more flexibility can be combined with Branch By Release to provide even more options for supporting multiple versions. Additionally you don’t have to manage quite as many branches as with other branching strategies. However, it does mean that the management of each branch may get complicated. It allows for branching for isolation, hotfixes and the easy promotion of code from dev to test to production.

The basic structure as shown here involves three branches (or code at various qualities) – Dev which is code that is in constant change and is therefore referred to as “soft” code, QAwhich is code that changes very infrequently (no active development is ever done on this branch) and is considered more firm than the code on the Dev branch and finally Prod which contains code which has been released to production (development is never done on the Prod branch). The red lines represent the branch hierarchy. In this case Prod is the parent of QAwhich is the parent of Dev. In reality the branching can be done in any way which creates a relationship between Prod and QA and QA and Dev. Using TFS you can branch QA to Prod and QA to Dev or Prod to QA to Dev or Dev to QA to Prod to create this relationship. The only thing this effects is the default selection of drop down boxes in the merge wizard.

The reason why we refer to this pattern as Branch By Quality is because of the quality of each branch. Dev is potentially testable, QA is potentially releasable and Prod is released code. Each branch must maintaint this quality. Another hallmark of this pattern is a single path to production. Branch by Release has multiple paths to production – each new branch is a new path to production. With this pattern, builds for production ALWAYS come out of the QA branch (not the Prod branch as you may have assumed).

The process for releasing code is to promote it from Dev to QA, build the QA branch and test the compiled assemblies. If all tests pass, you do NOT rebuild the code – you simply deploy the already compiled assemblies. Then you promote the code from QA to Prod as a safekeeping. Note also the pairs of green lines which represent merges. As before, always Merge down, Copy Up to promote the smoothest transition possible.

Dealing with Bugs

Let’s look at a simple example where code has been promoted from Dev to QA (so the users can perform User Acceptance Tests (UAT)) and development on the next release has begun on the Dev branch. In this situation, assume a bug has been found by users during the UAT and needs to be fixed. Figure 2 shows this scenario.

Figure 2 – Bug in Test

Unlike Branch by Release, you cannot just make a fix on the testing branch because code on that branch is always potentially releasable which means that making changes to that branch can potentially cause problems. To avoid this you take a branch for isolation – in this case it follows almost the same pattern as performing hotfixes (discussed next) – from the QA branch. You can’t branch from Dev because if you do there is no direct way to merge from the isolation branch to the QA branch (you can do a baseless merge but in this case it causes more problems than it solves). So, take a branch from QA (at the point you promoted the code from Dev to QA and labeled the code on the QA branch) and fix the code on this branch. Perform testing on this branch and when it is ready for UAT, promote it to the QA branch. When UAT is complete and the fix verified, merge the QA branch down to the Dev branch. This ensures that the fix makes it back into the next version of the code.

This is a fairly simple example but they will get more complicated as I add to this strategy. I will do this in a new post in the near future.