Chaos and SCRUM

Wreckage of a downed British aeroplane, originally uploaded by drakegoodman.

Josh Nankivel at pmstudent.com asked me to write about some of the challenges and obstacles that we face in our environment. In order to do that I should explain a little about that environment. Over the past 36 months we have transformed ourselves from being a software development team in crisis to a strong team that is focused on delivering on it's commitments. In 2005 we determined that the current approach to writing software wasn't working. We were the poster child for "garage-band" style of software development, having inherited 1 million+ lines of classic (poorly written) ASP code [aka 1MLOC - I'll let you guess what the "C" stands for], and having very little recognition from corporate management that anything was wrong with the way things had been done. When I came to lead the group in early 2007, I had put in countless hours trying to lead the business toward implementing a PMO. While we actively maintain the 1MLOC, we have been working closely with a third party partner to replace the application. In parallel, we have a number of other small LOB applications that we have written using accepted best practices and design patterns. Needless to say, this means that there is a significant amount of churn and chaos when it comes to determining what's going to be worked on and by whom. Enter SCRUM.

In early 2008 I approached the IT leadership and explained that an iterative and incremental approach to developing software was far superior, in our environment, to the code-and-fix triage approach to project management that had been promoted in the past and that this would be well suited to supporting the behemoth 1MLOC application while still allowing us to be agile enough to respond to the business' needs in a timely manner. They bit.

So, what is our environment like, how do we handle some of the obvious gotchas, and what are the advantages/disadvantages to how we do business? Well first, I'll say that this is what works for us, in our environment. I can't speak to your situation, although if I can help, please drop me a line. Well, we currently have over a dozen product backlogs that are somewhere in the development/grooming workflow. We have roughly half that many Product Owners [none have been "officially trained", all have gone or are pending admission to the school of hard knocks] and the expectations of what it means to be a Product Owner are communicated loudly and often. Some gravitate toward the role, others are a little thicker and require some additional coercion encouragement. Regardless, we do the classic planning poker estimating that I described here and then, based upon our standing team velocity we let the Product Owner how much of the backlog we will be able to knock out during the next sprint. At that point, for any one of a couple reasons [the Product Owner needs more functionality than can be delivered in a single sprint/the team has some groundwork to lay in one sprint in order to deliver the key business deliverable in a second or third sprint/etc.], the team may decide that they need to deliver the next increment of work over several concurrent sprints. In this case, the backlog of backlogs is queried and the next product owner is queried for budget and schedule requirements [we may even look at the current state of that PO's product backlog] and a reasonable determination is made [by my boss and the IT Steering Committee] whether it's acceptable to the business for the subsequent development to 1) be delayed 2) be preempted or 3) be split [deliver Sprint 1-Product A, then Sprint 1-Product B, then Sprint 2-Product B]...from the team's point of view, this decision is somewhat irrelevant as long as the sprint backlog is ready to be loaded into the SCRUM Dashboard [which is integrated into the Microsoft Team Foundation Server - our source control and overall ALM solution]. Once the tasks are entered into the Scrum Dashboard (which is a lot like SCRUMY except that it's tied to coding tasks, time tracking, check-ins, build events, etc.) the team tracks their own progress daily which automatically updates the Sprint Burndown, Product Burndown, BurnUp, and a hundred other reports that are hosted in the Microsoft SQL Server Reporting Services site that comes out of the box. The daily stand-up meetings are usually run by the team, as the ScrumMaster I sit in on them regularly; however, my prime directive is really just to make sure that they're happening and that there are no discovered impediments that are/have not been communicated to me that come up as an outcome of the meetings. We have a weekly code review meeting for the whole team to see where we are, and we usually use this meeting to demonstrate the application state to the product owner. As part of our development team, we expect for the business to provide testing and QA resources so that we can be sure that the feedback loops are as short as possible.

So, you ask, what are the gotchas in this environment? That's an easy list to start, but a very difficult one to call complete...so here's my "easy list"...the hard ones will quickly become evident in your environment.
1) Training Product Owners - especially in a "Command and Control" environment, it's difficult to convey the concept and importance of servant leadership. Similarly, it's hard to get "busy" business folks to commit their time to things that are commonly deemed to be unproductive

2) Training Product Owners- getting them to understand the concepts of story points, estimating, life without requirements documentation and use cases, life without earned value, and ultimately the fact that the ROI and the "fit for purpose" functionality is their #1 responsibility
Death by meeting - herding all of those backlogs, training all of those Product Owners, daily stand-ups, retrospectives, code reviews, "demo days", Planning Meetings, not to mention the other "normal" meetings that one usually gets pulled into [weekly staff meetings, employee meetings, annual reviews, etc, etc, etc]...leads to about 80% of your time being tied up in one meeting or another...it really is hard to do a lot more than manage your schedule and check your business card to make sure it matches your driver's license
3) Leadership & Management - if your management doesn't back you up and get out of your way, you will be tripping over one another. They MUST support you and then get out of your way. The proof is in the pudding...deliver a few times and they'll get it.
4) Tools & Co-location - We are one team, in one location. We tried the distributed team thing for a while and, to be honest, the requirements for the team to be available to meet to review something on very short notice makes co-location extremely difficult. Likewise, much like there isn't really a "SCRUMBOK" because SCRUM is prescriptive as far as Engineering Practices, there are hundreds [if not thousands] of electronic tools for the team to use...a definite gotcha is the temptation to jump into the midst of the tools arena before you really understand the process that needs to be supported by a tool. We found TFS and SCRUM Dashboard....it's what works for us....that means squat to you....make your process work analog, and then digitize it....I can't say this any more clearly...learn to love the whiteboard, post-it, and index cards...worry about the business now and worry about the rest later [it might actually take care of itself].



OK. So there's a start to a list...far from comprehensive, or even thorough. That list could go on for a while. So, to wrap up, you're probably interested in understanding what are some of the pros and cons?

PRO: We deliver working software, and we do it pretty frequently. This means that our Product Owners are pretty happy. Happy PO = Happy Business = Job Stability = Happy Employees = Better Software
PRO: We get to deliver at a sustainable pace and we LOVE what we do. I hope that the death marches of 2006 are gone for good. We value people over process. Happy Employees = Better Software = More Projects = Job Stability = Happy Employees
PRO: We get to be honest and transparent. With the PO on the team, there are no secrets about something taking longer than planned, or something not working as expected. The PO and the team get to figure out how to work thorough these challenges rather than worrying about doing damage control.
CON: Estimating is really difficult, especially at first. Then, to make matters worse, budgeting is even more vague. For some organizations, this may be too much to overcome. I hate budgeting season...to ask me how much something is going to cost is to ask me for a complete and estimated product backlog, which isn't terribly different than a preliminary project plan and schedule. Ugh. Budgeting Sucks.
CON: If you are a "Command and Control", "Do first, ask questions later", "take the bull by the horns" kind of manager, you will really struggle with Agile. The CORE of Agile is in empowering the team, transferring that control to them, and settling into a support role....for a lot of folks, this will be impossible.