Tuesday, September 3, 2013

Stability vs.value

The hero, his trusty sidekick, and the fair maiden make their way through the dark and eerie forest, muggles seemingly hide behind every tree....it's a familiar yarn, we've seen the scenario played out many times on the big screen. Have you ever felt like you're one of those characters when you go to work? There are days when I'm quite sure that there is a conspiracy afoot and there are days when I'm convinced that whatever it is that I'm working on is going to revolutionize how the world turns. In the end I'm left wondering if I (or at least one of my colleagues) are neurotic and over the top paranoid, or if I/we are justified in our concern. I'll just say that sometimes it really sucks to work for a big company.

Tuesday, November 8, 2011

CrowdSourcing. Tapping the real hidden value in the organization?

When will companies recognize the untapped potential of crowdsourcing to their employees, customers, partners, and/or vendors? If you've ever watched the TV game show called 1 vs. 100 (as well as Who Wants To Be a Millionaire), you'll recognize the reality of crowd-logic. It's not like you would be forced to accept the general consensus when making a business decision; however, using aggregated group knowledge and opinions when making decisions (specifically those that impact the group) seems to me to be a total no-brainer. Because it seems so obvious to me, I'm left wondering why companies don't do it. Thoughts? Please chime in!

Tuesday, November 1, 2011

Scrum tune-up

In my previous post I referenced a conversation with a colleague about a Scrum implementation that hasn't produced the results he expected. His expectations from the outset seem to be primarily focused on productivity, but motivation and drivers for choosing Scrum is for another day. As I was mulling this over I kept coming back to the variations in the Scrum technique and what seem to be gaps in the technical practices that are mandated in other Agile methods. When I compiled the list I was wondering if this list might be of value to someone else. if so, here you go. I also need to make sure that you know that this is by no means a comprehensive list, but just some things that, in my opinion, every team using Scrum (especially in the. NET realm) should be doing.
  • Be SOLID - Learn the S.O.L.I.D. principles. Practice them. Take no prisoners.
  • Get DI - make sure that you and everyone on your team REALLY gets Dependency Injection and uses it where appropriate. If you do it right, you'll thank me later. If you don't you'll probably curse me. If you find yourself cursing me, re-read that.
  • List, List, List - Really, really, really know the product backlog. Know where the Product Owner wants to take the product. Get behind that. Give the team whatever they need to succeed and then get out of the way
  • Be debt free - Perform an HONEST Technical Debt assessment. Identify it, put together a plan to either abandon it, call it legacy (thereby putting in a plan to abandon it), or fix it. Before doing anything else, do something to prevent it from happening again. The best thing to do is the next item.
  • Invest in the team - People are your your greatest asset. Treat them fairly, hold them accountable, expect them to hold each other accountable. The best, and most important way to do this is to foster a safe haven environment, implement code reviews as a means of avoiding technical debt as well as a learning tool. Discourage/punish anyone/anything that violates the safe haven.
  • Get TDD - If your standard development practice does not include unit testing [which dovetails nicely with the SOLID principles], you're missing out on the opportunity to take advantage of the benefits provided by the Single Responsibility Principle or the Liskov Substitution Principle. The idea is pretty simple, write the code once and then test it every time something changes.
  • Get CI - If you already have a Continuous Integration environment, props to you. If you don't and are writing unit tests, don't walk but rather run to get a jenkins or some other CI solution running. Chances are if you've embraced TDD and don't have a CI solution, you're leaving most of the value of TDD on the table.
  • Bigger is better - Make an honest assessment of the team's communications. Look at the tools, protocols, frequency,etc. This is especially important for teams that are not colocated. If your team is relying on any type of asynchronous communication (email, IM, twitter, LCS, etc.) you're penalizing yourself. Preferred modes of communication: 1) face to face 2) telephone or Skype 3) Instant messenger or twitter 4) Email 5) Snail Mail.
  • Embrace Legalism - Follow Scrum by the book for a while in order to figure out where it doesn't work. When you find something that doesn't work for you, try it again the original way, and if that still doesn't work, keep trying until you are absolutely sure that there is no alternative. If you change something, consider yourself warned.
  • Evangelize - Put forward a concerted effort to gain product owner investment. If the person you identify as the Product Owner wants to delegate the responsibility, you have a sales job to do and will likely require some convincing to the "real" PO (not a proxy PO) in order to to take away the cya/escape hatch (aka finger pointing/plausible deniability) that business people are so accustomed to when dealing with software projects.
  • Be a mechanic - Inspect and adapt the technical/engineering practices in order to continue leveraging value. Think outside of the box and try new ways of investing in the skills of the team. For example, if you've never done it, try pair-programming for a couple of sprints to see if it helps or hinders the team's performance.

Scrum is a world-view

I've been thinking about how to help a colleague get his arms around a Scrum implementation that seems to be out of control and rife with what my friend Ken Schwaber refers to as "ScrumButs". During our conversation, I cringed several times when reference was made to their "version" of Scrum as well as their version of "TDD". Yikes. I think that this is a common trap that managers fall into when trying to adapt their organizational culture to become more "Agile". The bottom line is that if you're adopting Scrum because of business value motivators like profitability, product turn rates, cost control, or to compress/reduce feature release windows then I would tell you that your heart is in the wrong place and that you really need to re-assess what the value proposition is for implementing Agile software development methodologies/techniques. In my opinion, being successful at Scrum requires more of a change in personal and managerial philosophy than it does any type of  technical or procedural change. You see, I really struggle when I use the term "methodology" in the same sentence as Scrum because it's not really a methodology...nor is it a technique....framework...technique...bleh. It's all of those and none of them at the same time. Scrum is a philosophy. It's a world-view relating to managerial interaction with those involved in production. If you want to know how to succeed at implementing Scrum, I would recommend that you start with a read of Ken Blanchard's book titled "The Servant Leader". Once you have your heart in the right place, you've put aside the "command and control" mindset and embraced the TPS/Lean concepts of empowering and enabling your staff to make decisions, and resolved to give the power to make those decisions to the team, now you're really ready to tackle the hard stuff like finding, training,and getting on the same page as the Product Owner and starting down the Scrum road. Much like political or religious world-views, If you can't put aside, or your organization isn't structured to let you put aside these fundamental beliefs and opinions about people, roles, and responsibilities, you really need to abandon the idea of implementing Scrum/Lean/Agile until you can come to terms with what it means to lead people versus what it means to manage them.

Wednesday, February 10, 2010

Chaos and SCRUM

Josh Nankivel at pmstudent.com asked me to write about some of the challenges and obstacles that we face in our environment. In order to do that I should explain a little about that environment. Over the past 36 months we have transformed ourselves from being a software development team in crisis to a strong team that is focused on delivering on it's commitments. In 2005 we determined that the current approach to writing software wasn't working. We were the poster child for "garage-band" style of software development, having inherited 1 million+ lines of classic (poorly written) ASP code [aka 1MLOC - I'll let you guess what the "C" stands for], and having very little recognition from corporate management that anything was wrong with the way things had been done. When I came to lead the group in early 2007, I had put in countless hours trying to lead the business toward implementing a PMO. While we actively maintain the 1MLOC, we have been working closely with a third party partner to replace the application. In parallel, we have a number of other small LOB applications that we have written using accepted best practices and design patterns. Needless to say, this means that there is a significant amount of churn and chaos when it comes to determining what's going to be worked on and by whom. Enter SCRUM.

In early 2008 I approached the IT leadership and explained that an iterative and incremental approach to developing software was far superior, in our environment, to the code-and-fix triage approach to project management that had been promoted in the past and that this would be well suited to supporting the behemoth 1MLOC application while still allowing us to be agile enough to respond to the business' needs in a timely manner. They bit.

So, what is our environment like, how do we handle some of the obvious gotchas, and what are the advantages/disadvantages to how we do business? Well first, I'll say that this is what works for us, in our environment. I can't speak to your situation, although if I can help, please drop me a line. Well, we currently have over a dozen product backlogs that are somewhere in the development/grooming workflow. We have roughly half that many Product Owners [none have been "officially trained", all have gone or are pending admission to the school of hard knocks] and the expectations of what it means to be a Product Owner are communicated loudly and often. Some gravitate toward the role, others are a little thicker and require some additional coercion encouragement. Regardless, we do the classic planning poker estimating that I described here and then, based upon our standing team velocity we let the Product Owner how much of the backlog we will be able to knock out during the next sprint. At that point, for any one of a couple reasons [the Product Owner needs more functionality than can be delivered in a single sprint/the team has some groundwork to lay in one sprint in order to deliver the key business deliverable in a second or third sprint/etc.], the team may decide that they need to deliver the next increment of work over several concurrent sprints. In this case, the backlog of backlogs is queried and the next product owner is queried for budget and schedule requirements [we may even look at the current state of that PO's product backlog] and a reasonable determination is made [by my boss and the IT Steering Committee] whether it's acceptable to the business for the subsequent development to 1) be delayed 2) be preempted or 3) be split [deliver Sprint 1-Product A, then Sprint 1-Product B, then Sprint 2-Product B]...from the team's point of view, this decision is somewhat irrelevant as long as the sprint backlog is ready to be loaded into the SCRUM Dashboard [which is integrated into the Microsoft Team Foundation Server - our source control and overall ALM solution]. Once the tasks are entered into the Scrum Dashboard (which is a lot like SCRUMY except that it's tied to coding tasks, time tracking, check-ins, build events, etc.) the team tracks their own progress daily which automatically updates the Sprint Burndown, Product Burndown, BurnUp, and a hundred other reports that are hosted in the Microsoft SQL Server Reporting Services site that comes out of the box. The daily stand-up meetings are usually run by the team, as the ScrumMaster I sit in on them regularly; however, my prime directive is really just to make sure that they're happening and that there are no discovered impediments that are/have not been communicated to me that come up as an outcome of the meetings. We have a weekly code review meeting for the whole team to see where we are, and we usually use this meeting to demonstrate the application state to the product owner. As part of our development team, we expect for the business to provide testing and QA resources so that we can be sure that the feedback loops are as short as possible.

So, you ask, what are the gotchas in this environment? That's an easy list to start, but a very difficult one to call complete...so here's my "easy list"...the hard ones will quickly become evident in your environment.

1) Training Product Owners - especially in a "Command and Control" environment, it's difficult to convey the concept and importance of servant leadership. Similarly, it's hard to get "busy" business folks to commit their time to things that are commonly deemed to be unproductive

2) Training Product Owners- getting them to understand the concepts of story points, estimating, life without requirements documentation and use cases, life without earned value, and ultimately the fact that the ROI and the "fit for purpose" functionality is their #1 responsibility
Death by meeting - herding all of those backlogs, training all of those Product Owners, daily stand-ups, retrospectives, code reviews, "demo days", Planning Meetings, not to mention the other "normal" meetings that one usually gets pulled into [weekly staff meetings, employee meetings, annual reviews, etc, etc, etc]...leads to about 80% of your time being tied up in one meeting or another...it really is hard to do a lot more than manage your schedule and check your business card to make sure it matches your driver's license
3) Leadership & Management - if your management doesn't back you up and get out of your way, you will be tripping over one another. They MUST support you and then get out of your way. The proof is in the pudding...deliver a few times and they'll get it.
4) Tools & Co-location - We are one team, in one location. We tried the distributed team thing for a while and, to be honest, the requirements for the team to be available to meet to review something on very short notice makes co-location extremely difficult. Likewise, much like there isn't really a "SCRUMBOK" because SCRUM is prescriptive as far as Engineering Practices, there are hundreds [if not thousands] of electronic tools for the team to use...a definite gotcha is the temptation to jump into the midst of the tools arena before you really understand the process that needs to be supported by a tool. We found TFS and SCRUM Dashboard....it's what works for us....that means squat to you....make your process work analog, and then digitize it....I can't say this any more clearly...learn to love the whiteboard, post-it, and index cards...worry about the business now and worry about the rest later [it might actually take care of itself].

OK. So there's a start to a list...far from comprehensive, or even thorough. That list could go on for a while. So, to wrap up, you're probably interested in understanding what are some of the pros and cons?

PRO: We deliver working software, and we do it pretty frequently. This means that our Product Owners are pretty happy. Happy PO = Happy Business = Job Stability = Happy Employees = Better Software

PRO: We get to deliver at a sustainable pace and we LOVE what we do. I hope that the death marches of 2006 are gone for good. We value people over process. Happy Employees = Better Software = More Projects = Job Stability = Happy Employees
PRO: We get to be honest and transparent. With the PO on the team, there are no secrets about something taking longer than planned, or something not working as expected. The PO and the team get to figure out how to work thorough these challenges rather than worrying about doing damage control.
CON: Estimating is really difficult, especially at first. Then, to make matters worse, budgeting is even more vague. For some organizations, this may be too much to overcome. I hate budgeting season...to ask me how much something is going to cost is to ask me for a complete and estimated product backlog, which isn't terribly different than a preliminary project plan and schedule. Ugh. Budgeting Sucks.
CON: If you are a "Command and Control", "Do first, ask questions later", "take the bull by the horns" kind of manager, you will really struggle with Agile. The CORE of Agile is in empowering the team, transferring that control to them, and settling into a support role....for a lot of folks, this will be impossible.

Saturday, February 6, 2010

Implementing SCRUM...the basics - part 2

Once again, I published this on pmstudent.com but thought it would be appropriate to post it here as well.

In this post, I explained that SCRUM is not a silver bullet and that there some significant barriers to entry for organizations wanting to adopt SCRUM. In this post, I'm going to introduce Agile estimating, planning poker, fibonacci, and velocity.

Let's say that your organization is ready to go, you've turned command and control on it's ear, you've identified the Product Owner and you're going to be the ScrumMaster. You've put together a team of seasoned developers, a creative designer, and a QA Lead, but the designer and QA Lead have never worked on an Agile project and they also have never worked with this team of developers.

Let's say that the Product Owner has put together a pretty comprehensive list of User Stories [let's say that for argument sake that these have been validated and prioritized and you have verified that they follow the INVEST principles...more on that in another blog post]. What now?
First, you need to work with the "Team" to make sure that they understand what their responsibilities are, this includes the talk about being a self-organizing team, that they're responsible to one another, the Product Owner, and ultimately to the organization to do what they say they're going to do.

The next step is for the SCRUM Team [that's the development team, the Product Owner, and the ScrumMaster] to decide which items in the Product Backlog you're going to deliver in the upcoming Sprint. In order to accomplish this, you need to be able to quantify how much time or effort each item on the Product Backlog will require. This quantification of the Product Backlog is called estimation and is commonly accomplished by having the development team play a game called Planning Poker.

Game playing? Man this SCRUM stuff is wierd. OK, let's talk about Planning Poker.

The idea behind User Stories is to collect the requirements in bite-sized pieces. The size of those bites will inevitably vary. The Agile solution to this problem is to rate each User Story according to relative "size". The best analogy I've heard is to compare the stories to dog breed...compared to the other stories, if the current User Story a Toy Chihuahua or a Great Dane? Where in the middle...Jack Russell Terrier? Labrador Retriever? Once each person has an idea for the relative "size" of the item, each person has to play their hand, and the group consensus wins...highest and lowest have to explain why they think the feature is larger or smaller than the rest of the team. When I do this, as the ScrumMaster, I intentionally don't play a hand because I don't think I should influence the team's collective insight into how big or small a feature is. To each their own. The planning poker deck we use was purchased from Mountain Goat Software and was well worth the few bucks it cost. There are online versions and of course with a pair of scissors, some index cards, some markers, and a little free time you can make your own. The important part is that you understand the notion of the relative scale of accuracy and the variability that is introduced as a feature gets larger. You're probably familiar with the Fibonacci series of numbers. We use this series of numbers to incrementally scale the "size" of a feature. Why do this at all you ask? Well, if I ask you to estimate in inches, the distance from the tip of your index finger, where it currently is, to the tip of your nose, you will probably be able to estimate with better than 85% accuracy. Now if I ask you to estimate using the same criteria the distance from the tip of your index finger to the middle of the letter "O" on the nearest Stop Sign [depending where you are right now] you might be able to estimate with a significantly smaller degree of accuracy, and finally if I ask you to estimate, in inches, the number of inches from the tip of your index finger to the tip of the Empire State Building in New York City, you are likely to have an accuracy approaching 0%. With this in mind, we use the Fibonacci sequence because as a feature increases in size, our inherent ability to accurately estimate it's size diminishes. After all of the items on the backlog are estimated, the next step is to determine the Sprint Backlog...the estimated subset of product features that will be addressed in the Sprint. We had a hard time understanding how much we could bite off in our first few sprints [in other works, how many "story points"...or Fibonacci values we could consistently complete in a sprint].
In order to get around this, we took two decisive steps. First we committed to micro (one week) sprints and second we intentionally rigged the sprint backlog with items that we knew were small enough to complete in a week, yet big enough to validate [one way or another] our estimates. This process allows us to determine our "velocity". Velocity is the average rate at which a team can complete story points for a unit of time. Once we were able to determine out weekly velocity, we were able to extrapolate our bi-weekly velocity. Once we were able to determine the velocity for a "normal" sprint, we have become increasingly confident in our ability to deliver on that velocity. I will put in a big caveat...the velocity statistic is only a general guide, and as you adopt SCRUM, the accuracy of the velocity is only as good as your ability to limit the variability of external change within the team. In other words, if you determine the velocity for a specific team, that velocity will be affected by making changes to the team, and even in some cases by changes that the team may make from within.

For more information on these topics, there is a great book on this topic written by Mike Cohn called "Agile Estimating & Planning". I highly recommend it.

Monday, December 22, 2008

Introduction to SCRUM - Part 1

I wrote this post originally on pmstudent.com and thought I would post it here as well because not everyone knows about that site.

I started to write a response to a recent blog post about the effect that a slowing economy will have on the adoption of Agile project management methodologies when I realized from the comments on that site that there may be some folks here that are not yet familiar with Agile/SCRUM. I'm going to speak specifically about SCRUM since it's currently the most popular "flavor" of Agile. If you're already familiar with SCRUM, then this is just another SCRUM 101...if you're new to the idea, then I recommend that you read this before continuing...

If you've made it here, then you should understand what a departure these four tenets are, for most organizations, from the way they've always managed solving technical problems. As we begin discussing SCRUM, I'll be painting the picture from the standpoint of a business that is using a single small team doing internal, LOB application development. There are minor variations of this content for those producing commercial products, and for instances where you might be a cog in a much larger machine. I hope to cover these scenarios in depth in future posts. That said, you need to understand that SCRUM lays out a few, simple roles:

  • A "Product Owner" [someone from "the business"] owns ultimate responsibility for the behaviors and functionality of the system as well as the schedule of what features will be delivered and when.
  • A "ScrumMaster" is responsible for removing any barriers - organizational, technical, inter-personal, etc. that are/might be preventing the team from delivering on it's commitments.
  • A "Team" consists of everyone involved in delivery...that may include architects, developers, designers, business analysts, DBA's, QA pros, and perhaps marketing, training developers, technical writers, etc. Popular guidance suggests that a team should consist of 5-9, although it can be larger or smaller, as always, you should really have a valid reason for doing it.

and a few ground rules:

  • All meetings are Time-Boxed.
  • Development is organized into fixed duration, completely encapusulated, development windows [typically 30 days] called a Sprint.
  • The Product Owner is responsible for collecting, capturing, prioritizing, and communicating the system requirements [called User Stories]
  • Each Sprint consists of four major meeetings...
    • Sprint Planning Meeting #1 is used by the team to review the Product Backlog and discuss the desired outcome of the Sprint.
    • Sprint Planning Meeting #2 is used by the team, having used the interim between the previous meeting to estimate the size of each item on the Product Backlog, to choose how many of the top items in the list can be completely delivered in the next 30 days. This subset of the Product Backlog is called the Sprint Backlog.
    • The "Daily Standup" is a daily, 15-minute meeting used by the team to report progress since the last meeting, report what each person will be working on before the next meeting, and finally to indicate what, if any, impediments there are to their progress.
    • The Retrospective meeting happens at the end of the Sprint and is intended as an opportunity for the Team, ScrumMaster, and Product Owner to candidly discuss what went right and what went wrong during the Sprint, and then to discuss and assign specific actions in order to adapt the process for the next Sprint.
  • The "Team" is self-organizing and is responsible, as a unit, for understanding and delivering the product envisioned by the Product Owner.

So, now you should have a general feel for who's involved in the effort and what each person's responsibilities are.

Now, the first thing you need to do to understand SCRUM is to set aside your feelings and understanding of organizational structures, reporting structures, and anything else that smells of command and control. SCRUM requires a new organizational philosophy that typically manifests itself as a shift from the scenario where management controls production to one where the workers are empowered to innovate and the role of management is to alleviate anything that slows or prevents production. Naturally, worker bees are usually OK with this because they are positively affected and empowered by this shift, manager bees struggle with it because it takes their "power" away and expects them step into a role of "Servant-Leader". If your ogranization [specifically management] can't adapt to this shift and you still try to "make" SCRUM work, you're either going to have severe managerial attrition, failed projects across the board, or best case will be marginal successes with burned-out developers who feel caught in the middle/terrorized by the PM. You should also understand that SCRUM mandates very little, in fact, the SCRUM methodology is extremely lightweight and does not provide specific proscription for how you might implement it in your organization. The result of this has been mis-application of the principles and philosophies behind SCRUM, but more on that in a later blog post.

The second big mindshift you need to understand is the Agile approach to requirements. Contrary to the "waterfall" and "modified waterfall" approach to development, the collection of detailed requirements in SCRUM is deferred until the last moment. In a "traditional" or "waterfall" development process, the requirements are collected in detail at the beginning of the process, and then the development team goes off into a closet and works to deliver according to the Statement of Work. In SCRUM, a User Story is a component of work and represents a reminder for the developer to have a detailed requirements collection conversation with the Product Owner regarding the specific piece of functionality. I'm sure that right now you're saying "this can't work", "my developers are too lazy to do that", and/or "what about architecture", and a thousand other questions. I'll address these in future blog posts as well [this things is already approaching book status]. The benefit and net result of this approach to requirements is that the "customer" [Product Owner] can change the "scope" of the project [by manipulating the Product Backlog] in the midst of the project. In truth, and practice, this type of "change" is actually encouraged as long as the change to the Product Backlog is not made to the Sprint Backlog [although this can also be accepted as long as the Product Owner chooses an item of comparable size that can be moved from the Sprint Backlog to the Product Backlog and that the item being removed is not already in progress]. Because SCRUM welcomes scope changes, the product that is ultimately delivered should be an exact match to the customer's business requirement.

The first thing to do when implementing SCRUM is to evaluate your organization's willingness, capacity, and commitment to making the change. Second, make the same assessment about your willingness to be an organizational change agent because whoever adopts SCRUM will have to be prepared to make a firm and consistent stand that you CAN, with some effort, produce a ballpark estimate of time and cost to deliver a project and you CANNOT produce a detailed cost and schedule estimate, Gantt Chart, Project Plan, and the plethora of other documentation that a traditional waterfall approach mandates. This is a particularly difficult concept to endorse when you're accustomed to managers, customers, executives, etc. that demand to know the "Not To Exceed" costs and schedule up front. This "cannot" stems from the fact that, with a new team at least, you cannot estimate velocity [again, that'll be in the next post] and as such you cannot determine how many Sprints it will take to work through the complete Product Backlog.

SCRUM and Agile are not silver bullets, and adoption of them requires a serious and significant commitment from the organization to embrace change. It's not for everyone, but for those that can embrace change, do believe that it's better to deliver smaller functional bites of an application than to deliver a monolithic application, and are willing to turn their organizational structure on it's ear....then you're on the right path...please stay tuned for more.