While moderating a panel discussion recently, one of the first questions after the introductions was asking to explain what was meant by allowing a team to fail. The important understanding that I think was misunderstood is that failure is not meant to allow releases or products fail, but to generate learning opportunities to ensure ultimate success.

Saving a company is an extreme example of what learning often is trying to avoid. Roger Ehernberg has a thoughtful post describing the lessons learned from the recent closure of his company, Monitor 110. In it, he acknowledges his failure to fail as a major contributing factor to the company’s downfall:

We talked about “release early/release often,” but were scared of looking like idiots in front of major Wall Street and hedge fund clients. Is it better to wait a bit before releasing to have a more compelling product or to begin getting feedback on a less impressive offering? We chose #1; in retrospect I think we should have chosen #2. By choosing to wait we lost our intimacy with the customer falling into the classic trap of pursuing a “science project,” not building a commercially salable product. Dumb.

We should have gotten it out there, been kicked in the head by tough customers, and iterated like crazy to address their needs. Woulda, shoulda, coulda. Didn’t.

Don’t wait until you’re writing the post-mortem on your defunct company /product/ team/ release to realize the power of early failures. In contrast, Dean Leffingwell advocates the “fail soft” option:

Agile master Alex Yakima once commented while looking at an upcoming release plan and the few remaining iterations, “nope, not enough times to fail.“ So we shortened the iterations and made the release on time.

Creating opportunities to fail allows people, teams and organizations to speculate, explore and adapt while they can still make adjustments that lead to ultimate success.

I talked at a high level in my Measure Outputs post about the types of measures that are useful. That post stated broadly that input measures do little to help improve, control or predict performance. In this post, I will expand on that concept to cover in more depth what types of measures focus on output and help drive process improvement. 

Perhaps the best known framework for creating measures is the balanced score card. This sets up categories of metrics that support an organizations strategies and vision. From the Balanced Score Card Institute, this framework is shown to the right. I find it useful to apply it to business processes as well as overall strategy. Even though processes are just one component of the model, they ultimately support part of a business’ strategy. A framework helps to ensure we considering measures that contribute to the overall strategies and goals the process supports.

Once a framework is chosen, the next thing to consider is the customers and stakeholders that the process serves and what they value. This identification is important to ensure we know the objectives that the process is to support. If the stakeholders or customers tell us they want high performance, personalized, niche process (a Rolls) the measures will be different than if they want a quality, efficient, repeatable process (a Toyota.)

The next question is what measures are valuable. I continue to see the same types of measures used across process improvement projects. For each category of the framework, there are common recurring measures that are useful:

Process: There are three key measures that are critical to almost all processes: Productivity, Quality and Cycle Time.

Productivity: Always measured as Output / Input. I once saw a company that measured productivity as the % of time spent on coding. I get the thought, but it is only half the equation and, frankly, the less important half.

Output: Products, features, backlog or other work units should have standard “story point” or relative size estimates assigned. This normalizes the output across different work effort or output value. It also helps to facilitate planning so that resources can be assigned to work based on size and with pricing so that appropriate prices are assigned per output unit.

Input: Measures the capacity of available resources used to create the outputs. Be comprehensive and don’t give resources credit for working off task. If you are paying for the resources, they should count against output even if not always focused directly on it. Also, you should be able to factor out overtime from this component of the measure as it is not sustainable capacity.

In Agile development, Velocity or BurnDown is a Productivity measure. You may say it only shows output, but since capacity is held steady by a dedicated team, only the output need be measured. 

Quality: Repeat after me… “Never use defects found as a quality measure.” You want to find defects. The way to find them is to ensure quality is built-in through proactive testing.  The output measure for quality is the testing done to support acceptance.

Tests planned, run and passed is a proactive measure that attempts to ensure that the quality of a process is planned and built into the delivery. The testing process defines acceptance criteria and should be built into construction of deliverables and happen at various stages with the earliest coming during early requirements and build. The only rules here are from test driven development: the test should only test one criteria, it should only fail for that criteria, and it should be the only test that fails for that criteria.

OK, one exception to the mantra: Measure defects found by the customer. Track these with a vengeance and apply process and measures to them to ensure they are adequately corrected and customers stay satisfied.

Cycle Time: From iSixSigma:

Cycle time is the total time from the beginning to the end of your process, as defined by you and your customer. Cycle time includes process time and time spent waiting to take the next action.

So, cycle time is a measure of time from request to delivery or concept to cash. It includes not just how fast development happens, but the time it takes to capture, prioritize, design, develop, test and deploy functionality and all the time in between. Ensure your cycle time measure is expansive or you will sub-optimize the overall process. Also, realize that not all requests are created equally. Ensure those that have the most value get priority treatment. The features in your backlog that are low priority may deserve to grow old on the backlog and eventually drop off.  Or perhaps there is value in tracking cycle time for bug fixes vs enhancements vs new features.

As long as your prioritization process is fast and exiting the backlog requires deployment, the average age your backlog is an easy proxy for cycle time.

My First Law of Development is about maximizing productivity while minimizing cycle time if you’d like a deeper read.

Some organizations use On-time Delivery as a proxy for cycle time. While this is an OK measure for customer satisfaction, it does little to report the health of a process. If important to your customers, use it as a customer measure, but also use cycle time as a process measure.

Customer: Often, quality is used as a proxy for customer measures. This can have the unintended effect of missing other critical voice of customer measures. The best way to ensure the customer is satisfied, is to ask them. Simple surveys or votes are perhaps the best way to capture this measure. Ask customers about what is important to them. There are many online survey tools that will allow you to ask relevant questions from your customers and track trends over time. Or, use a net promptor score by simply asking “How likely are you to recommend our product / service.”

Mike Griffiths has an eloquent sponsor/ customer tracking approach that asks for a  vote of confidence or satisfaction. In aggregate, this provides a customer-focused index over time. I also like the emoticons in his customer chart.

Learning: Learning should measure the capabilities of the team members. A matrix of who can use what tools and develop what products or product components with a level of expertise assigned is a good starting point for tracking this measure. 
% coverage becomes a good aggregation of this measure. Be sure to include current skills as well as upcoming skills and both soft and hard skills in the matrix.
(From The Lean Six Sigma Pocket Toolbook: A Quick Reference Guide to 100 Tools for Improving Quality and Speed, Mike George, et al)

Financial: As high-level financial measures go, I have a pre-disposition to ROIC. It balances profitability with the investment costs required to generate it - and is a topic for another post. Most processes will not require this comprehensive of a measure, so simple cost tracking usually suffices. This can usually be further simplified to time tracking since development is largely made up of salary costs. Make sure you at least consider equipment and other capital and marketing costs before you settle in on time tracking as your financial measure. Normalizing financial tracking to earned value or planned spend helps to show tolerances of spend on a project (and display productivity with the financial measures.)

PS: I have intentionally tried to make this post very visual. The key to making measures useful is to make them easily understood and highly visible and public.

When launching any improvement initiative, the first step is typically forming a team that can get down to the work of getting the job done. To get things off on the right step, I cannot emphasize enough the importance of a team charter. This is true for new agile teams, quality teams and any improvement initiatives when more than one person is needed to get results.

Why, because creating a charter makes sure that proper thought is devoted to the define the problem, set goals, scope and approach, and identify the people and other resources that are necessary to be successful. This simple step forces initial action that sets teams up for long term success.

The format for a charter should be quite simple. Typically, they are written with the following headings:

• Team Name
• Problem Statement/Business Case (why team created)
• Scope (what will and will not be addressed by team)
• Team’s Customers and their needs (how will the team define success)
• Team Leader (and champion/sponsor if different)
• Team Members (with expertise and time commitments for each)
• Specific Objectives and How Measured (updated monthly or quarterly)
• Plan (initial team duration, activity backlog/tasks, milestones, meeting schedule, stakeholders and communication expectations – updated as needed)
• Resource needs (other people, budget, tools, etc.)

The charter creation process should start by having sponsors designate an initial team for the initiative. This team builds each of the above bullet points collaboratively and works with the sponsor and others as needed to gain agreement. Having a team charter at the end of this exercise documents and communicates a team’s purpose. More importantly, the process of creating and maintaining the charter builds understanding of and broad commitment to that purpose.

Consulting organizations often require signoff on deliverables to ensure that the client acknowledges that contractual obligations are met. Across organizational lines, these signatures, serve mostly a legal purpose. I often wondered about the use of signoff on deliverables inside of an organization where there is no external relationship. I’ve seen some development shops average as many as 15 or 20 of these sign-offs a day.

The intent of getting signatures on product deliverables, of course, is to ensure that the proper attention was given to the creation of the product. The cynic in me says it is to fix blame. In reality, I believe it points to a need to compensate for poor collaboration and broken processes. Once the decision has passed on whether to create the product or deliverable and allocate resources, forcing signatures at various check points in a process serves as an attempt to ensure everyone is kept informed and had an opportunity to participate. Of course, the problem with this is that at the point of sign-off, the ability to give meaningful input that can help formulate an improved solution is usually past. Unless those on the signature line are involved when needed in the discussions and decisions that get built, the signoff sheet at the end of the creation is likely to only be only a rubber stamping exercise.

What is a better solution? Ensure stakeholders get involved in the decision of whether or not a deliverable is of high enough priority to start. Once that decision is made, create a process that demands collaboration. Make one person (the product owner) responsible for representing and getting input across as many interests as they can understand. Involve those on the signature lines in planning and closeout sessions on a set basis where they can see the product or deliverables as they are built. If some input is of limited nature, start with that at these meetings or have the product owner represent the interest. Ensure the collaboration continues between these formalized checkpoints as well. By focusing on a collaborative process, everyone’s input can have a positive impact on the fit and quality of the product as it is created.

If your organization is signoff heavy, it’s probably time to find out what systemic problems need fixed.  

At the risk of becoming overly link heavy in recent posts, I want to highlight two sources for trouble-shooting scrum implementations.  The first is from Mike Griffiths’ Introducing Agile methods to Organizations: Mistakes to Avoid” (part 1, part 2, part 3). These meaty articles share plenty of wisdom.

The second is from the ScrumAlliance. They have a wiki to share scrum experience. In a recent addition, they have succinctly started to expand on a Toward A Catalog Of Scrum Smells by Mike Cohn:

Catalog of Scrum Smells

• Loss of Rhythm 
• Talking Chickens 
• Missing Pigs 
• Status not clear from Daily Scrums
• Lack of Progress Part One: Failing in Backlog Management
• Persistent Signatures
• ScrumMaster Assigns Work
• The Daily Scrum is For the ScrumMaster
• Specialized Job Roles
• Testers will not integrate with Team
• Reluctance to estimate Backlog Items
• Is It Really Done
• Nothing Ever Changes Around Here
• No One Wants to Attend Retrospectives
• Executive Pressure
• Missing Sprint Commitment
• Technical Debt
• Not Acting Like a Team 
• No Engineering Practices
• Gorilla in the Room

Let me know if you have any others to share.

Mishkin Berteig has a great post up at his blog titled Measuring Process Improvements - Cycle Time. In it, he eloquently details why development teams should care about and manage cycle time. It complements nicely my First Law of Development and I encourage giving it a deep read.

Megan Sumrell has 10 nice tips up on working with remote teams. I like that these tips include a softer side to the remote challenge. To them, I add a few fundamentals:

1. Keep an open IM channel. One to one is great, but a team chat room is even better. Create a room that the team logs into at the start of their work day. This creates virtual co-location and a place where everyone can come to work. It also creates an “across the desk” ability for collaboration.
2. Don’t forget the phone. IM is great for simple communication. The phone is still the next best thing to being there when topics are more complex. If cost is an issue, get the team on a free Internet voice service like Skype.
3. Set up systems for measuring outputs. It is one thing to say you focus on it. It is another to effectively tack and hold everyone, in-house or remote, accountable to actual measures. I recommend using tracking tools like Rally, VersionOne, etc for remote teams. They can organize and track agile development and keep everyone up to date easily. On the down side, they are not as visual as task boards.
4. Remember that when you are remote, manners still matter. Even simple IM hellos at the start of a day lend virtual humanity. Tone does not come across well in text, so be sure to check for it before blasting emails and IMs. What’s in your head may not be what comes across on paper.
5. It is not just about coordinating tasks. Take time for leadership and social interaction with remote members as well. They need to know what is happening in the broader company. They want personal touch when they have concerns and questions. Make virtual time for this interaction as well.

There is a great post up yesterday from Rohit Bhargava, author of Personality Not Included: Why Companies Lose Their Authenticity And How Great Brands Get it Back. In it, he talks about Brand Personality. I mentioned in Tell Me a Story, that I think product managers and owners would do good to give their products personalities early in the visioning process. As I read through his post, I thought perhaps giving them a brand is an easier way to think about this concept. Are you creating a Cadillac or a Chevy? A Blackberry or an iPhone? Even though it’s once removed, it may be more relevant to many people.

I spent the last 2 days at the BusinessAnalystWorld Symposium Conference in Denver. My key takeaways were:

1. Even as agile awareness and interest grows in organizations, there are still many cultural hurdles and misconceptions to overcome before it becomes widely adopted
2. Grass roots on small projects is still the most common method of introducing agile (at least in larger organizations)
3. Remarkable people accomplish heroic things everyday at their work, often without even recognizing how remarkable they are

For those that asked for slides to my presentation at the conference you can download them at this link: Agile Thoughts – Exploring the Philosophy and Mechanics that Make Agile Work. I appreciate the participation and feedback from everyone in attendance. Please leave me a comment if you have any additional feedback from the presentation, the panel, or if I can help clarify anything we discussed there or have thought of any new questions.

I also thank the great panelists we had that shared their thoughts and answered questions at The Role of the BA in an Agile Environment Panel. Their companies are lucky to have them at the helm of their agile transitions. I endorse each of them: Grace Goerdel, Product Manager from Graebel, Inc.; Greg Haynie, Vice President Application and Database Development, Health Grades, Inc.; Alicia Yanik, Project Manager and Advanced Scrum Master; and Von Rhea, Director, Corporate Express.  Also, thanks to Pete Behrens for helping to fill a last minute need.

BA World put on a first class conference with a great agenda and great attendees. I look forward to their return next year. Congratulations to Adam Kahn and his team from BusinessAnalystWorld and for an outstanding performance.

I was having a conversation with a client last week about things to consider when moving to an agile development organization.  Here’s some notes from the conversation:

• Create product and customer focus: Design around products and processes necessary to fulfill current and future customer/market requirements.
• Enable agile development: As product development is transitioned to an agile approach, the organization must be structured to support the new teams and create product focused team effectiveness, not task efficiency.
• Create local ownership: Development teams should have ownership of and accountability for the entire development process. Team members should have visibility into the full processes and responsibility for products (not just tasks) and be encouraged and empowered to make improvements. The distance between issue emergence and resolution should be minimized.
• Ensure organizational standards: Despite being autonomous, teams must conform to standards that optimize organizations overall effectiveness and ensure specialty skills are maintained and enhanced across teams.
• Optimize project release management: Development management, the PMO organization and charter system must recognize how agile teams function and plan, prioritize and release work to those teams in a compatible manner that maximizes throughput while minimizing the time to create new functionality. Additionally, measurements should focus on ensuring the development process is working efficiently and effectively to deliver customer-valued results.

This constitutes a pretty good list of guiding principles and goals. Hard work is still needed to apply these to their organization.  Even with the same objectives, every company will need to design their own details, staff new roles and plan and execute their transition. Depending on the organization’s size, this can take a couple months to multiple years to fully implement.