Member Q&A – Measuring the Success of a Project
I am currently looking into some common key performance indicators or metrics which I can use to measure the following:
- success of the project deployed
- effectiveness of the project team leader
- effectiveness of the project team members
Michelle in Singapore
Thanks for writing, Michelle! This is a great question and one that's a little tricky to answer in a universal way, because the suitability of any given metrics will depend on the environment of the project. In some companies on-time delivery is key; in others a flawless product/service launch may be more important, even if it isn't on the original project schedule.
There are some commonalities we can point to, of course. The so-called Iron Triangle of scope, cost, and schedule comes to mind. (Were the planned features included? Did the project meet budget? Was it delivered on time?) But that naturally assumes that the project objectives were feasible to begin with (not always the case) and that the project environment, resources, and so on consistently supported continued feasibility throughout the project (even less frequently true). So metrics and effectiveness need to take that into account as well.
So in order to answer your question, I'll start by providing a few quick KPIs you may want to consider. Following this list you'll find further musings on identifying the indicators that may be most important—or most influential—in your organization.
KPIs: Success of the project deployed
- On time
- within 2 weeks
- more than x weeks late
- All scope items fulfilled
- all major scope items fulfilled
- major scope items missing
- Cost targets and budget met
- budget and targets met within 80%
- off greater than y%
- Customer satisfaction: e.g. as expressed by official customer acceptance testing results and/or number of customer complaints or help calls within x amount of time after deployment
- Customer usage or market results:
- Internal project: within x weeks of delivery, how many people used what the project delivered?
- External product: sales results
Last, but not least, effectiveness of the project team and team leader. The results of the items above imply team and leader effectiveness. For more general effectiveness measurements, I'd refer to the performance appraisals for the team leader and team members, assuming that those appraisals include their project performance. (If they don't, they should.) More on that below.
While the quick hit items above certainly cover the bases, they may not tell the whole truth, or the truth as it's most important to your organization. That's where the questions and suggestions that follow come in.
What is being measured now? Presumably if you're at the point of applying common metrics to your projects, status reporting is reasonably mature in your organization; there are likely certain standards and pieces of information that are required for a report to be complete. Reviewing those reports and formats should provide a view into what your organization considers important and valuable as a measurement of success. If your organization is all about on-time delivery, status reports are likely to be strongly skewed to time measurements; if they aren't, there are probably a lot of confused and frustrated executives wondering what's really going on! That's not to say that what's included in your status reports should be the only or primary metrics for project success, but it's very likely to be important in any successfully received measurement effort. You can see some example formats from other organizations in the Status Reports section of our list of Communications templates (at the bottom).
Regarding the measurement of team member/leader effectiveness, is this considered in appraisals? If so, you have a ready-made view into the items valued there. If not, you may want to consider including this in your appraisal process. Measurement of effectiveness on a project is a good motivator only if people are actually getting credit for doing well! One way to build this into your appraisal process (and potentially also get some metrics for project effectiveness in other venues) is described in our Project Performance Appraisal guidelines. A word of caution: appraisals are usually honest and valuable to the extent that they are also confidential and anonymous. Consider this implication carefully when folding appraisal metrics into metrics of project success.
When will these things be measured and the results recorded? This may seem like a simplistic question at first, but "when everything is done" isn't always the best time to take and record measurements. In longer projects it may make more sense to record progress metrics at the end of a phase, or as team members enter and leave a project effort. Perhaps you want to include the metrics measurement as part of the regular status reporting effort. Whatever you choose to do, the timing of the measurement should be predictable and public; otherwise, it may not accurately reflect the true state of affairs. (It's usually a bad practice to "test" people who don't know they're being tested.) And of course if you wait till everything is over to measure success or failure, you may have missed a chance to cancel a failing project before it caused momentous problems, or to save it when there was still an opportunity to do so without damaging other efforts.
How will the organization measure and account for change requests, quality, etc? If you choose to measure based on the traditional Scope/Cost/Schedule metrics, some allowance should also be made for these things. Managing change processes effectively takes skill, tact, and time, and should be included as a part of the project's overall success. Accepting all project change requests willy-nilly will inevitably cause problems, but so will refusing everything out of hand. Review our Requirements and Change Management templates for ideas. Quality measurements that focus only on bugs found may overemphasize testing reports at the expense of avoiding the coding problems in the first place. Our columnist Alan Koch has written several excellent articles on the subject of measuring project quality. In addition to these factors, your organization may need to consider others like safety, regulatory compliance, or something else we haven't thought of. If someone has been yelled at for it (justifiably), then it may be worth considering as a key project metric. ;)
What do the customers think? An argument could be made that this should be the primary project metric to determine success. Of course, what makes the customers happy doesn't always make the business successful; most customers would prefer their project deliverables were perfect, instantaneous, and free. But in general, unhappy customers will indicate an unhappy project, and vice versa. How does your organization measure customer acceptance (internal or external) and satisfaction? Those measurements are likely to be important to your project metrics.
Did the final deliverable(s) match the original vision well? Assuming that the original project vision was well written, this may be a good general pointer to project success or failure. There will almost never be a perfect match up of course—change requests and resource issues will usually sabotage all but the most vague and amorphous project visions. But a good project team will be able to manage those issues to at least be in the same ballpark. If most of your projects end up wildly different from the original project vision, that may say as much about your vision process as it does about the project teams. In that case, you may want to review some of our suggestions for successful project visions.
Brian Willard has written an excellent article on this subject at Max Wideman's site. I'd recommend starting in the middle for a good example of projects that might be considered either failures or successes, depending on the metrics used and the people asked.
Got your own suggestions for determining the best project metrics and KPIs? Share them here with your comments.