As a project manager, I’ve often been approached by people asking me why their software project wasn’t successful. This may seem like a simple question, but sometimes it really isn’t. In order to determine success, we need to understand the Critical Success Factors (CSFs) of a project and the way that these factors relate to each other.

At some point in the dim and distant past, someone once told me that the only things that are important in determining a successful project are: budget, schedule and quality.

• Budget: Did the project cost fit within the estimated budget?
• Schedule: Was the project completed on time?
• Quality: Does the resulting application meet quality standards?

So far, we’re talking universally understood axioms of project management, right?

Let me ask a rhetorical question. Is it possible for all three answers to be yes and yet the project not be considered a success? The answer to this question is, surprisingly, yes. This is because there is another factor to be considered.

Let’s return for a moment to the basics. Perhaps you’ve seen a diagram that looks something like this.

When I originally encountered this depiction of a project’s CSFs, it did not contain the blue circle. Ultimately a project can be considered a success only if its stakeholders are satisfied. Stakeholders, as the word implies, are people that hold some sort of stake or interest in the project, whether that stake is as a business user, a business sponsor/funder, someone that reaps a profit from the project or a vendor that contributes to the project, among many other possibilities.

The complication here is that projects may have many stakeholders and different projects may have different classes of stakeholders. These varied stakeholders may not share a single common understanding of what comprises success, so it isn’t easy to make sure that the project delivers what they expect it to. Later we’ll refer to this as their “expectations.”

Imagine that you work for a small software development company and are contracted to build an application for a client. The client was offered a fixed price and you delivered the project on time, with no change requests and to a high quality. When you originally calculated the cost of the work, you underestimated the effort required and so, in order to complete it on time, you had to load on some extra resources to handle the unexpected scope. Your client, who is one of the stakeholders, was ultimately satisfied because you met the CSFs from his point of view (no scope changes so no change requests and no price increase). However your management is also a stakeholder, and is concerned because you used up all of the contingency budget to the point that there was no longer a profit margin on the project, perhaps even a loss. We are left hoping that this client is so pleased that he returns and you can make up your losses in successive contracts with him!

The moral here of course is that as a PM you should be very clear about what all the stakeholders expect from the project, and work with them to make sure that their expectations are realistic. You then should deliver what they expect.

Let’s consider the same case as above, but in this alternative you didn’t add resources but, instead, you forced the project team to work overtime to cover the hidden requirements, so you could still finish on time. Perhaps in this case, your boss was pleased because you actually made a little margin on the project, assuming of course you didn’t pay the team overtime.

Could this be considered a success? It isn’t a success if the project team blames you for the overtime and three of your top performers in the development team then move on to greener pastures where they are not required to work such ridiculous hours!

Your project team is clearly a stakeholder and their satisfaction matters too. In this case, you may have sacrificed their needs for the sake of making the project look good on paper.

The Balancing Act

As a project manager you’re always spinning many plates, and making many critical decisions throughout the life of the project. Each of these decisions will affect the outcome of the project.

Consider our little case study again. Let’s say you cleverly decide that, by avoiding some of the work to ensure that the application will be scalable, you are able to cut a few corners and avoid adding resources or working your project staff to death, allowing you to finish on time. Despite the shortcuts, the application sails through User Acceptance Testing (UAT) and goes into production with nary a bug reported in the first six months. The application is wildly successful beyond yours or your client’s dreams. Then after one year the system collapses under the sheer weight of its own success because it can’t handle the transaction load being placed upon it.

So now, is the project still a success? Clearly client satisfaction is compromised because they’re ringing up your boss and telling him that this situation is intolerable and must be fixed immediately. No doubt he’s not very pleased at this point either. Obviously there is a time component to this stakeholder satisfaction rating. The answer to the question about success in this case is, “it depends.”

Part of the balancing act is to ensure that every stakeholder is made clear about what to expect throughout the life of the project. Clearly if you set the expectation that the application would perform under the current transaction load and it doesn’t, then it is fair to say the project was not a success. But if you did your analysis thoroughly and correctly, documented the client’s performance requirements and tested that the application met them, ultimately this becomes an exercise in damage control. It still is not an enviable situation to face, but at least it’s one where you may come out of it with your reputation intact. In general the moral here is to take care to document all non-functional requirements carefully, because these are often overlooked and may end up causing avoidable project failures down the road.

Be sure that, once you have done your utmost to make everyone clear about what to expect from the project, you don’t play fast and loose with the requirements. Especially to the extent that you make decisions or judgment calls that will have an adverse impact on what is delivered to meet those requirements.

Back to the Triangle

If you do a little research on the Project Management Triangle, you’ll see a slightly different depiction that shows scope, budget and schedule pushing on each apex of the triangle with quality in the middle. In the case of CSFs, scope has sort of been removed from the equation because, hopefully, it has been fixed at the project start. Our CSFs triangle depicts the fact that you cannot change one edge of the triangle without impacting one or both of the others. All three sides affect the stakeholder satisfaction.

When scope changes during a project’s life cycle, the process to handle that is called change management. Using this process, by the end of the project, scope should once again be fixed. As long as you deliver the agreed scope within budget, on schedule and with sufficient quality, as a PM you’re pretty much covered.

Woe betide the project manager that doesn’t understand the change management process or fails to identify changes when they appear. Earlier on, we talked about the need to document what is expected to be delivered and the same is true about scope. If the scope is properly documented, and this includes quite a bit of documentation that explains what’s outside the scope, then it is usually easy to agree what comprises a change. All that is necessary is to show the variance in comparison to the expected scope. Of course, it can be just as bad to fail to act on scope-creep when it happens as not documenting it in the first place.

Just as entropy in systems always increases, so does scope. Scope rarely if ever narrows without someone, most likely the project sponsor, making a conscious effort to do so., Sometimes, particularly when faced by budget shortfalls, it may be possible to offer some scope increase in exchange for limitations on the agreed scope in other areas.

Even when you do your job as a PM and navigate the perilous change management process with success, you may find that some of your stakeholders will be less than enthused with your skills. You just offered your client a fixed price, and now what? You’re changing that price? Of course, we justify it by gaining the client’s approval, which may only be given begrudgingly. I’ve had clients before whose unannounced policy for contracted projects was “no change requests.” This was in fact one of the CSFs that the client’s PMs were held accountable to.

You may find that excessive management of changes may very well impact your stakeholder satisfaction, whereas failure to do so will most certainly impact the other project CSFs.

Manage to Your CSFs

As a PM you must manage to your project’s CSFs. This means that you must know at its inception, what is critical to ensure that your project will be a success, and then manage the project to ensure those CSFs are precisely achieved.

It almost sounds silly to say it that way… that you have to manage to your CSFs. You’re probably thinking “of course you do!” To most PMs that first sentence would probably sound silly because of course you have to manage to achieve budget, schedule and quality expectations. Failure to understand and take into account what it will take to satisfy all of your stakeholders could easily doom the project to failure. The stakeholder satisfaction CSF in the end may be the most critical that you must manage to.

Let’s pose another rhetorical question. Have you ever delivered a project early? Or perhaps you’ve delivered a project significantly under the proposed budget. How were those projects received, i.e., were they perceived to be successful?

Often times, you may be surprised to learn, the answer may very well be no. Perhaps the manager had a specific budget to burn before the end of the year and you didn’t use it all, causing a reduction in the budget for the following year. When you finished that project early you found that the client wasn’t ready on their side to slide it into production. So it became shelfware for a few months while they prepare (this reflects badly on them and of course they’ll blame you). Then just when you’ve successfully managed all change out of a project, and you deliver it ahead of time and under budget they’ll be wondering why you didn’t simply allow for a little drift in the scope as long as it could be managed within the schedule. How do you think that’s going to affect your satisfaction ratings?

A good rule of thumb when it comes to schedule and budget, is that you should try to deliver just slightly early (no more than 5% of the overall schedule) and just under budget (by no more than 10%).

By now you’re probably thinking I’m crazy; that never happens because projects are always late and always over budget. I will acknowledge this to be quite common, but it doesn’t change the fact that these are some of the CSFs that you should manage to if you don’t want to unduly impact your stakeholder satisfaction ratings.

The Flexibility of Quality

While the schedule and budget legs of our triangle are rigid and unbending, perhaps the quality leg has some flexibility that can be exploited to contribute to our project’s success. More often than not this ends up being the actuality whether or not it’s a good idea, meaning that quality is often sacrificed when a project runs late or the cost becomes excessively high.

Excellence in quality exhibits itself by a lack of issues in deliverables, while poor quality is easily exposed. Happy is the project manager whose application flies through User Acceptance Testing (UAT) with few or no issues, only to see that smile turn into a frown once those issues surface in a production environment.

You should have set your client’s expectations that there will be issues during UAT and there sure better be some, because if there aren’t any issues, then it is highly likely that the client wasn’t doing their job properly.

Here is where our balancing act is most apparent. All clients believe that the level of quality must be acceptable to the service activity to which the application is applied. It is the PM’s job to interpret exactly what that means (read it closely), set expectations accordingly and then deliver to them. Remember that quality of delivery has greater importance when applied to a Business to Consumer (B2C) banking application than to a content management web site.

The typical client’s expectation is that they won’t want to see any bugs in their production application. That is pretty specific yet utterly unattainable, unless you test until you’ve expended your available testing resources and then continue to test some more. So we introduce some balance by acknowledging that there probably will be bugs and even though the client says he doesn’t want to see any, we know that there are likely to be some. This is an endemic risk of application development that we mitigate by a little education and a whole lot of after service care. The trade-off is allowing that a few bugs may persist but they’ll be quickly rectified.

Proper testing is the key; however quality of the testing is much more important than the quantity. While the breadth of functionality must be tested, its depth must be explored in places where bugs surface the most often.

More important still is to build quality into the development of the application, well before it is uncovered by the testing, in order for it to be remediated at lowest cost. If you haven’t implemented code reviews and daily builds into your development process, you’d best do so now or your quality will never be there by the time the application gets to the testing phase.

Whatever you do, do not sacrifice too much quality in order to achieve the budget and schedule legs of the triangle because this will surely impact stakeholder satisfaction.

Conclusions

You may think, after reading up to this point, that project management is a thankless task and no matter what you do right in the end you’ll be blamed when things go awry. However project management can be challenging, exciting and fun as long as you keep in mind the fundamental CSFs and manage each from the perspective of your many stakeholders. You will always need to make trade-offs and achieve compromises. To maximize the likelihood that in the end your project is considered a success, the following steps are strongly recommended:

• Cleary identify your project’s complete list of stakeholders and clarify with each precisely what constitutes a successful project for them.
• Craft detailed documentation relating to scope, non-functional requirements and the CSFs for the project, so that upon conclusion the result cannot be in doubt.
• Pay careful attention to budget (cost), schedule and quality, but do not fail to take into account those intangibles that will most likely result in satisfaction for the majority of your project’s stakeholders.

If you follow this advice, you will not often have to ask the question, why wasn’t my project successful?