A survey of IT experts revealed 43 percent of their organizations had recently killed an IT project. The study, conducted by ISACA, an independent IT governance group, highlighted the top 5 reasons these organizations named for terminating projects prior to completion.

Here’s the list, with my commentary on each issue:

1. Business needs changed: 30%

There are many conditions and situations where a business legitimately changes its requirements after starting a project. If the project no longer provides meaningful value, then it’s best to stop throwing good money after bad.

On the other hand, some organizations deliberately obscure a flawed project requirements process by claiming business needs evolved. Obviously, that’s unhealthy and a true sign of failure.

2. Did not deliver as promised: 23%

This is a typical expectation-setting problem: promise anything to get funding and worry about the consequences later. Shortsighted managers don’t realize that funding is less important than delivering substantive value. Failure is inevitable when managers don’t clearly identify and deliver business value.

In some cases, the project really did provide value, which the organization did not recognize due to communication problems. I recently blogged about one CIO seeking a publicist, presumably to address this issue:

Many organizations take a CIO for granted when his IT department consistently delivers the goods without fanfare and attention; sadly, this human failing is all too common. In that case, PR might be a great idea, especially if the CIO isn’t a great communicator. Of course, the CIO should improve his communication skills, but that’s another story.

3. Project was no longer a priority: 14%

If the organization shifted direction without good reason, thus making the project superfluous, then flawed strategic planning was the culprit. However, if business requirements changed for a good reason, as suggested in point one, there’s not necessarily a problem.

In general, and this is an obvious point, canceling projects without a darn good reason is a definite sign of failure.

4. Project exceeded the budget: 13%

On the surface, over-budget projects are the basic metric for failure. I’m actually surprised this number isn’t higher, because unanticipated cost is always such a clear red flag.

At the same time, some projects run over-budget due to intelligent scope increases that provide additional value. For example, while automating two departments, the project team realizes it can add a third department for only marginal increases in cost. In such cases, going forward is probably the right decision despite the higher spend.

Although tempting to use budget performance as simple metric of success or failure, that approach can be overly simplistic and ignore important nuances related to business value. Nonetheless, anytime a project goes over-budget the team must offer a detailed explanation.

5. Project did not support the business strategy: 7%

This classic indicator of failure often suggests a project rooted in poor requirements analysis. However, as with previous points, it’s also possible changing business needs made the original project goals obsolete.

=====

The survey is most interesting to highlight significant issues related to project failure. However, some of the questions are too ambiguous to provide straightforward conclusions. In general, understanding whether a project is successful requires examining the business environment and context.

[Image of man concerned about his budget via iStockphoto.]

In this ideal world, business users gratefully accept whatever techno-babble nonsense IT throws at them, with technology having become utterly divorced from any business purpose whatsoever.

Call it “Stepford IT,” where business people have (finally) been trained to carry forth IT’s bidding with flawless accuracy and pleasing smiles. Yes, this is indeed IT utopia in action.

[Oliver Widder, creator of Geek and Poke, drew the original cartoon in response to a post by ZDNet colleague, Joe McKendrick, called “SOA targets the business, but is the business sold on SOA?“.]

Texas Governor, Rick Perry, ordered state agencies to stop transferring data to IBM to prevent data loss on a failing project. Texas also fined IBM $900,000 for failing to meet data backup requirements specified in the state’s $863M outsourcing contract.

This backup situation represents only tip of the iceberg with lots more failure to come. Be forewarned: as they say, “You ain’t seen nothin’ yet.”

The Dallas Morning News reported:

[A] July server malfunction in the attorney general’s Tyler Medicaid fraud unit destroyed nearly half of eight months’ worth of documents – compromising scores of prosecutions. In the months before that crash, more than 10 agencies complained about network breakdowns and server backup problems with IBM….

In an unusual move, Perry has positioned himself as IT program manager on the disastrous project:

IBM has failed to perform “the crucial backup of data for more than 20 state agencies,“ Mr. Perry wrote in a letter to Brian Rawson, who oversees the IBM contract for the state’s Department of Information Resources [DIR]. “The agency has failed to implement a system of checks and balances that ensures data security, jeopardizing the ability of state agencies to deliver services to their constituencies.”

IBM’s response provided no insight whatsoever in the company’s position or thinking:

“IBM takes very seriously the issues that have been reported,” [a spokesperson] said. “We are committed to helping the state to better serve its citizens through the innovative use of information technology.”

In a separate article, the Morning News did report that IBM acknowledges things haven’t gone well with the Texas contract:

In a Nov. 3 letter to the governor’s office, IBM acknowledges the company overreached by assuming responsibility for existing technological conditions that are inadequate, inconsistent and not sustainable.

“The state and IBM will have to evaluate the investments required to remediate the current deficiencies, develop a funding model, and prioritize accordingly,” wrote two IBM officials.

The Morning News added:

At the time the contract was signed, the state and IBM said taxpayers would save $25 million over the first two years. An early cost analysis by the accounting firm Grant Thornton showed the project had saved the state about $500,000 overall through February.

Texas initiated the project in 2005 with House Bill 1516 , which called for the state to consolidate purchasing and outsourcing of IT-related goods and services. According to DIR, the outsourcing program, called Team for Texas, will reduce waste and redundancy caused by IT system overlap in departments across the state:

Four statewide goals governed the development of this plan. Reducing government costs and driving effective technology contracting, expressed in the 2005 State Strategic Plan, Shared Success, remain as key components of this plan and are encompassed within the single goal of leveraging the state’s technology investment. The goals are:

• Leverage the state’s investment in shared technology infrastructure
• Protect and secure technology assets and information while safeguarding citizen privacy
• Simplify citizen, government, and business access to public-sector services and information
• Promote the innovative use of technology that positively impacts the state’s business, as well as its economic development

THE PROJECT FAILURES ANALYSIS

This failure seems to be the result of overly ambitious transition time frames, driven by IBM’s desire to secure the business combined with Texas’ need to quickly push its outsourcing goals forward. Significantly, poor change management also played an important role, as suggested by this comment in the Morning News:

One IT employee, who asked not to be named because of his connection to Team for Texas, acknowledged that IBM walked into state agencies ill-prepared. But the employee said the problem was compounded by hostile treatment from longtime state employees.

Solving this problem will require substantial investment for both IBM and the state. IBM must acknowledge its own mess, recognizing it accepted the contract without fully preparing to handle the scope of potential problems.

A more cynical view might suggest IBM accepted the contract despite these issues, knowing that project scope, and therefore IBM billings, would eventually increase as a result of these project start issues. Given the interlocking and conflicting agendas inherent in Devil’s Triangle relationships, this scenario is not far-fetched.

• Related: Consulting’s dirty little secret

Texas is hardly blameless. The governor must conduct an urgent session with agency heads to gain their support. IBM cannot, and will not, succeed without the state’s active cooperation. The governor has inserted himself into the middle of this nightmare, and now he must find a way to get the departments to work with IBM. That’s going to be a tough job: Gov, I feel your pain.

Based on progress to date, I don’t believe either IBM or the state of Texas will meet this challenge, thus creating large-scale delays and cost overruns for years to come. My prediction: the iceberg of failure is rising in Texas.

[1912 illustration of Titanic hitting the iceberg from Ulster Folk and Transport Museum.]

During testimony before Congress, in a hearing titled “The Dismal State of Federal Information Technology Planning,” Norm V. Brown, Executive Director of the Center for Program Transformation, an IT failures think tank, presented what he calls “Laws of IT Physics™.”

These “Laws” highlight hidden pitfalls the hurt many projects, and help explain why some projects succeed while others fail. They recognize that successful IT project delivery is primarily about managing people, process, and deliverables. Yes, technology is important, but the people side comes first. Here’s the list, with slight editing:

1. Planning is a continuous process, not a one-time event. A Project Plan cannot survive past contract award and must continually change based on actual experience (requirement additions or modifications count as actual experience).

This is the silent killer! Lewis Cardin, senior project portfolio management analyst at Forrester Research, calls this “first number syndrome,” where senior management becomes tied to initial estimates and refuses to accept change.

• Related: ‘Debunking IT Project Failure Myths’ [podcast]

It’s completely unreasonable not to expect that lengthy projects will evolve over time. Change isn’t necessarily bad; for example, business needs may evolve and force a corresponding project adaptation. Balancing shifting priorities through scope changes is the difficult part of this equation.

2. Complexity kills IT projects since defects and security vulnerabilities increase non-linearly with increased complexity. Minimizing and controlling complexity are key to successfully achieving a large-scale system development success both in the development of individual releases and in the cost and schedule of downstream upgrades to operational software.

3. Schedules and project chaos create event horizons, from which a project cannot recover. Avoid the Project Event Horizon; Compute Schedule Compression and Monte-Carlo simulate the Task Activity Network.

According to Wikipedia, in general relativity an event horizon is a boundary inside which events cannot affect an outside observer. Combine a complex technical environment with byzantine project scheduling and workflow, and meltdown becomes likely. Simplify wherever and whenever possible.

4. The initial requirements for any large system will be incomplete, independent of the resources expended to develop them. Ensure planned requirements can be delivered within cost and schedule estimates, but also include budget for anticipated and actual requirements change; rigorously test, accept, and track requirements as they are met.

5. Unvalidated requirements pave the road to project failure. Test and validate requirements as early as possible before basing significant projects upon them; use pilots where possible before fully committing.

Requirements planning is an absolute foundation for successful project delivery, which is always rooted in well-defined user requirements and carefully managed estimates. If you screw up here, failure is inevitable.

6. You can’t manage what you can’t see. Track Project Status and Progress against small, testable, incremental product deliverables and use quantitative project parameters, such as Earned Value, to make projects visible and manageable.

7. Not controlling the right things assures failure. Use well established best practices such as Risk Management, Requirements Management; Defect Management; and Integrated Baseline Reviews to control projects.

8. Poor defect management causes high rework and leads to project failure. Use automated testing and continuous integration to prevent defects, and continuously identify out of phase defects to correct their root causes.

Yes, testing matters a lot. Just ask retailer J.Crew, which lost the ability to ship product to customers after deploying a new web site and content management system without sufficient testing.

Related: J.Crew: Failed upgrade hits financial performance

Testing isn’t rocket science, but it must be performed in a disciplined and consistent manner.

9. Unknown and untreated vulnerabilities originating in ineffectually implemented processes destroy IT projects. Automate vulnerability identification and prioritize fixes which root-out and fix processes lacking critical essential detail needed to achieve bottom-line objectives.

10. Development contractors will do what is in their financial interest, and government organizations may be led toward a project event horizon. Incentivize well and wisely, trust but verify, and use Award-Fee type contracts; carefully construct the Award-Fee criteria to address principal project objectives over the near term; identify what Award-Fee structure will sufficiently motivate the development contractor.

Every major IT project consists of a partnership I call the Devil’s Triangle: customer, technology provider, and system integrator working together to achieve a common goal. These groups have interlocking and sometimes conflicting agendas, which makes management a balancing act of priorities.

Related: Consulting’s dirty little secret 

Most consulting companies are honest folks doing the right thing for their customers. Still, you need to protect yourself from bad apples by carefully controlling contracts, incentives, and penalties.

11. Thoughtful, knowledgeable, committed people operating as a team are critical to IT project success. Treat people as the valuable resources that they are; take actions to create and maintain “jelled” teams.

Although these 11 laws are oriented toward government projects, the lessons are equally applicable to projects in the private sector. The list is worthy of your thoughtful consideration.

Jonathan and I are taking the IT failures discussion off the blog page, and into the real world, on December 3 at AMR’s office near South Station in Boston. There’s no cost and you’ll meet other folks interested in chatting about this topic. This interactive, unconference-style session will be fun and informative; Jonathan will serve as discussion initiator to kick things off.

And maybe best of all: I’ll supply coffee and bake homemade muffins for all attendees! Yeah, I love to cook so this will be fun.

Topic: IT Failures Town Hall: Lessons to be learned so you can stay off the list

Session description:  As the economy worsens, many organizations have higher expectations, and less tolerance, for failed projects. We’ll discuss strategies for reducing failures, increasing project success rates, and demonstrating higher project ROI. Most of all, we plan to have a great time!

Discussion initiator: Jonathan Yarmis is Vice President at AMR Research, where he is responsible for the firm’s coverage of advanced and disruptive technologies. A longtime analyst, having previously covered enterprise issues for Gartner, he’s a frequent and highly in-demand conference speaker.

Date and time: Wednesday, December 3, 2008 from 8:00-9:30 AM

Location: AMR Research, located at 125 Summer St, 4th floor in Boston. Map and directions.

Cost: Free. I’m bringing coffee and baking fresh muffins.

Dress: Casual

Registration: Everyone is welcome to attend, but please register by signing up.

[Photo by Michael Krigsman.]

The Senate introduced a bill establishing guidelines for transparency and reporting on troubled IT projects. The bill defines specific exception thresholds for measuring critical and large government IT projects. Projects exceeding these thresholds will be evaluated for remediation and possible termination.

Here’s how the Congressional Budget Office (CBO) described the bill, which is called the “2008 Information Technology Investment Oversight Enhancement and Waste Prevention Act” (S. 3384):

S. 3384 would amend federal law regarding the oversight of project planning for information technology (IT) systems. The legislation would require chief information officers to identify critical IT projects and would subject each IT project to additional reporting, planning, and monitoring requirements, including corrective actions for projects that fail to meet applicable standards. S. 3384 also would establish an IT Strike Force (a group of information technology experts) overseen by the Office of Management and Budget (OMB) to respond to problems with individual IT projects.

NextGov added:

The Act would force agencies to think more deeply when establishing cost, schedule and performance baselines for major IT investments. Agencies would be held accountable for their initial projections; if a project varied more than 20 percent from the original baseline estimates according to the standards of earned value management, the chief information officer would be required to notify

If a project varied more than 40 percent from the original baseline, then OMB would have to determine if there were other ways to meet the objectives, or if the agency should pull the plug on the project altogether.

THE PROJECT FAILURES ANALYSIS

This is a highly significant development representing a state of the art effort to control IT failures. Although the bill text is brief, it establishes several key points:

• Defining failure
• Specifying remedies
• Requiring transparency

Perhaps most remarkable, the bill offers a general formula covering a vast range of present and future IT projects. From that perspective, the bill is forward thinking and ingenious. Of course, theory often breaks down in practice and these general guidelines may not work as intended. Still, it’s a great start toward building a framework of acceptable limits to IT failure.

The guidelines are beautiful because they’re so specific and clear; if you’re running a screwed-up project, this bill specifies your legal reporting obligations. I won’t repeat the details here, since you can easily read them yourself.

Fellow Enterprise Irregular, Jason Wood, Twittered his concern that the bill will create more red tape. However, it’s important to note the CBO estimated implementing this bill will cost “between $25 million and $50 million over the 2009-2013 period.” To put this number in perspective, the CBO added:

According to the Government Accountability Office, OMB and federal agencies have identified more than 400 IT projects—costing $25.2 billion in 2008—as being poorly planned, poorly performing, or both.

The bill is neither perfection nor panacea. However, it is an excellent early stage effort to monitor, track, report, and fix troubled government projects. Although some large private sector organizations already have similar mechanisms in place, such discipline is unusual and definitely shows the future of strategic IT management.

Believe me; removing IT failure wiggle room is a big deal.

[Image showing project management denial, mixed with attempts to wiggle out of transparent disclosure, via beforethearchitect.com.]

General Motors issued a recall for almost 13,000 Cadillac CTS vehicles, due to a software bug in the airbag sensor. Transportation-related software problems happen all the time.

The National Highway Transportation Safety Administration (NHTSA) lists details:

GM is recalling 12,662 my 2009 Cadillac CTS vehicles for failing to conform to the requirements of federal motor vehicle safety standard no. 208, ‘Occupant Crash Protection.’ Under certain conditions, a software condition within the passenger sensing system may disable the front passenger air bag when it should be enabled or enable it when it should be disabled.

In a vehicle crash, if the front passenger air bag does not operate as designed, increased personal injury could occur.

I never cease to be amazed at the ubiquity of programmable technology in our daily lives; of course, more technology means increased opportunities for failure. As systems, including cars, become more complex, fully testing every component seems like an increasingly elusive goal.

This is hardly the first airbag software problem an automaker has faced. For example, earlier this year Volvo recalled 65,000 cars when a software problem caused airbags to deploy improperly. Also in 2008, Ford recalled 470,000 Mustangs because the “passenger-side frontal airbag could potentially deploy with greater-than-allowable force for a petite, unrestrained passenger.”

• Related: Volvo recall due to airbag software problems

Airplanes are also subject to software and computer failure issues. Just last month, a Qantas jet plunged unexpectedly during mid-flight, injuring 70 passengers. A faulty data stream caused the problem.

The Australian Herald Sun reported:

The [Australian] transport safety bureau has already declared that the aircraft’s air data computer — or inertial reference system — sent erroneous information to the flight control computer causing the autopilot to disconnect.

How can you protect yourself from software bug-related problems when traveling? Cross your fingers and hope equipment manufacturers invest the time and resources needed to more fully test their products before release.

[Via Product Defect News blog. Image © Copyright General Motors and used by permission.]

During a presentation at AMR Research’s Business Technology Conference, J. Chris Scalet, Chief Information Officer of pharmaceutical manufacturer Merck discussed the changing role of CIOs in large organizations.

Scalet said:

The CIO role is at a key tipping point. Unless CIOs take on added responsibilities, the role could revert to a pure technology focus, which would be a shame.

He added that in the past 75% percent of a CIO’s time was spent talking about technology and 25% was spent discussing the business. Today, the role has evolved so those numbers are now reversed. In many progressive companies, CIOs have taken full responsibility for core business functions such as supply chain, procurement, facilities, and back office operations.

In addition, organizations are demanding that CIOs actively participate in:

• Reducing costs
• Developing business strategy
• Improving market share
• Championing change
• Driving innovation

CIOs maintain a unique position inside many organizations, sitting at the juncture of business and technology. From that perspective, Scalet believes these additional functions make sense, especially in organizations where the CIO serves as partner with the CEO in delivering value to the organization. He acknowledged this evolution has been the subject of debate, as some observers believe this expanded scope may distract the CIO’s attention away from core technology issues.

Scalet believes CIOs should ultimately serve as business architect, addressing issues such as:

• Technology architecture
• Information architecture
• Process architecture

Significantly, he thinks CIOs should serve as executive sponsor for large-scale change initiatives, taking accountability for the outcome and results. I believe this approach is overly risky for most organizations, where gaps between business and IT already cause many problems and failures. On the other hand, Merck has developed a highly evolved, and unusually close, partnership between business and IT.

[Photo of AMR CEO, Tony Friscia, and Merck CIO, J. Chris Scalet, by Michael Krigsman.]

I’ll be attending AMR Research’s Business Technology Conference in Boston on Wednesday, November 10. If you’re also attending, please say hello so we can talk failure!

The conference is heavily weighted toward the business issues associated with IT. From the conference description:

The best companies execute an IT strategy that can both lead and enable operating performance. There is a call to arms for the CIO to take on strategic initiatives, and translate them into prioritized business and technology decisions.

IT success and failure is all about how the business manages, and then executes around, technology investments, so this is an important topic. Hope to see you there.

[Photograph of Beacon Hill in Boston by Michael Krigsman.]

Here’s a list of 16 IT failures worth remembering. Covering a full-range of devastated projects from both government and private industry, this list shows the true diversity of IT failure.

I found the list on a site called Lessons from History. As you browse the failures, consider the waste involved. Bear in mind, these 16 are nothing more than a few representative samples of failures that happen every day, all around the world:

1. The IRS project on taxpayer compliance took over a decade to complete and cost the country an unanticipated $50 bn.
2. The Oregon DMV conversion to new software took eight years to complete, the budget grew by 146% ($123m) and public outcry eventually killed the entire project.
3. The State of Florida welfare system was plagued with numerous computational errors and $260m in overpayments!
4. August 2008 Unencrypted memory stick lost with names/dates of birth of 84,000 inmates, England ’s entire prison population. Home addresses of 33,000 who had six convictions.
5. Feb. 2007 £20bn UK NHS computer system ‘doomed to fail‘a senior insider has warned.
6. 2007 laptop with records of 600,000 recruits was stolen from Royal Navy recruiter’s car
7. In September 2006 Department of Homeland Security admitted project failure and closed the Emerge2 program $229m (a new financial IT system).
8. In May 2006 the disastrous Seasprite helicopter program for the Australian Navy, with $1bn spent, the helicopters were grounded due to software problems.
9. In April 2005 inter-departmental warfare played a significant role in the failure of a $64m federal IT project.
10. In 2005 British food retailer J Sainsbury had to write off $526m it had invested in an automated supply-chain management system.
11. In 2005 US Justice Department Inspector General report stated $170m FBI Virtual Case File project was a failure, after five years and $104m in expenditures. Over one 18-month period, the FBI gave its contractor nearly 400 requirements changes.
12. In 2005 the UK Inland Revenue produced tax payment over payments of $3.45 bn because of software errors.
13. May 2005 major hybrid car manufacturer installed software fix on 160,000 vehicles. The automobile industry spends $2 to $3 bn per year fixing software problems.
14. July 2004 a new government welfare management system in Canada costing $200m was unable to handle a simple benefits rate increase. The contract allowed for 6 weeks of acceptance testing and never tested the ability to handle a rate increase.
15. In 2004 Avis cancelled an ERP system after $54.5m is spent
16. In 2002 the UK government wasted £698m on Pathway project, smartcards for benefits payments, & £134m overspend on magistrates’ courts Libra system.

[Sinking ship photo via Wired.]