Why 70% of Software Projects Fail and Why 30 Years of Innovation Have Not Solved the Problem

Why Software Projects Still Fail – Insights from 30 Years of the CHAOS Report

Software development has fundamentally changed over the past decades. From the early days of object-oriented approaches to the rise of the internet, cloud architectures, agile development, continuous delivery, and artificial intelligence, hardly any industry has experienced a comparable pace of innovation. And yet, a look at the reality of many projects reveals a surprisingly consistent picture.

The Standish Group CHAOS Report, one of the best-known longitudinal studies on IT projects, has come to a sobering conclusion for more than 30 years: the majority of projects do not go as planned. At first glance, there has indeed been some improvement. While only around 16% of projects were considered successful in 1994, that number is now about 30%. At the same time, the share of completely failed projects has dropped from around 31% to roughly 19%.

That sounds positive at first. But a closer look quickly puts this into perspective. Because in reverse, it still means today: around 70% of all projects either fail or are burdened with significant issues. They are delivered late, exceed budgets, or fail to achieve the originally planned scope.

And this is where it gets interesting. If almost everything has changed technologically over the past 30 years, why does the overall outcome remain so similar? Why do better tools, modern architectures, and new methods not automatically lead to better projects? This article explores exactly that question.

‍

The Real Causes of Failed Software Projects Have Not Changed

The contradiction described in the previous section suggests an obvious assumption: if results barely change despite massive technological progress, then the root cause is not primarily the technology itself.

A look back at the first analyses of the CHAOS Report confirms exactly that. As early as 1994, the main reasons for failed or problematic projects included unclear or incomplete requirements, lack of user involvement, unrealistic expectations, and insufficient coordination and management. What is particularly remarkable is that these causes are still almost unchanged today.

Even current analyses describe essentially the same problem areas, albeit with more modern terminology. From “incomplete requirements” to “lack of alignment.” From “missing user input” to “insufficient stakeholder involvement.” From “weak management” to “lack of sponsorship” or “limited decision-making capability.” The terminology has changed. The reality has not.

At the core, the same pattern appears again and again: there is no shared, reliable understanding of what is to be developed. Expectations are not clearly aligned, assumptions differ between stakeholders, and decisions are made on uncertain foundations. A simple example: two teams work on the same feature but interpret a requirement differently. Both deliver “correctly,” and yet in the end, nothing fits together.

In addition, requirements inevitably change over the course of a project. This is neither unusual nor inherently problematic. It only becomes problematic when these changes cannot be properly traced, evaluated, and managed. This is exactly where it becomes clear why these causes are so persistent: they are not isolated issues, but structural deficiencies in how requirements, alignment, and decision-making are handled. These deficiencies cannot be resolved by better technologies or new methods alone.

‍

Why Poor Requirements Are Not an IT Problem

If the core causes have remained unchanged for decades and are clearly not primarily technical in nature, another conclusion becomes obvious: this is not a problem specific to software development.

Although the CHAOS Report is often discussed in the context of IT projects, the underlying patterns can be found in almost all types of development projects. Wherever complex systems are created, requirements must be understood, aligned, and translated into concrete solutions.

This applies equally to traditional software development, product development in mechanical engineering, medical technology, or plant engineering. Regardless of the industry, the same challenges keep emerging: expectations are not clearly defined, assumptions differ among stakeholders, coordination happens too late or not consistently enough, and changes only become visible when their impact is already significant.

The result is delays, increasing effort, and reduced quality. The technologies used may differ significantly between industries. The underlying problem, however, remains the same. It is not primarily about how something is implemented. It is about how clearly understood it is what should be implemented in the first place.

‍

Why Inefficient Software Projects Have Become the Norm

A particularly critical aspect is that many organizations no longer perceive this situation as a problem. Over the years, many companies have developed ways of working that indirectly compensate for these deficiencies.

Additional iterations, extensive coordination loops, and repeated corrections are no longer seen as exceptions but are planned from the outset. Schedules deliberately include buffers, budgets implicitly account for uncertainties, and additional effort is accepted as an unavoidable part of projects. In other words: a significant portion of inefficiency is already built in.

This leads to a misleading perception. Projects are considered successful as long as they stay within these expanded expectations. The fact that much of this effort could actually be avoided is often no longer questioned. Exactly therein lies the real problem. Because when inefficiency becomes the norm, the incentive to fundamentally question it disappears.

This raises a crucial question: how much more efficient could projects be if there were greater clarity from the start? The honest answer is uncomfortable. In many cases, effort could be significantly reduced, not through even better technologies or additional methods, but through a cleaner structure, clearer requirements, and a shared understanding that holds from the beginning.

‍

Requirements Engineering: The Most Important Lever for Successful Projects

How can this problem be addressed systematically? This is exactly where Requirements Engineering comes in. Requirements Engineering, the formal term for requirements management, describes the discipline of systematically eliciting, documenting, validating, aligning, and consistently evolving requirements throughout the entire project lifecycle.

It is not just about writing down requirements. It is about creating a shared understanding, making relationships visible, and basing decisions on a solid foundation. Requirements Engineering makes visible what often remains implicit in many projects.

In many organizations, requirements emerge in conversations, emails, or tickets. Decisions are made without their basis being properly documented. Relationships remain invisible, and the impact of changes is only recognized late. This is where a structured approach comes in.

Requirements are clearly formulated, linked to each other, and kept traceable throughout the entire project lifecycle. Changes are no longer implemented ad hoc but are evaluated, classified, and integrated in a controlled manner. This turns an implicit and difficult-to-control process into a consciously designed and manageable part of development.

A common objection is that requirements constantly change and can therefore never be fully clear. That is true. But that is exactly why Requirements Engineering is so important. It is not about having perfect requirements from the beginning, but about handling changes in a controlled and traceable way. Without this structure, changes lead to chaos. With this structure, they become manageable.

At the same time, practice shows that Requirements Engineering often does not realize its full potential. Not because the approach is wrong, but because it is implemented incorrectly. Too complex processes, too much documentation, and a lack of integration into daily work prevent the benefits from becoming tangible. The key question is therefore not only whether Requirements Engineering is practiced, but how.

‍

Why Requirements Management Is Often Underestimated in Practice

If Requirements Engineering addresses exactly the problems that have been slowing projects down for decades, why does it often play only a minor role in practice? The answer lies less in a lack of knowledge and more in how the topic is perceived.

Requirements management is rarely seen as innovative, rarely as exciting, and certainly not as “trendy.” While new technologies, architectures, or methods regularly receive a lot of attention, Requirements Engineering appears unspectacular in comparison. It stands for structure, discipline, and solid groundwork. That is precisely what makes it unattractive to many.

From developers to top management, attention tends to focus more on topics that are perceived as modern and forward-looking. New frameworks, platforms, or trends offer visible progress and are easier to communicate. The fundamental question of whether it is even clear what should be developed fades into the background.

In addition, the consequences of insufficient requirements management often appear with a delay. The causes lie at the beginning of a project, while the symptoms such as delays, increased effort, or quality issues only become visible much later. This leads to a paradoxical behavior: organizations continuously invest in optimizing execution while neglecting the foundation for it.

The result is a pattern that has been repeating for decades: execution becomes more efficient, but not necessarily more correct.

‍

Why We Finally Need to Rethink Requirements Management

If the fundamental problems have barely changed for over 30 years, it is not because there are no solutions. It is because the focus is wrong.

For decades, execution has been continuously optimized. But this very optimization is now reinforcing the problem: unclear requirements are implemented faster, misunderstandings spread more quickly, and wrong decisions have a greater impact.

We have reached a point where incremental improvements are no longer sufficient. It requires a conscious shift in focus. Away from the question of how we can develop faster. Toward the question of how we can better understand what we are developing.

This does not mean replacing technological innovation. It means finally giving it the foundation it needs. Structure, clarity, and traceable decisions must be given the same importance as speed and efficiency.

Because technological capabilities today are more advanced than ever before. What is missing is not the ability to implement, but the clarity beforehand.

Thanks to Dr. Marcus Trapp for the inspiration for this article during a pleasant dinner.