Summary
Large-scale Systems Development is a high risk undertaking with staggering industry failure rates. This White Paper helps to explain fundamentally why this is so and sets the stage for describing Pro-Found solutions.

For more information, contact:

William Frenkel
Pro-Found Software, Inc.
4555 Henry Hudson Parkway - A206
Riverdale, NY 10471
646-831-2990
wfrenkel@pro-found.com

The Symptoms
Consider these frightening statistics: 80% of large-scale applications either are scrapped during development, are shelved after completion, or are ‘working failures’ that do not meet expectations.¹ Even worse, for those systems which survive development blow-outs, annual system maintenance consumes more than 50% of the 1.7 million programmers in the United States.² These manifestations of software failures transcend techniques, methodologies, platforms and industry segments and have defied leading industry and academic practitioners since the birth of computers some 50 years ago.

Complexity – The Root Problem
Software systems are more complex for their size than perhaps any other human construct.³ This complexity occurs along multiple dimensions and is insidious: One must truly understand this complexity in order to successfully conquer it.

“From the complexity comes the difficulty of communication among team members, which leads to product flaws, cost overruns, schedule delays. From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes unreliability. . . . From complexity of structure comes the difficulty of extending programs to new functions without creating side effects.

Not only technical problems, but management problems as well come from the complexity. It makes overview hard, thus impeding conceptual integrity. It makes it hard to find and control all the loose ends. It creates the tremendous learning and understanding burden that makes personnel turnover a disaster.” ⁴

Essential Complexity
Unlike scientists who search for “natural order” and “unifying theories”, software developers face arbitrary complexity created over time by people and institutions. In many cases, new software must conform to existing systems, rules, and business practices which have preceded it. To a great extent, complexity is the very essence of software and cannot be abstracted away. Software cannot be any less complex than the domain which it models.

Artificial Complexity
Artificial complexity refers to the difficulties associated with the production of software; the technology of software development. Artificial complexity is brought on by operating systems, compilers, programming languages, networks, database managers, etc. Problems associated with version control, testing, performance, and security are also part of this artificial complexity. Most attempts at slaying the software development dragon have focused in these areas. Unfortunately, the dragon is stronger than ever. Consider the following quote relating to client-server technology:

“If host-based computing is a 747, client-server computing is 10,000 parts flying in close formation” (Bob McGovern, VP of Legent Corporation).

Bob McGovern, VP of Legent Corporation.

Internet applications tend to be even more complex. Additionally, software has a unique characteristic which compounds these complexities geometrically. Unlike manufactured things such as buildings and cars which are infrequently changed after manufacture, software is thought to be infinitely changeable. After all, software has no real physical existence.

As people begin to interact with software, they think of new or better ways for the software to work. In many cases, people push software to the edge or beyond its original design. If software were a car, people would want it to fly. Thus, the very act of using software becomes a strong catalyst for change. Moreover, software lives within a world of rapidly changing business practices, users, laws and technologies which all impose change.

Countering Complexity – The Solution
Many consultants and software vendors try to convince us that they have tools and/or techniques which represent the only “true solution”. They are all wrong. Yet many of their tools and techniques have significant merit. Many practitioners believe that software construction is a purely creative process much like music or art and does not lend itself to standardized approaches. They, too, are wrong. Yet study after study shows that the very best developers consistently create software which is better, faster, cleaner and produced with less effort. What is common to all effective tools and techniques and to all successful software “artists” is one characteristic - the elimination of complexity. Engineers often refer to elimination of complexity as elegance.

“An elegant solution reduces the problem to its root and addresses all of its aspects in the simplest possible manner. . . . Elegance counters complexity, knocking it down enough that we can move forward a little more without becoming hopelessly lost. . . . Elegance endows software with a simple and robust architecture that seems to have endless applications and extensibility.” ⁵

We believe that there are practical approaches to increase software elegance which can be applied across many technology environments and which can coexist with a wide range of tools and methodologies. Further, one need not be one of the chosen few software “artists” in order to benefit from these approaches. Visit our Technology Demonstration Center or call for more details.

References
1. Scientific American, Sept. 1994. Software Productivity Research
2. Capers Jones, chairman of , In. in Burlington, MA.
3.Fredrick Brooks, “No Silver Bullet: Essence and Accidents of Software Engineering”, COMPUTER vol. 20, No. 4.
4. Ibid.
5. Jeff Fisher and Dale Gipson, “In Search of Elegance”, Computer Language, Nov. 1992.