The Fall of WaterfallThe Fall of Waterfall

While you're at it: A symptom that causes project scope to spiral out of control. A false belief in ever-diminishing marginal feature costs often make business users act like kleptomaniacs on the "Supermarket Sweep" game show when formulating requirements.

Once-in-a-lifetime opportunity: Business users often feel that future enhancement requests will sit on an infinite backlog. This fear tends to make them inclined to ask for everything they ever anticipate needing and hesitant to prioritize anything below "critical."

Lifetime project opportunity: A strict waterfall methodology with a long development life-cycle limits adaptability to business changes or ideas. This is why the system users needed yesterday never gets used when they have to wait six months to get it.

Long-and-dirty solution: The waterfall methodology forces the solution to be implemented with a design that seemed appropriate at the beginning of the project, but usually ends up being either over- or underengineered, resulting in poor performance and excessive maintenance costs.

Stuff that we rightfully get blamed for: Common mistakes in the engineering process, such as misestimating, shortcutting, or de-emphasizing quality, are accentuated by the waterfall methodology.

Hate/hate relationship: The "bond" formed between IT and business stakeholders based on an accumulation of tensions that result from project difficulties and subsequent failures. This adversarial relationship usually further degrades from project to project, exacerbating the issues caused by the waterfall methodology.

A development cycle with shorter repeated iterations. Implementing aspects of the system followed by requests for additional user feedback or requirements repeatedly will result in fewer deviations between the user's vision of the system and its actual implementation.

Ability to focus on high-priority functionality. Agile methodologies provide more opportunities to shift a project's direction and emphasis to stay in concert with ever-changing business needs.

Forced revisiting of past work for quality improvement. Effective developers not only develop a system that meets today's feature requirements but also design for ease of maintenance and extensibility. Often within a waterfall project with a looming deadline, a developer sacrifices future maintainability in order to implement the requested features. The iterative model encourages devoting time during the iterations to improve the quality of what was developed during the previous iteration, resulting in more stable and easier maintained systems.

Project outcomes have less variability. Using an iterative methodology means fewer "I hope a miracle occurs to bring this all together in the end" pits in the stomach because the process is much more controlled. Misestimating by 20 percent during a four-week iteration is much less problematic than missing by the same percent in a nine-month waterfall project.

Logistics of stakeholder interaction. EXtreme Programming, by far the most well known of these methodologies, requires that the business stakeholders be in close proximity to the system developers so there can be interaction on a continual basis to prioritize new features and provide feedback. In general, assuming that the business stakeholder has a primary job outside of the project, this interaction is not always feasible.

Logistics of IT resource allocation. Many organizations have very specialized IT roles (such as a data modeler who works with the team during the design phase), and it would be difficult to allocate the diverse set of necessary resources with an iterative schedule where there involvement would be needed much more frequently.

Financial objections. Business users and financial analysts like to know what they are getting, and for how much. The iterative approach's quintessential response to this question is "you're going to get as many of the high-priority features that we are able to do in the timeframe and budget that are available to us." Needless to say, this usually doesn't go over very well with the CFO. The fact that these investments usually require ROI justification makes the financial objections even more germane. Additionally, a large project can be capitalized, meaning that the business can pay for it as a one-time expenditure and in most cases depreciate the solution's cost over time with less of a negative effect on short-term profits. In some cases, the expenses associated with a full-time staff that continuously develop a solution in an iterative fashion would be difficult to capitalize in the same manner.

Perception issues. Many of the issues that have tainted the use of iterative methodologies relate to perception. Many still believe that "eXtreme programmers" are college dropouts who say "dude" a lot and can't be counted on to be at work during high tide. However, these prejudices often lead to "It isn't for our company" dismissals without truly understanding the methodology's issues and benefits.

Effort required. The iterative approach requires a significant amount of effort and management because the timeframes are shorter. If a developer is out for a week during a four-week iteration, it is a big loss to the project. Many feel that the iterative approach is too demanding because it focuses on consistent workload rather than workloads with very high variability.

Cost of change. Finally, those who are convinced of the merits of agile methodologies typically quiver at the idea of trying to justify the cost and effort involved to make the transition. These hard and soft costs often include new training, new tools, different resource allocations, resource changes, and the loss of productivity during the transition.

Reducing projects' scope and team size to about one half of your current larger projects. Break up larger efforts into multiple projects.

Allocating 20 to 30 percent of the project budget to perform post release refactorings, stabilization, improvements, developer training, and documentation.

Budgeting like waterfall but developing in an iterative fashion. A good friend suggested this idea, which works well to allow projects to be prebudgeted, but still benefit from an iterative approach.

Doing more interim releases and demos in an iterative fashion. Enhance your ability to get early feedback and raise your team's confidence in meeting deadlines by releasing on a frequent basis. Much like completing a project, the most satisfying culmination to an iteration is in deployment — even if it's to only a subset of users.

Staffing the projects with dedicated resources. Resources allocated as part time contribute little and often detract from progress.