The Waterfall model is originally invented by Winston W. Royce in 1970. He wrote a scientific article that contained his personal views on software development. In the first half of the article, he discusses a process that he calls “grandiose”. He even drew a figure of the model, and another showing why it doesn’t work (because the requirements always change). This model is the waterfall. He used it as an example of a process that simply does not work. In the latter half of the article he describes an iterative process that he deems much better.
OK, so why do people still advocate the waterfall? If you look at the scientific articles on software engineering that discuss the waterfall, they all cite Royce’s article. In other words, they’re saying something like “The waterfall is a proven method (Royce, 1970).” So they base their claims on an article that actually says the opposite: that the model does not work.
This is how science (unfortunately) often works – researchers just cite something, because everyone else does so as well, and don’t really read the publications that they refer to. So eventually an often cited claim becomes “fact”.
Agile methods and the linear method were both originally used already in the 1950’s. The scientific mis-citing boosted the waterfall’s popularity, but in the 1980’s it was being phased out – because people had found out (the hard way) that it does not work.
But, things change as a busy engineer in the US defense organization is asked to come up with a standard for military grade software projects. He doesn’t know what a good process would be, and he’s told that “Hey, Royce already came up with the correct method: the waterfall. Use it.” So the waterfall becomes the US military standard DoD-2167.
A bit later the NATO countries notice that they too need a software engineering standard. And they ask the US for their standard – no need in reinventing the wheel, right? And thus the waterfall is again used everywhrere, since if the military uses it, it must be good. Right?
Here in Finland we had already dropped the waterfall in the 1980’s, but after the DoD-2167 was sent over the Atlantic like a weapon of mass desctruction, it became the standard, again.
What have we learned here: The agile (iterative, incremental) methods are not new. They’re as old as the waterfall. There is no scientific evidence that the waterfall works – on the contrary, most projects fail. There is, however, massive evidence that agile methods improve productivity, quality and success rates.
So what should we really have learned: People remember images. And people often read the beginning of an article, but not the end.
So: Don’t draw figures or diagrams of wrong models, because people will remember them. And in the worst case that could cause hundreds of thousands of failed projects and billions of euros and dollars wasted for nothing.
Update (February 2010): Commenters have been asking for references to back up my claims, so I added links to the most central documents referred in this blog post.
Great discussion of the Waterfall Model.
Great article! I was just studying the model “by heart”for an exam and I realized I should be more critical. Thank you very much.
So, basically, waterfall method is still widely used in the military?
What I’ve heard is that the US military has long since adopted empirical, iterative methods, but many of the European Nato countries’ militaries are still clinging on to the good old waterfall. But this is just hear-say.
And the waterfall does have some benefits: it’s easy to understand, budget and schedule. Unfortunately it’s basically an idealized model that doesn’t reflect reality except in very rare cases, where the software production is more like mass production (the client knows exactly what he wants and can describe it unambiguously, and the developer knows the technology perfectly, and makes no mistakes in the design or implementation of the application). And software development very rarely is that. I’d rather compare it to innovation, where you need to come up with a new product. Innovation never follows a predetermined path (like a waterfall), but is littered with odd findings, surprises, jumps and setbacks. And all other industries know this and use empirical lifecycles (even car manufacturing lines are empirical, adjusting the robots’ actions to each car based on the differences in the individual car, not to mention car model design processes), except software engineering, which is still stuck in the 40s.
Great discussion!
Think you already know this statment: http://www.idinews.com/waterfall.html
Thanks for the link, Karsten, I hadn’t read it before. I don’t agree with all Conrad’s points, though. He is right in saying that the waterfall doesn’t exist, since Royce did not use that name, and did not advocate its use. However, even though Royce said doing a strict phase-by-phase lifecycle doesn’t work, researchers still started to cite him when talking about a strict phase-by-phase method. When the waterfall name came from, I don’t know. But it’s a real name for a real model that has been in widespread use for decades. It’s a castle floating on air, since there’s no research to back it up (and most current research actually shows that it does not work).
I do know that there has been a lot of software development using an inflexible lifecycle model, and there are even currently lots of prominent consulting companies that advocate the waterfall, like Accenture. And most of this development has ended up in failed projects, or projects that have overdone their budgets and schedules 10 times.
What Conrad is talking about is flexibility. And that’s what iterative development is all about. But I don’t think it’s enough to just let the customers make a “rational choice based on the costs and expected benefits of making or not making any non-trivial change to previously accepted results”, but in addition change the working practices so that making changes is as easy and cheap as possible. Because unless you deep-freeze your customers, their requirements will always change. Therefore allowing for maximal flexibility is something that should be aimed for.
Thank you very much. Talk about a breath of fresh air!
Scott Adams take note: where is your cartoon about this? Don’t just rest on your laurels, boy!
And yet we are still taught waterfall models in university courses.
[...] Et pourtant people still believe in waterfall [...]
[...] One thing I was just pondering is that if one works for the goal of making a ‘complete’ spec, one is assuming a waterfall model of development (Wikipedia: Waterfall model, Why people still believe in the waterfall model), which is bad. In real life, a spec is a sort of snapshot of a continuing process, which is therefore not ‘complete’ (unless your philosophical orientation says that a thing is always complete in itself by its own definition). [...]
Bravo. Congratulations on your independent rediscovery of the fundamental truth taught in any Mass Communications 101 class. This would be that the title of an article matters more than its content, that the first paragraph matters more than the second, and that the last matters not at all.
And if you’ve got a political bent then you should know that this is how right-wingers propagandize in the media, by putting all the conventional pro-authority, pro-right wing, assumptions in the first half of the article, and then making the article “fair” by pointing out what a load of crap they all are in the second half of the article.
This is how science (unfortunately) often works – researchers just cite something, because everyone else does so as well, and don’t really read the publications that they refer to. So eventually an often cited claim becomes “fact”.
I think it’s hilarious that you say this with no citations, for that quote or the entire article.
If you can figure out a way to build software (or anything else) without (1) discovering what is needed, (2) determining out how to build it, (3) building it, and (4) checking to see whether it works, more power to you. All the “waterfall process” really is is a statement that these practices might be applied in this order. It’s certainly “grandiose” to expect that all of this will flow perfectly downhill on a very large project without tight controls. For smaller projects, though, or for organizations where these stages are tightly controlled by management and contracts, this is not the world’s worst software plan. It’s probably much better than no plan at all.
Some “agile” methods achieve their agility not by reordering, interleaving, incrementing or iterating these stages, but by more-or-less skipping one or more of them. The most likely victim is (1); the results usually aren’t too pretty. I’m not aware of the “massive evidence that agile methods improve productivity, quality and success rates.” In my world, such claims are best accompanied by citations.
I’m a big fan of lightweight process, agile methods, and open source development. But I think it’s a bit harsh to believe that folks have adopted the waterfall model just because Winston Royce wrote an article mocking it. I think it has a lot more to do with the conditions of the environment that produced the “waterfall mandate”.
Models are simplified representations of the world or doing certain things in the world. Many abstract details yet important details, such as changing requirements, which contribute to the complexity in the ‘real world’ are left out to serve the purpose of making a model simple enough to understand. There are only so many variables you can throw into a model before the average person starts getting confused. Like other models such as iterative development, the waterfal model has its drawbacks as well as useful insights.
What I question is whether simplified but inevitably imperfect models are the reasons why things go wrong as often as they do or if the real reason might lie with misguided applications of models and approaches in inappropriate contexts and ways, by those who may be less than discerning or critical.
Не рисуйте неправильные модели…
Модель Waterfall development была изобретена Винстоном Ройсом в 1970. Он написал научную статью, которая содержала его личное видение разработки про….
You need to consider GETTING PAID !
The “iterative process” makes one assumption that may be _invalid_ and that assumption is that the intended users of the result are willing to pay the cost to have it developed!!!
What does the “waterfall” have that the “iterative” does not? It has a means to provide and estimate BEFORE the work is done!!! With the “iterative process” the user-community/customer/project manager does not have any way of knowing how much the development will cost….. until it is finished (which it NEVER IS).
So, the “iterative process” makes the (socialistic) assumption that everyone is entitled to their job.
“Waterfall” works in capitalism, ‘iterative process” serves to justify socialism.
I think that many models are needed for the differing needs for the development of a piece of software. There are many more teams than the engineers and each one of these teams might have a need for a different diagram. So I would say, be careful to show the right diagram to the right people. After all software development is more like baby sitting than building a building.
How can you deliver a product whose requirements are constantly changing? Under this assumption, a delivered product will never meet its requirements.
We used “waterfall” methodology at my old defense company and it worked beautifully. The resulting code and hardware was well-structured, well-tested, and elegant.
Great Discussion . strange thing is that i gave a talk on Fall Of Waterfall method last week at BarCamp Bangalore. and point in my talk resonant from what you highlighted here in your post . i guess main problem with waterfall is stems from the fact that often project manger and designer are reluctant to admit that they started a project with out fully understanding the requirement . so to play on the safer side they tends to generalize the design to the extend they became the proverbial Tat Pit Of “Mythical Man Month” . but they also have maintenance releases after the first commercial relase so the claim that they don’t do evolutionary prototyping is false . what they (Most of The time) can’t do is to fix a design level goof up /sub optimal choice in post release bug fixes . thats the reason folks at long -long dev cycle and substantial end of the cycle integration effort.
but Agile , XP ,Scrum starts with a point of weakeners that they require highly skilled workforce so you can’t scale . in some venture backed by VCs you need to have some milestone to deliver to maintain cash flow so we can’t keep on re factoring and prototyping . even though i like Agile methods a lot ,still i sometime feel that it is out of sync with Market Realities , scale and revenue . like other methods it too , study dev process in silo from big picture . still i love it for the collaboration and design flexibilities it offer
Thanks for a Great Post . i am regular to your blog since a year or more . discovered it from digg i guess . your post “what i learned from Failure ” is one of my fave reference for Sw project management . keep up the good work .
Thanks everyone for your critical comments on my bashing of the waterfall. Of course, no generalization is ever completely true (including this one
so all truths should be taken with a “grain of salt”. The waterfall certainly has its uses: if the requirements won’t change at all (which is quite rare in my experience) and if the developers know the technology well enough (so that there will be no surprises and the project is actually quite routine), then you can actually get the waterfall to work in a single iteration. But most of the time this is not the case.
And certainly the waterfall is better than no model at all. But the problem with the waterfall is that budgeting and scheduling is done with the most optimistic estimate possible (we’ll be successful on the first try), while in reality you do need to fix your high level designs, make minor changes in requirements, and will run into a few nasty surprises during development. So as a comment to Keystroke, I’d say that while a waterfall gives you an estimate, it’s not the most likely estimate, but the most optimistic one. Quite often it is common practice to do the estimate as a waterfall and then just add 30-50% to the schedule. Where’s the accuracy there? Why use a method you don’t trust yourself?
If you want comprehensive project management and agile makes you feel uneasy, just opt for one of the non-agile iterative methods, such as the Rational Unified Process, which is quite heavy, but explicitly iterative, and will more closely reflect the reality of software engineering.
As I said before, one of the biggest advantages of the waterfall is its marketability – every manager will be able to understand how it works. Iterative methods are more complex and will require more skillful managers to get them to work properly, but they will give better estimates and better change control. And agile iterative methods will require courage as well, since the managers are giving away much of their illusionary control of the project and they will need to trust their developers more. However, it’s a funny thing that empowering developers will motivate them, and motivation counts for quite a lot of the efficiency of software development.
Written to: Randy Yates.
The problem here is that you’re asking the wrong question. The question your asking one which doesn’t address a more serious question: How can you get the correct requirements the first time?
In the defense company you probably worked for an unusuall customer: One who knows what they want.
There are groups that have this sort of customer. But most groups don’t. Most have either a vague consumer they’ve never met, or a customer who isn’t sure what GUI means. Communication obviously becomes difficult, and the customer has likely thought very little about their requirements.
People seem to be missing the problem with waterfall on here: It’s that the phases are frozen when done. You get reqs, and stick to them. But customers change their minds, the problem domain expands, the problem domain becomes better understood, and etc.
Suddenly when you do things like come back and revise phases you no longer have the waterfall model (water doesn’t flow uphill you know). You’ve got iteration, spiral, or some variation.
I think the single greatest problem with the waterfall model is that when you think you know the problem domain, it will probably turn out you didn’t.
And I think it’s popularity stems from its cost: It’s cheap. You have a few really nice business guys talk to customers. Then some really bright design guys design the software. Then you pay some grunts to write it, except for the parts those bright guys do, then you pay for testing, ship and start again. None of this reinventing your code multiple times stuff.
But sometimes more expensive things are worth their cost.
[...] Et pourtant people still believe in waterfall [...]
Tamo, it’s a good article. I’d suggest that the every vaguely intelligent developer knows that the waterfall is a lousy way to make good software, but it survives because it’s the easiest way to interface the software development process to the outside world. That’s the tension. Agile makes better software but is hard to fit with the process that the customer expects.
In this sense, software as a rented service might be helpful. If I rent a hosted service from a SaaS provider, I can enter into exactly the kind of maintenence oriented, incremental model of development which will produce better software.
Randy, because, yeah, defence contractors are paradigms of efficient, value-for-money, production right
The reason this stuff is so prevalent in defence is because defence is the least scrutinized, least accountable industry on earth. As a branch of government they aren’t responsible to the market or competition. And under the cover of “national security” they aren’t responsible to public, political oversight either. Why should they use a good methodology?
To be fair, this kind of system is one where you probably want to trade efficiency (or satisficing) for reliability. And there’s an expected cost and redundancy that comes with that. But even so …
For all the nebulous talk about Agile development, noone has ever shown me how it is in any way is superior in *my* industry. I build test sytems. People say build me a tester for this part. I say how? They tell me, I then sit with them for days and weeks determining exactly what will test it they way they want. If they don’t know, then we do experiments at their cost (I suppose you would call this ‘iterative’ development, I call it requirements gathering). Eventually we know how to test it, I then create a high level design with screenshots, databases, configuration files, etc… that I show the customer. During this process, bad requirements become clear and we refine them (this breaks the orthodox waterfall method of having perfect documents at each stage, but I’ve never heard anyone seriously advocate that as anything other than theory), we also tend to identify risky items here and do some testing to address them (”does this thing ‘really’ put out a 60hz signal when 5V is applied to it?”) again, this may be seen as ‘iterative’, I call it risk mitigation. I then create a low level design, in the process I may expose questionable requirements or design issues and I work up the chain as need be. Finally comes, coding (an automatic process due to the extensive low level design), testing, and acceptance. If a customer throws a fit at this stage, that’s tough, we agreed to all these things together, there are no surprises, at times when there have been surprises it’s due to improper risk mitigtation (not checking that a part actually does what we assume)
In my estimation, all this talk about alternative ways of developing software is really dancing around one big topic: Requirements are hard to define, Design is also a little tricky. We really need to be discussing how to get better at doing those activities (e.g. Rapid prototyping is a tool for better understanding requirements). All these other methods are tools to deal with the difficulty of the wicked problem which is getting the correct requirements and a good design. I think people have lost site of that and it leads to things like talking about ‘throwing out’ the waterfall method. Which is really a way of engineering anything that is as old as the wheel.
In the end, if you don’t know what you want, you can’t design it, if you don’t have a design, you can’t build, if you don’t build you don’t have anything to test, if you don’t have anything to test, you don’t have anything to release.
[...] Psyc+Tech » Blog Archive » Don’t draw diagrams of wrong practices – or: Why people still believe in the Waterfall model [...]
Thanks a lot for your sharing.
By my experience, whether a model works is not dependent to its productivity but its popularity. When oldies discuss a project in terms of Waterfall Model, you can’t join the discussion till you use the same terminology as well.
If you draw diagrams in UML, your oldies boss will shout at you: “Have you learnt Waterfall Model in school?” Even if you retort saying: “Waterfall Model does not work!”, he will challenge you by “don’t bark by your textbook. Waterfall Model is an industrail standard. Do it or leave!”
It’s a typical example that a wrong thing will become right when most people believe.
[...] Currently, commercial software companies often follow something akin to a waterfall model and treat features as long-term work items and bugs as short-term (as an aside, the original inventor of the waterfall model didn’t believe in its effectiveness). This is a risky game to play if your competitor can implement new features before your model allows you to. It’s likely that as commercial software becomes under increasing pressure from open-source competitors, often using nimble development methods such as agile programming, commerical software production companies will need to turn around production of new versions with new features faster than before to survive. [...]
Here’s another interesting angle. If your company uses a waterfall methodoloy and your client prefers iterative, how do you sell the waterfall? And try to keep your job!
Sometimes I feel that all these methodologies reflect the changing times rather than any particular way doing things. We, as human beings, are good at observing and learning. We observe how other industries manage projects in their areas and try to apply them to our industry after analysing the pros and cons of their approaches. Hence there is no such thing as a bad methodology. The methodology should be able adapt to the project objectives defined at the time of defining the project. The three things – strategy, process and technology – are constantly changing and need constant adaptation of the methodologies. The global workforce has resulted an absolute 24 work hours in 24 hours of the day due to developments in technology. Proceses are continuously changing to reduce the wait times. Strategies adopted by large corporations will constantly change just to keep the business going. Hence the methodologies must be mindful of these three things at all times to become a good methodlogy that works!
Ajjanpur, I believe like you that agile methods represent a change in times. We are evolving from “Tayloristic” management – managers separate from and assumed smarter than workers. Finally, we are seeing the days when folks closest to the work are allowed to make decisions about the work, and in some cases this renders management ineffective. It’s very basic: those who can adapt will survive. That is true with life species and in business ecosystems as well. I believe that Agile is the next shift in management style, and certainly not the final innovation. The cool part about Agile is that it supports innovation at the three levels you cite – strategies, process and technology – guaranteeing adaptation and success in the long run. We are experiencing history as it is made.
I stumbled on this site while looking for research on the effectiveness of the waterfall model and of CMM (Capability Maturity Model). I had assumed for years that everyone knows the waterfall is a poor practice and I recall a study that found no correlation between CMM level and software quality.
So, imagine my surprise, after starting a new job, to find myself in a large organization where the waterfall is accepted as the default model for software development. I’d like to marshal some arguments to counter this assumption but would prefer to base them on empirical evidence – that’s just the kind of guy I am.
Can anyone point me to any studies addressing the effectiveness of either of these? Of course, I’m open to all valid evidence, pro or con.
Sorry but your own opinion is biased as well (surprise surprise). On the one hand you complain about the inadequacy of scientific citations about the waterfall model (”researchers just cite something, because everyone else does so as well”), which is probably true but on the other hand you fail to acknowledge that Royce himself has written his paper on a scientific basis. He is building the model gradually, resulting in fig.10. This is not surprising at all due to the date the paper was released. Royce gives an insight of his reasoning how he came up with the last model instead of just saying “Hey, tonight I had this dream, check figure 10 (you dont need figures 1 to 9 though)”. He does not claim at any point in the paper that someone should use the “simplified” model, he does suggest however to use the one on the last picture. So he didnt use the “simplified” waterfall model as a basis to show “a process that simply does not work” but as a basis to further improve the process he started to develop (starting simply with analysis/coding on page 1). So its distorting to conclude that he built a model just to show it doesnt work (implying a deliberate seperation of the paper in two parts). Another example for your distorting view is that he does not call the process of fig.2 “grandiose” but “MORE grandiose” (than the previous one) which, of course, changes the meaning significantly.
I know you will probably disagree, which is the nature of such a debate
Thomas, I’m sorry to disappoint you by agreeing with you. I should really reread the article with more thought. I appreciate your insights into the article, and your interpretation is most likely more correct than my view, which was aimed to be a bit provocative. This is the problem in popularizing science – you need to cut corners and simplify, otherwise your message gets hidden underneath all the details.
But maybe now in the grand new blogosphere there are people who actually can take the time to study an issue in-depth. Maybe I’ve been working in the printed press for too long and learning to simplify towards my biased message. Sigh… Unlearning is always such a drag.
Back to your comment: You’re right in that I’ve drawn a distorted conclusion from the article. Instead of saying that the linear model was “just to show it doesn’t work” I should refer to it as something like “an intermediary step towards an iterative model, which he considers to be the one people should use”.
But the essence of my message still stands, I think. The decades following Royce have seen the emergence of the waterfall method, and most articles about it refer to Royce 1970. And for Royce this linear model was just step 2 (”more grandiose than step 1″) in his 10 step introduction to refining your software development process.
[...] The story goes that somehow Royce’s initial diagram of a process which did not work was adopted as what we now know as the ‘waterfall model’. And was later adopted as a US Department of Defense standard (DOD-STD-2617) in 1985 and shipped across Europe in to Government and Defence projects though NATO. [...]
Hi,
I disagree with the following statement: “This is how science (unfortunately) often works – researchers just cite something, because everyone else does so as well, and don’t really read the publications that they refer to. So eventually an often cited claim becomes “fact”.”
I would like to say that this is generally not how science works. Claims(predictions) made by any scientific theory is verified by multiple laboratories/people over a period of time and they are always falsifiable.
The problem with software development is that it is not based on a scientific theory yet (I am not talking about computer science here, only softwarre development). It has been said many times that it is more of an art than science.
[...] and the waterfall method (again) Tags: Software development, Technology, waterfallI originally discussed the waterfall method in a previous blog post. Pranav just made a comment that I feel needs some further discussion: I [...]
[...] tässä yhteydessä vesiputousmalli siten, kun se on linkatussa wikipedia-artikkelissa selitetty. Tarmo Toikkanen kertoo kiinnostavasti, kuinka malli julkaistiin alun perin esimerkkinä toimimattomasta prosessista, mutta [...]
“What does the waterfall have that the iterative does not? It has a means to provide and estimate BEFORE the work is done!!! With the iterative process the user-community/customer/project manager does not have any way of knowing how much the development will cost?.. until it is finished (which it NEVER IS).”
Not quite – there is a stage where a project becomes ‘good enough’ for development to stop.
But the real flaw in the thought process behind waterfall is this – assuming it is worth spending several times the amount of money extra to know in advance how much money you are going to spend. This alone is enough reason to go agile, but when you consider that the method consistently leads to failure, well…
I wonder if people who like this model would also spend $150 for an extra 1 year warranty on a $200 DVD Player?
I love agile. Been working on a agile project for the last 3 months. No one knows what the product is supposed to do, beyond a handful of glittering generalities, but we have six people currently working on use cases for a logon screen (two weeks so far). Last project was a waterfall. We designed hardware, wrote 15,000 lines of code, built, tested, and delivered a customer certified product in 6 months. Think I will get back to writing more use cases. Less stressful, same pay.
[...] After Tianamen 1989, Seattle 1999 and Manila 2001, this #myyrmanni case was the very first what kicked me off: is there some kind of pattern with these cases, or inside the cases some phases like project-based coordinative actions or so? For example, Tarmo Toikkanen has the brilliant blog marking about waterfall project model and its quests. [...]
My problem with this article is it lacks references to support claims. I’ve heard for years that waterfall methods fail regularly and agile rarely fails, but nobody has data supporting the claim, or a definition of what constitutes success and failure. As a manager who has used multiple methodologies including waterfall and agile, I can say that the methodology needs to fit the life needs of the product being developed, and also that a lot depends on how well a methodology is executed. Hardcore waterfall might be viewed as tyranny. Unfortunately, many shops actually follow the extreme anarchy approach of just code and test. In reality, an iterative approach like agile is almost always used that falls somewhere between anarchy and tyranny.
Hey thanks a lot for the article! Sounds like the Waterfall is what we want! Agile and Extreme sound scary. Waterfalls are nice.
It’s common (sense) that in communication you have to avoid the negative tense of a message. Because the only part the audience will remember is the ‘do’ part instead of the ‘do not’ part. If you say: “There is not an pink elephant in the room”, people will picture a pink elephant in the room and try to get rid of it in their mind. This scaping of a virtual mindset is very dangerous if the audience is not able to get rid of the view that is created. So, don’t try to picture something if you have to remove the same mindset later on to convince people of your message. Especially if the audience loses concentration and forgets the last part of your message to get across. You never get a second chance to a first impression is a very very good example of this principle.
- Unomi -
[...] Shared Don’t draw diagrams of wrong practices – or: Why people still believe in the Waterfall model. [...]
[...] Don’t draw diagrams of wrong practices – or: Why people still believe in the Waterfall model Wäre Peteris Krumins meine Mathelehrer gewesen und hätte ich mir damals mehr Mühe gegeben, wer weiss [...]
Regarding referencing papers without reading them – this sounds somewhat similar to hungarian notation in that what the paper said is not at all what people commonly believe.
Thanks for a great article.