Corrections to the Joel test of software development quality

I just listened to an interview of Joel Spolsky on It Conversations, and checked out his Joel Test. While Joel has good argumentation for his opinions, I think a different perspective is needed for a few of the steps in his test, especially concerning testing and user involvement.
Here’s my version of the Joel test. It starts with the same 12 steps (although modified more or less), but continues with a few additional steps that I consider important.

1. Do you use source control?
   This is too obvious to go into detail – even a one man team needs source control. It gives you history traceability, undo, coordination of simultaneous work, and backups. I personally use it even for all of the documents that I write.
2. Do you have automatic builds? (was: Can you make a build in one step?)
   The one step build that Joel wants is a good start, but I’d rather see an automatic build done every night, and a big fat red warning on the frontpage of the development site whenever something breaks.
3. Do you make daily builds?
   If you can’t build the software every day, you won’t build it until you need to. That means a convergence period before releasing, which is in fact just fixing technical bugs that could have been fixed when you wrote the code. So build automatically, and daily. And make each developer build and verify his code before he commits to source control.
4. Do you use a ticket system? (was: Do you have a bug database?)
   Again, an obvious necessity. In addition to bugs, tracking all enhancement and auxiliary tasks should be carried out in the same ticket system, especially if you’re doing distributed and/or dispersed development.
5. Do you prioritize bugs? (was: Do you fix bugs before writing new code?)
   As Joel says, bugs shouldn’t be left lying around, since they’re getting more expensive to fix as time goes by. But a black-and-white rule of “fix all bugs before doing features” won’t work – most bugs can be tracked to one developer, who should be the one to fix it (since he’s most likely the most efficient on that bug). Others shouldn’t wait, but continue with other tasks. Related to this, if you do frequent releases, then you usually have a one day convergence period, where all the remaining bugs are squashed.
6. Do you have an up-to-date schedule?
   No disagreement here. Having limits on what is done are needed. They discourage feature creep, for one. So timebox everything – two weeks between releases, for example.
7. Do you have a plan that you can change? (was: Do you have a spec?)
   Joel wants a spec. I don’t. In the majority of development projects, the customer cannot exactly specify what he wants. Thus instead of a spec, use a format that the customer also understands, like user stories. The collection of accepted user stories is the spec. But it’s a spec that can be changed by the customer, and that is refined as needed. Of course, you need development practices that support a changing spec, like automated unit testing.
8. Do you provide developers with conductive working conditions? (was: Do programmers have quiet working conditions?)
   Here I completely disagree with Joel. Most of the problems are best solved by more than one mind. The flow that Joel talks about isn’t necessarily an individual experience, but can even more easily be experienced by a pair or team of developers. Therefore: have all developers in a single room, preferrably facing the walls, with an open area in the center, so they can at any time turn around and talk to whomever. Of course, there are those people who will need 15 minutes to get their focus back on what they’re doing after being interrupted, as Joel says, but I wouldn’t want those people on my team.
9. Do you use appropriate tools? (was: Do you use the best tools money can buy?)
   Not the best money can buy, as Joel says, but appropriate. There is no added value in getting the compile done in 5 seconds instead of a couple of minutes. Having time to relax your mind and getting some distance to the code generally improves the result. Plus it’s critical to have time to reflect on the work you’ve done. Waiting a few minutes for the compilation or the unit test suite to run is a good time for all this. Plus it makes the code that much better, since the developers actually need to think their code through instead of just trying quick fixes and seeing if the compiler likes it or not.
10. Do you do quality assurance? (was: Do you have testers?)
    Not dedicated testers, as Joel says, but quality assurance. This can come in many forms, including dedicated testers, or by doing automated unit tests by the programmers themselves. Even Joel admits that good testers are hard to find and harder to keep, since testing is not good work. He argues that it’s still cheaper to use testers than have developers do testing, since testers have lower wages. I on the other hand argue that having developers write unit tests is cost-effective. Automated unit tests have several advantages: they make sure that any defect can only be created once (since once found, a test is written for it), they improve the quality of the code (not allowing laziness to cause bugs), and they allow you to respond to a changing spec by changing your architecture as needed. After the initial phase (after you have the first 50-100 tests done) the unit tests actually save developer time, instead of consuming it. Plus you don’t need as many testers as you otherwise would.
11. Do you only hire competent team members? (was: Do new candidates write code during their interview?)
    Instead of just making the potential new recruits write code, as Joel suggests (which is an excellent idea), it’s also important to let the development team meet the candidates and see how the chemistry works. Since development is a team effort, it’s best to let the team have their say in who should be hired.
12. Do you do usability testing? (was: Do you do hallway usability testing?)
    Joel’s suggestion of hallway testing is not enough, if the company is filled by developers. What you need are representatives of the focus group. No developer can say if a primary school teacher can understand a certain functionality, and few developers actually know the general usability guidelines that do exist. So using usability experts to iron out general usability issues, followed by a focus group test, where the vocabulary, approach, and priorities of the software can be tuned to the focus group, are necessary.
13. Do you do user-centric design?
    One reason Joel doesn’t like XP is the need for an on-site customer. In Joel’s opinion customers can’t provide you with good specs – they either say the obvious, or ask for features that they really don’t need. This is where user-centric design comes to play. Instead of asking the customer what he wants, find out what the customer needs. This is not an easy task, and involves a number of practices, one of which is a representative of the customer, who has the authority to make decisions, and the expertise to find out things. This representative is not enough alone, of course. You need to do participatory design, for example.
14. Do you have a heterogonous team?
    Unlike Joel claims, I don’t believe that a software developer can learn to understand the customer’s field in just a few hours. You need the customer’s expertise in some form. But you also need the expertise on the theoretical backgrounds and advanced technologies that are to be applied in development. It’s not enough to just have developers. You need user interface logic specialists, you need graphic designers, you need usability professionals, and most importantly instead of separate code drones and architects you need people who are excelled programmers, great architects, and that have some expertise in the field being worked on, all wrapped into a single human. Most of the failures in software development are due to miscommunication between the customer and the programmer. Misunderstandings are cut radically when you have more roles packed into a single mind. Not all of your developers need to have problem domain expertise, mind – it’s usually enough to have just one, because he can translate between customer-lingo and programmer-jargon.

In summary: This is just my experience of how people and code work. Joel’s test is an excellent resource for job seekers and managers, but in some aspects I disagreed so much that I had to write this response – Joel test as it is is biased away from some very good new practices that I consider a comptetitive advantage. Comments are welcome. And thanks to Joel for sharing his thoughts with us all.