Tuesday, December 14, 2010

The Cathedral and the Bazaar: Excerpts

Book Link: http://www.redhat.com/support/wpapers/community/cathedral/

1. Every good work of software starts by scratching a developer's personal itch.

2. Good programmers know what to write. Great ones know what to rewrite (and reuse).

3. "Plan to throw one away; you will, anyhow." (Fred Brooks, "The Mythical Man-Month", Chapter 11)
Or, to put it another way, you often don't really understand the problem until after the first time you implement a solution. The second time, maybe you know enough to do it right. So if you want to get it right, be ready to start over at least once.

4. If you have the right attitude, interesting problems will find you.
But Carl Harris's attitude was even more important. He understood that

5. When you lose interest in a program, your last duty to it is to hand it off to a competent successor.

6. Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
The power of this effect is easy to underestimate. In fact, pretty well all of us in the free-software world drastically underestimated how well it would scale up with number of users and against system complexity, until Linus showed us differently.

7. Release early. Release often. And listen to your customers.
Linus was keeping his hacker/users constantly stimulated and rewarded -- stimulated by the prospect of having an ego-satisfying piece of the action, rewarded by the sight of constant (even daily) improvement in their work.

8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.
Or, less formally, "Given enough eyeballs, all bugs are shallow." I dub this: "Linus's Law".
"Debugging is parallelizable". Jeff observes that although debugging requires debuggers to communicate with some coordinating developer, it doesn't require significant coordination between debuggers. Thus it doesn't fall prey to the same quadratic complexity and management costs that make adding developers problematic.

In practice, the theoretical loss of efficiency due to duplication of work by debuggers almost never seems to be an issue in the Linux world. One effect of a "release early and often policy" is to minimize such duplication by propagating fed-back fixes quickly.

9. Smart data structures and dumb code works a lot better than the other way around.
Fred Brooks, Chapter 11 again: "Show me your [code] and conceal your [data structures], and I shall continue to be mystified. Show me your [data structures], and I won't usually need your [code]; it'll be obvious."

------------------
Four steps to open-source software development:
   1. I released early and often (almost never less often than every ten days; during periods of intense development, once a day).
   2. I grew my beta list by adding to it everyone who contacted me about fetchmail.
   3. I sent chatty announcements to the beta list whenever I released, encouraging people to participate.
   4. And I listened to my beta testers, polling them about design decisions and stroking them whenever they sent in patches and feedback.

--------------------
10. If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource.

11. The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.
Interestingly enough, you will quickly find that if you are completely and self-deprecatingly truthful about how much you owe other people, the world at large will treat you like you did every bit of the invention yourself and are just being becomingly modest about your innate genius. We can all see how well this worked for Linus!

12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.

13. "Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away."
When your code is getting both better and simpler, that is when you know it's right.

14. Any tool should be useful in the expected way, but a *great* tool lends itself to uses you never expected.

15. When writing gateway software of any kind, take pains to disturb the data stream as little as possible -- and *never* throw away information unless the recipient forces you to!

16. When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
Another lesson is about security by obscurity.

17. A security system is only as secure as its secret. Beware of pseudo-secrets.

----------------------------------------------

It's fairly clear that one cannot code from the ground up in bazaar style. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode. Your nascent developer community needs to have something runnable and testable to play with.
When you start community-building, what you need to be able to present is a plausible promise. Your program doesn't have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is convince potential co-developers that it can be evolved into something really neat in the foreseeable future.

I think it is not critical that the coordinator be able to originate designs of exceptional brilliance, but it is absolutely critical that he/she be able to recognize good design ideas from others.

But the problem with being clever and original in software design is that it gets to be a habit -- you start reflexively making things cute and complicated when you should be keeping them robust and simple.

-------------------------------------------------
18. To solve an interesting problem, start by finding a problem that is interesting to you.

In "The Mythical Man-Month", Fred Brooks observed that programmer time is not fungible; adding developers to a late software project makes it later. He argued that the complexity and communication costs of a project rise with the square of the number of developers, while work done only rises linearly. This claim has since become known as "Brooks's Law" and is widely regarded as a truism. But if Brooks's Law were the whole picture, Linux would be impossible.
A few years later Gerald Weinberg's classic "The Psychology Of Computer Programming" supplied what, in hindsight, we can see as a vital correction to Brooks. In his discussion of "egoless programming", Weinberg observed that in shops where developers are not territorial about their code, and encourage other people to look for bugs and potential improvements in it, improvement happens dramatically faster than elsewhere.

That is, that while coding remains an essentially solitary activity, the really great hacks come from harnessing the attention and brainpower of entire communities. The developer who uses only his or her own brain in a closed project is going to fall behind the developer who knows how to create an open, evolutionary context in which bug-spotting and improvements get done by hundreds of people.

19: Provided the development coordinator has a medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.
I think the future of free software will increasingly belong to people who know how to play Linus's game, people who leave behind the cathedral and embrace the bazaar. This is not to say that individual vision and brilliance will no longer matter; rather, I think that the cutting edge of free software will belong to people who start from individual vision and brilliance, then amplify it through the effective construction of voluntary communities of interest.
Perhaps in the end the free-software culture will triumph not because cooperation is morally right or software "hoarding" is morally wrong (assuming you believe the latter, which neither Linus nor I do), but simply because the commercial world cannot win an evolutionary arms race with free-software communities that can put orders of magnitude more skilled time into a problem.




--
prasanth prahladan
( ~jprah~ )