Linus Does Not Scale

One of the darkest moments in the history of free software occurred in September 1998. For perhaps the first - and one hopes the last - time the Linux kernel came perilously close to forking.

The problem was simple: Linus had become a victim of Linux's success. He was unable to cope with the volume of patches that were being sent to him. In the memorable words of Larry McVoy at the time, "Linus does not scale."

That scaling problem was solved by working on a better version control system (what became BitMover, later replaced by the memorably-named Git), as wll as handing off some of Linus's work to others. In the case of the kernel, this could be achieved by mutual agreement, but more generally it is hard to divide up a task among many contributors.

There are now several sites that have sprung up to address this problem. One of them is Amazon's Mechanical Turk, which I wrote about some time back, although I rather missed the key point, which is the use of distributed human intelligence to carry out those kind of tasks that computers presently struggle with. A more recent entrant is Mycroft, discussed in this C|net piece.

Also worth noting is the Crowdsourcing blog, which is a follow-up to the Wired article on the same (and doubtless a feeder to the inevitable book on the subject).

What's interesting about the crowdsourcing idea is that it represents a kind of open source without the openness: that is, participants are essentially computing drones with no way of knowing what the bigger picture is, unlike open source programmers, who can always look at the code. In a sense, then, crowdsourcing is a dilution of the idea at the heart of all the opens, but it's also a broadening in that it enfranchises more or less anybody with basic human processing abilities.

Update: And here's another crowdsourcing blog, called, aptly enough, Crowdsource.