November 1, 2007
The Wisdom of Crowds
In 2004, James Surowiecki wrote a book called "The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations."
The basic idea is this: if you quickly poll a bunch of people about what they think is true, the mean of their guesses is usually close to right. Surowiecki marshaled a lot of evidence to prove the point.
I tested this recently at a gathering of high level librarians. By that, I mean the folks who run state libraries, or are deans of university library systems, or manage multi-state library networks. I asked, "Which technological trend or idea, in your judgment, will have the biggest effect on libraries over the next 5-10 years?"
I didn't give them any time to think about or discuss it -- just respond.
Both to my surprise and theirs, three clear trends emerged.
The first was "open source library systems." For the past 20 years or so, the library automation market has been doing what markets do: compete and consolidate. Now, only about four main software systems remain. And all of them frustrate librarians.
Why? Because none of them is as good as the two big search systems that most people use most frequently: Google and Amazon.
That's a shame. Librarians were among the first information scientists, the first to use automated billing, the first to build comprehensive computer inventories, the first to seize the Web as a tool for public information.
But despite our early lead, today's library systems have fallen behind. They're clunky. They don't allow our patrons to post comments on our holdings. They don't consistently pull up the most requested titles for a search term.
We trusted to the commercial market, and it let us down.
Oh, and incidentally, both Google and Amazon didn't write all this software themselves. Nor did they buy it.
Instead, they used the "free" software created by programmers, then given to the world. The Linux operating system. The Apache web server. The SQL database system. And so on.
Meanwhile, some folks down in New Zealand used longstanding library standards to build another absolutely free system that does most of what the big commercial systems do. It's called "Koha."
It turns out that a lot of librarians are just sick and tired of paying for second generation systems when the rest of the net has moved onto the fourth generation.
Not surprisingly, the second big trend was open source software generally. "Open source" mean that the software is often both literally free, but also non-proprietary. That is, you have express permission to look at, and change, the underlying code. It's the wisdom of crowds.
Open source is moving into the mainstream, and a lot of librarians think that its deployment in the public sector makes a lot of sense. For a long time now, the public good has been held hostage to the business plan of people for whom "the public good" doesn't mean much.
Today, there are open source office suites (see Openoffice.org), open source browsers and email clients (Firefox and Thunderbird), open source databases (SQL), open source IM clients (Pidgin), and much more. It's not only free, it's good. And it's not only good, it begins to open a whole new world of international communication and collaboration.
The last trend was "the convergence of mobile devices." In brief, librarians believe that handheld devices, and the instant availability of high quality information, are bound to affect the way people use information.
I should point out, of course, that we know without question that the growth of information appliances and Internet ubiquity not only doesn't kill the desire for books, it seems to drive it up. But just possibly, getting the world's library in the palm of your hand is a worthwhile goal.
So what's my point this week?
Many people believe that there are just two kinds of markets in the world: for profit, and not-for-profit.
That's false. There's another market: an international meritocracy. That meritocracy is predicated on values that do a good job of capturing the purpose of the public library.
It might just be that the next wave of library technology will come not from the proprietary world of venture capitalists, but from the world of open source.