Dan Gillmor's got a great post on what's wrong the the major newspapers' approach to their Web archives. I've long been mystified by the way the newspapers have approached the Web. Papers like the New York Times have decided that their archives — which were previously viewed as fishwrap, as in "today it's news, tomorrow it's fishwrap" — are their premium product, the thing that you have to pay to access; while their current articles from the past thirty days are free.
The thing is that while there is certainly a small commercial audience for newspaper archives — corporate researchers, the occassional grad student with a grant — the noncommercial audience for archives is much larger: people who want to read the news from their birthdays, researchers amateur and pro looking up historic dates, Bloggers writing about seminal moments.
Conversely, there is a large commercial audience for new news, that is, people who'll pay to see today's news while it's still news and before it becomes history. That's why the news business is so much larger than the history business.
The problem with the NYT's system is that it ensures that the Times can't be the paper of record any longer, because even if a thousand bloggers point to a great article on the day it comes out, thirty days later it will be invisible to the 99.999 percent of the Web who won't pay for access to fishwrap, no matter how interesting.
News is increasingly a substitutable good: there are so many ways to get the basic facts on an article, from Yahoo's AP wire to the Sydney Morning Herald to pastebombed articles in the archives of mailing lists like Interesting People and Politech that a savvy searcher or blogger has no good reason to pick the NYT to get a set of basic facts on any subject. The NYT often does an extraordinary job of covering the facts, but it doesn't matter a whit to posterity if a link to that job will staledate in a month.
If the NYT can't make it on advertising alone, it might just be dead in the long run, since these substitutable goods that require no subscription will crowd it out of the market eventually. But if it wants to try a subscription-based system, then for heaven's sake, why not charge money for the news (which lots of people want to pay for!) and give away the history (which relatively few people want to buy)?
There was a Wired News article a couple months back that suggested that the paywalls on newspaper archives were being driven by their agreements with Lexis-Nexis, a company that provides expensive search services to newsrooms, lawyers and other specialized entities. I think that protecting the Lexis-Nexis deal at the expense of relevance is the wrong move: if the NYT's need to lock up its archive makes it irrelevant, then Lexis-Nexis will drop its contract with the Times anyway. Meanwhile, if the NYT puts takes its archives out from behind the paywall, it won't need Lexis-Nexis to market them for it any longer — hell, maybe it can start charging Lexis for access to its database!
One of these days, a newspaper currently charging a premium for access to its article archives will do something bold: It will open the archives to the public — free of charge but with keyword-based advertising at the margins.
I predict that the result will pleasantly surprise the bean-counters. There'll be a huge increase in traffic at first, once people realize they can read their local history without paying a fee. Eventually, though not instantly, the revenues will greatly exceed what the paper had been earning under the old system. Meanwhile, the expenses to run it will drop.
And, perhaps most important, the newspaper will have boosted its long-term place in the community. It will be seen, more than ever, as the authoritative place to go for some kinds of news and information — because it will have become an information bedrock in this too-transient culture.
