A Never Ending “Virtual Stream” of Digitized Text
by Gary Price, Senior Editior

When Chris Sherman and I were writing and then giving book talks and presentations about The Invisible Web, we said John Mark Ockerbloom’s Online Books Page was an essential resource for anyone interested in digitized, full text books. Now referred by most as eBooks. More than eight years later I feel the same way about this awesome and well organized collection.

Where do you begin with a site so full of content? For me, that’s easy. Monitoring the latest additions to the catalog/page. I am always blown away by the amount of new listings (when does Ockerbloom sleep?) and the number of organizations digitizing books. If you think it’s only Google digitizing books (of course they are a major player) but not they’re far from the only one doing this type of work. Just look for yourself. The page even has an RSS feed.

So, the Online Books Page is not only a “must have” searchable directory of ebooks but it can also be a great collection development resource to find and add digitized content to your local collection/OPAC.

But wait, we’ve got more.

The Online Books Page new listings only includes some of the digitized text output from the Internet Archive (IA).

If you want to be able to review (at your leisure) all of the new digitized content text content that the IA produces, it’s possible by subscribing to this RSS feed. Even if you’re not going to review the titles, just let it run for a few days to see the AMOUNT of text material that’s digitized in variety of formats. It’s an understatement to say that the scanners at the IA are cranking it out on all cylinders. So, collection development types, subscribe to both RSS feeds and have a large virtual bookshelf to choose from each day. If you don’t do the collection development thing both feeds are useful to illustrate the amount of material being digitized each day, week, month.

UPDATE: Not an RSS user? No problem. Just visit this Internet Archive page and refresh it a few times a day. The most recent addition is at the top.

To avoid copyright problems the Archive has only digitized books that are in the public domain.  The full article from Xconomy Boston is available here.

The TeleRead take: BookServer, from Archive founder Brewster Kahle (photo) and colleagues, is a worthy project. I don’t want any company, Amazon or Google, to dominate book distribution.

Even so, the Archive will have to work hard to equal Amazon’s interface and its rich collection of customer-written reviews. Here’s an example of the issues at hand. Will BookServer capabilities be built into hardware e-readers, so to speak, the way the Amazon catalogue is part of the Kindle? Update, 12:32 p.m.: The existence of an RSS-style spec (draft here) is a good start, but that’s still not a full solution.

Might an Archive collaboration be possible with OpenInkpot—which provides software for users to install on dedicated e-reading devices? Imagine manufacturers including OpenInkpot from the start. Jeez, Brewster, you need to think more about the user experience—whether the device is an E Ink tablet, a commercial netbook, an OLPC laptop or regular desktop. This could mean anything from OpenInkpot to browser-plug-ins to dedicated apps.

There’s also the pesky DRM question. Will the master searcher provide detailed rights information, and what if publishers insist on DRM, which is anathema to Brewster? How to handle server-dependent DRM, or will such file be hosted on publisher sites?

On top of everything else, at least for now, there’s the downloading speed issue. When I tried to download an ePub file of a Dickens collection I found via the BookServe search box, it was taking forever. This was on an Archive server. What about files hosted on publisher servers, over which Brewster has no control?

I’d urge publishers to try BookServer, just as O’Reilly and Feedbooks are; but it would appear that this project is still very much in beta stage.

Q. Does this set back the cause of having full-text book content on the web? If Google stops its scanning efforts while this is renegotiated, how much time have we lost? What’s the up side?

No, in fact, I think it reinvigorates it. The efforts of the Open Content Alliance, the Internet Archive, Microsoft, Yahoo!, Google, and hundreds of individual libraries around the world have indicated the strong desire to have digital books online. We are witnessing an explosion of interest in digital content, new and old, and I think the discussions around the settlement provide an opportunity for us to consider how to engender a robust, competitive, and innovative market – and not just here, but in Europe as well.

Technorati Tags:
Google, Google book settlement, Paul Biba, TeleRead, Internet Archive

The TeleRead take: He and others such as John Mark Ockerbloom worry not just about the continued operation of valuable sites but also about the longevity of material posted to the Web. For example, to mention an Ockerbloom-cited example,  what if a site owner has cancer?  Or—closer to home—a heart attack, which I suffered last fall?

Roy himself runs an eco-related site, StanislausRiver.org, and notes that “I’m only one ill-considered walk across the street from annihilation. Should that happen, there is no clear path for any of my heirs to migrate this site into the hands of someone willing and able to take it under their control. It would die. Not immediately, but not long after my credit card no longer clears.”

Luckily Paul Biba would be around if a bus or fatal heart attack hit me, but as the operator of the oldest English-language site on general e-book news and views, I’m still far from smug about the current preservation situation.

The real answer

At a macro level, I’d suggest two important solutions—first, a well-stocked national digital library system that would archive more than books; and, second, the accelerated development of ePub and related standards for e-text. The longer it takes for standards to catch on, the more material may vanish. The move toward networked books, bringing in material from a number of sources, including Web sites, will just aggravate the problem.

Meanwhile,in case you’re curious, the Wayback Machine offers far from a complete preservation of TeleRead.org and an earlier site on the old ClarkNet. It archives just a fraction of our material. I wish Brewster Kahle all kinds of luck in changing this. Please note the existence of the Born Digital initiative from the Library of Congress, but like the Machine, its scope has been  limited so far.

For a while, the novel was posted free in its entirety on Vernor Vinge’s website. It has since been taken down; however, the Internet Archive still has it available in its entirety in the Wayback Machine’s archive of the page.

I’m actually surprised nobody reviewed it here back when it was newly published, but I can only find a few references to it on TeleRead. E-books—and some modern issues relating to e-books—actually play a pretty prominent part in the book’s plot, in a number of ways.

Vinge’s Other Work

Those who only know Vinge from his recent work prior to Rainbows End might be surprised at his return to the relatively-near future, more commonly the province of Cory Doctorow, Charlie Stross, and Neal Stephenson. After all, his books A Fire Upon the Deep and A Deepness in the Sky take place thousands of years in the future, when Earth is barely even a memory and humans have spread out across the galaxy.

But long before he wrote those, Vinge wrote another near-future book, The Peace War—and before that, his 1981 novella True Names helped to define the entire cyberpunk genre. With Rainbows End, Vinge returned to the near-future, adjusting and updating his predictions to fit the present-day.

(It is refreshing, by the way, to read a near-future extrapolation story for once that is not written in the present tense, or bombastic in the way that the authors like Doctorow can be. Vinge is old-school in his writing approach, which means I can appreciate his technological extrapolations all the more.)

The Plot

The story of Rainbows End is a third-person narrative woven together from the perspectives of several characters. The overall plot concerns an intelligence agency’s infiltration of a research lab that someone is using to perfect a workable mind-control technology—but unbeknownst to most of the intelligence agency, the mind-control mastermind happens to be the very man in charge of the search.

The infiltration plays out against a backdrop of fascinating new technology, culture clashes between fandoms, and protests against destruction of old media in favor of new. There are a number of memorable characters who get caught up in all these events.

The closest thing to a sole protagonist the book has is Robert Gu, an Alzheimer’s victim who is returned to lucidity by one experimental medical process, and returned to a youthful body by another. He lives with his son and daughter-in-law, Bob and Alice, and his grand-daughter Miri (another character of importance). Robert starts out as a fairly unsympathetic character—he used to be a real bastard, abusive of his wife and others around him—and it seems as though he is set to resume where he left off…until something changes him.

Since he was last lucid in our era, and has spent fifteen years or so in a mental fugue before being revived like Lazarus, he serves as a viewpoint character for modern readers. We learn about the startling technological advances in this new world just as he does, and we can sympathize with his future shock as he gradually grows accustomed to his new life.

Another important character is the mysterious Rabbit, a shadowy fixer (and trickster)from the Internet world who specializes in getting things done by getting people with complimentary talents together. His identity—indeed, even his very nature—is unclear, as are his motivations. However, he is a vital part of the plot to infiltrate the lab—whether the intelligence agency thinks it’s a good idea or not.

There are actually several different narratives within the story that interweave and cross over with one another in the most unusual ways. Characters watch others in secret, and exchange private messages to hold conversations behind each other’s backs. In the end, it all comes together in ways that are unexpected even by the participants.

Vinge’s Cyberspace

Rainbows End is an expansion and reworking of ideas found in Vinge’s 2001 short story, “Fast Times at Fairmont High” (found in The Collected Stories of Vernor Vinge). “Fast Times” features characters who would later appear in Rainbows End, though some of them have different names (or even entirely different species), and the plots are almost entirely unrelated. The technology is the same, however, as far as it goes.

However, Rainbows End also owes a lot to True Names. Both feature consensus fantasy realities constructed within computer networks. Both of these fantasy worlds have geographical correspondences to real-world locations—though True Names’s world is one that can only be observed from within while Rainbows End’s can be seen by anyone wearing special glasses.

Interactive Fiction: If You Build It, Will They Come?

And both True Names and Rainbows End also feature interactive “books”. The protagonist in True Names writes what another character refers to as “games” but he calls “novels”—apparently advanced multimedia versions of the text adventure games that were just reaching the height of their popularity at the time True Names was written.

Vinge has long held that the e-book medium of the future would be “interactive” in some respect. In 1993, when Vinge wrote an introduction for an “annotated” edition of his book A Fire Upon the Deep, he believed that hypertext (at the time, the newest new computer thing) was going to be the future of fiction:

I believe hypertext fiction will ultimately be an entirely new art form, as different from novels as motion pictures are from oil paintings. […] Guessing: There may not be hypertext sequels so much as the instantiation of new windows on the “reality” of the story. Group participation both during initial construction and in expanding the ongoing reality may be one of the most striking features of the art form. […] Hypertext fiction may evolve into immense art works that combine the essence of professional production teams with independent artists with the interests and efforts of the ultimate viewers.

This has, of course, never really come to pass. (And, as pnh noted in his interview, probably never will.) I always think of it as an example of the “if you build it, they will come” fallacy—projecting the popularity of something based on the ability to do it rather than the demand for it.

We can do hypertext fiction today very easily—but apart from a few Internet writing sites like Ficly or Writing.com, nobody seems to bother. Wikis would seem to fit best of all with Vinge’s vision for expanding upon someone else’s story, but there have not been many uses of them exactly like the ones he imagined. The closest things would be the forest of wikis that have sprung up to cover select interests or fandoms, such as the Avatar: The Last Airbender wiki, but these are usually reference rather than creative works.

On the other hand, there has been at least some serious study of the idea, as I found when searching TeleRead for another link. But it does not seem to have led to any commercial success outside of a few isolated experiments.

Belief Circles

With Rainbows End, Vinge’s conception of the interactive book of the future has evolved again, into a fully multimedia, virtual reality experience—though in most other respects it has a lot in common with his hypertext prediction quoted above. In Rainbows End, fans of published works create “belief circles”, which seem to be a mash-up of fanfic, Internet shared-universe writing circles, live-action roleplaying, and virtual multi-user environments such as Second Life.

In the virtual world that overlays the real world (and is viewed through displays embedded in contact lenses and controlled via sensors in clothing), fans of a given fictional or historical setting (members of that setting’s “belief circle”) create their own avatars and overlays for themselves and everything around them based on that setting. A skyscraper might be painted as a medieval tower, and cars might become horse-drawn carriages or low-flying magic carpets. Multiple different belief circles’ worldviews can overlap the same area; onlookers can switch between them like changing channels on a TV set.

For settings that are still under copyright, micropayments are charged to belief-circle members and credited toward the owner for each use of something relating to that world. (Though apparently there has been some copyright reform in Vinge’s cyber-fantasy world: at one point movies are said to have five-year copyright terms. I guess we can at least dream.)

One of the central conflicts of the book involves a clash between two belief circles over which one will dominate the University of California San Diego Library building—the “Dangerous Knowledge” setting in which librarians are knight-guardians of knowledge, and the Pokémon-ish “Scooch-a-mouti” children’s-fantasy-monsters setting. At one point the clash escalates into an out-and-out battle, with millions of viewers world-wide tuning in to see the outcome.

An Object Lesson in Obsolete Objects

The UCSD Library conflict actually grows directly out of the other aspect of the book of interest to e-book fans: the digitization of the contents of the library. In the timeframe of the book (sometime in the 2020s, apparently), physical books’ intrinsic value has declined to the point where the books themselves are considered much less valuable than their contents.

So, to get at the contents, a company is destroying the books themselves—feeding them through a shredder then blowing the shreds through a tunnel lined with high-resolution cameras. The cameras capture images of the shreds, then batteries of computers stitch them together into reconstructions of the pages, like jigsaw puzzles. The idea is to gather and collate all the world’s knowledge, to unlock synergies that had been prevented by it all being so inaccessible before.

Vinge wrote Rainbows End just as Google was beginning its own massive scanning project (which does get a mention in passing in the book), but well before the settlement with the Authors Guild (and the attendant controversy) was on the horizon. Thus, some of the predictions are already slightly obsolete. (It is amusing how little controversy there is over the idea of China digitizing the entire contents of the British Museum and Library, compared to how much uproar there is in Europe right now over Google.)

Still, it’s easy to see how this global scanning project inspired Vinge’s future version—scanning books a page at a time is a time-intensive process, even if you saw the spine off and put the stack of paper in a sheet-feeding scanner. With better computers it would be much faster to scan them in windblown fragments, and digitize all the world’s knowledge in a matter of weeks.

This does, of course, depend on printed books getting so deprecated that nobody minds if scores upon scores of them—some possibly valuable antiques—go through the shredder in the name of digital reincarnation. It is difficult to see that happening now—but on the other hand, if technology marches on as quickly as Rainbows End predicts, before long entire generations may come to think of printed books as akin to papyrus scrolls and stone tablets. (Even for an e-book fan like myself, that’s a scary thought.)

Faulty Crystal Balls

Still, I’m doubtful it will happen. Predicting the future is always an inaccurate game—if we go by True Names, we should already have direct-brain-stimulation cyberspace and mandatory computing licenses by now. Rainbows End is set as far in our future now as True Names was then—and if it seems like books written now are “more accurate” predictors, that is only because hindsight is 20/20. Things that didn’t come to pass in a book written thirty years ago are easy to spot—but those same things in a book written now will have to wait another thirty years.

There are already a few “predictions” (for the 2005-2010 years) that did not come true—and one irony. At one point in Rainbows End, a character refers to a (fictitious) Terry Pratchett book written after Robert Gu succumbed to Alzheimer’s Disease. Of course, we now know that Terry Pratchett himself has been diagnosed with latent Alzheimer’s.

But as far as prognostication goes, Vinge also isn’t above poking a little fun at himself. As the battle between belief circles rages at the library, one character reflects:

There had been a few debacles in the late Teens, when major belief structures had produced some awful art. Some were so bad that the circles themselves had shriveled and died. Who heard of Tines anymore, or the Zones of Thought?

The Tines and Zones of Thought are, of course, major elements from A Fire Upon the Deep.

Conclusion

Rainbows End is an interesting book for the future it predicts. It presents a fully-realized world, very well fleshed out and with more interesting predictions and characters than I have been able to cover in this review. I highly recommend it.

Ironically for a book where e-books are important, Rainbows End does not seem to be commercially available as an e-book—not on eReader, Fictionwise, or even for Amazon’s Kindle. The HTML version at the Internet Archive appears to be the only way to read it electronically.

There’s a chance that in the 1990s, you were more familiar with neighborhoods on GeoCities than with the neighborhoods in your own town. As one of the most popular and oldest (nearly 15 years running) sites for self expression on the web, GeoCities paved the way for other sites which would offer a sense of community and networking capabilities. Because it was one of the first ways for people to freely and openly become engaged with the internet, GeoCities will always be an important part of web history.

Yahoo! announced that it will close the site on October 26, 2009, steering users towards their paid service instead. We have been archiving GeoCities sites for years in our crawls, but, as goes with the territory of being web archivists, we want to make sure to gather as many of the pages as possible before the looming end of an era, 10-26-2009. If you have a page with GeoCities or are a fan of a particular page, please use our special collections page to ensure its preservation. Additionally, please refer to another independent project, the Archive Team, who is working to save cultural information that may be lost with the site closing. Yahoo! is also offering valuable advice at their help center.

Thanks to ResourceShelf for the link and lets all hang, draw and quarter the Internet Archive web designer who makes getting those internal links above so annoying by using snapshots. Shame on you!

But first, here is an interesting editorial about the Google Books settlement by Paul Carr, who makes it quite clear that he has strong opinions and is going to express them. Carr sums up the history of the Authors Guild lawsuit and settlement, then talks about the Open Content Alliance that is opposing it.

The OCA, it should be noted, has a number of noncommercial members—including libraries, universities, and net-archivist group the Internet Archive. However, it also has Microsoft, Yahoo, and Amazon—two of these Google’s biggest commercial rivals and the third a business with a substantial interest in pushing its own e-books—whose talk about how “content” should be “open” does seem to ring just a little hollow doesn’t it?

The heart of Carr’s argument is this:

All of which leads me to the real question that needs to be asked this week: what on earth are the Internet Archive and Gary Reback and the libraries, universities and other legitimate members of the Open Content Alliance thinking?

The stated aims of the Alliance – to ‘build a permanent archive of multilingual digitized text and multimedia material’ – are solid, and their position that Google’s legal immunity over orphaned works should be extended to all is laudable. But by palling around with anti-trust terrorists, self-interested champions of DRM and conflict-funded law schools, they’re undermining all of that by making themselves look like corporate shills.

Carr suggests Archive.org and other noncommercial interests should get out of bed with Microsoft and Yahoo and instead put their energy into lobbying for copyright reform—working with Google to make that happen, instead of against them.

I think Carr makes a good point. Whether the Google Books settlement is fair to authors is one thing. But when you think about it, it looks awfully suspicious for Google’s biggest competitors to come out self-righteously swinging against Google for doing something they would be just as ardently defending if they were doing it themselves.

… In fact, the Internet Archive total will likely increase double what was expected, as they were just approching 1,500,000, July 4, and reached 1,545,731 texts by noon EDT, July 27. This will likely reach additional totals of ~50,000 added just during the period from July 4 through August 4. Add to that the 25,000 or so mentioned above, and a few hundred more at Project Gutenberg sites and others, and the grand total by the end of the fair is likely to surpass 2,500,000, as follows:

1,550,000 Internet Archive
500,000 World Public Library
113,000 Project Gutenberg
130,000 ebooksabouteverything.com
37,000 Other eBook Sites
========= =======
2,320,000 Grand Total [Estimate]

In addition, Internet Archive offers:

191,421 movies
381,602 audio recordings
66,622 concerts [totals as of July 27]

[These are not added into our grand totals this year, but will probably be added in next year, so statistic oriented people should be advised]

The World eBook Fair is handing out a million files per day on the average with best day July 15 of 1.5 million files.

Reminder: some of the entries take more than one file so a one to one matching of files to books will not be accurate.

As you can see, our earliest most popular books were joined by some other classics, as well as computer oriented works.

Most Popular Titles At Half Way Point:

1. Emma, by Jan Austen

2. Linux Complete Command

3. Little Woman, by Lousia May Alcott

4. Nostromo, by Joseph Conrad

5. Workbook in Higher Algebra

6. A Child’s Garden of Verses, by Robert Louis Stevenson

7. A Journey to the Centre of the Earth, by Jules Verne

8. The Decameron, by Giovanni Boccaccio

9. Overview of Servlets and JavaServer

10. RedHat Linux Unleashed

11. Win XP Pro

12. Cousin Bette, by Honore de Balzac

13. The Beautiful Book Of Nursery Rhymes, by Frank Adams

14. Workbook in Higher Algebra

15. C+ Programming

16. MY SQL manual

17. Colonel Chabert, by Honore de Balzac

18. The Case Book of Sherlock Holmes, by Arthur Conan Doyle

19. The Art of War, by Sun Tzu

20. The Time Machine, by Herbert George Wells

Michael S. Hart
Founder
Project Gutenberg
co-Founder
World eBook Fair

Thanks to ResourceShelf for the link.

We’ve got a new release of our book reader, which we’ve been working on for the past few months.

In addition to a new theme and user interface, the reader has the following special features:

the reader includes unique (and simple to understand) URLs for each page, which update as you move through a book. These URLs can be used in citations and bookmarks, making it easier and more legible when referring to a particular page of a book.

books can be viewed in one or two-page mode.

in one-page mode, images can be zoomed up to 100% of the original scans. Because the Internet Archive scans are in color, this is an especially nice feature with illustrated books.

it has the capability of accommodating books that read right-to-left, such as those books in our Yiddish collection.

the reader is supported by all browsers (but IE 6).

there is an auto-play feature, so that you can set the pages to turn automatically.

As always, the reader is open source. If you have suggestions or bug reports, please add them to the book reader’s launchpad page so the engineers will see and prioritize them.

To test the reader, go to the book’s details page and, in the “View the Book” box, click on the “Read online” link or the animated gif.

I first interviewed Peter Brantley, in the Talking with Talis series, in July 2007 about his role in the Digital Library Federation and its place in the world of digital libraries.

In this conversation we look back over the last couple of years at the DLF and then forward in to his new challenge and opportunity at the Internet Archive.

We go on to discuss his thoughts and plans to make it easy to identify books and  information and their locations in a way that is currently not possible with the processes and protocols we use today.

T he Internet Archive has filed a brief against Google hoping to challenge Google’s  settlement with the “Author’s Guild” on the grounds that the Internet Archive has just as much right to protection from infringement claims on out-of-print “orphan” books as Google does.From the Archive’s letter to Judge Dennis Chin who is presiding over the Author’s Guild settlement.  …

“The Archive’s text archive would greatly benefit from the same limitation of potential copyright liability that the proposed settlement provides Google. Without such a limitation, the Archive would be unable to provide some of these same services due to the uncertain legal issues surrounding orphan books.

Dear Author also has a summary with a roundup of relevant Settlement articles.

As it happens, I actually have not read The Crystal Stopper in quite a while, so the review will not focus on the story within it (though it is worth mentioning that it is an excellent and suspenseful tale of high adventure with French literature’s best-known master thief). What interests me is the construction and presentation of the book itself.

Finding the Book

The ordering process starts by going to Public Domain Reprints’s website and searching on the book or author you want to reprint. The search process is a bit…problematic. It’s an embedded search, powered by Google, and searches both the Internet Archive and Google Books to find the result.

Each result is presented with a bold blue link to the work itself, and a "Request a Reprint" link below it. At the bottom is a list of pages of results, as well as a "More Results" link.

The problem is, the search doesn’t work very well. First of all, it is fairly inaccurate. I typed in "Maurice Leblanc" both with and without quotation marks and found a whole lot of books unrelated to what I was looking for.

Second, the links are confusing. If I click on the big bold blue link for the book I want, I expect to be taken to a page where I can order it. Instead, I get the book itself on Google Books (or presumably the Internet Archive, though I did not find any results there). I would have to click the smaller, unassuming "Request a Reprint" link below the result to get to the order page. This might be too confusing for unwary users.

Still on the subject of links, the results page numbers at bottom take you to more pages of results in the embedded search—but the "More results" link takes you away from publicdomainreprints.org entirely, to the Google Books search homepage. And while you can certainly find the book you want there, you cannot order a reprint of it from there.

I ended up having to go to the Google Books native search page, do an Advanced Search, and copy the query syntax it generated ("inauthor:Maurice inauthor:Leblanc date:0-1923"), then take that back to the publicdomainreprints.org page to get what I wanted: a list of books by Maurice Leblanc that were out of copyright.

Requesting a Reprint

After you find the book you want and click the "Request a Reprint" link, you may receive two possible screens. If the book you want has already been ordered by someone (as The Crystal Stopper has now) you are taken to an ordering page where you can view a preview PDF of the first one-third of the book as it will look printed, and place your order if it meets with your approval.

If it has not been ordered yet (for example, no one has ordered 813 as of this writing), it will have to be formatted for printing. In this case, you get a reprint request page with a note explaining that it could take up to 72 hours for the reformatting to come through, that requesting the reprint does not obligate you to buy it, and that the quality of scans may vary so you should preview the book before ordering it. There is a space for an email address to get nofication when it is ready, and a reCaptcha to prove that you’re human.

When I requested The Crystal Stopper, it turned out to be 80 hours before the book was ready to order, but close enough.

Placing the Order

When the book is ready to order, it is ordered through print-on-demand provider Lulu.com. The order is placed through Lulu’s website, and Lulu is the one who ships it to you. In my case, The Crystal Stopper cost $10.99—a reasonable price for a printed-on-demand book of its size.

Lulu offers a low cost Media Mail and higher cost Priority Mail shipping option. I chose Media Mail, and had my book within a couple of weeks.

The Book Itself

As pictured above and at left (with one of my cats for size comparison—as you can see, it is about one third of a cat wide), the book is a 6"x9" perfect-bound trade paperback, with green cover and black spine. The first page is a new title page stating the title, author, when the book was originally published, and when it was set up for reprinting. After that are a couple of pages of information about Public Domain Reprints and Google Books. Then the scan of the original book itself begins.

In a nutshell, all Public Domain Reprints really does is take the PDFs of public domain scans from archive.org or Google Books, format them so Lulu can reprint them, and then upload and list them on Lulu. Thus, what you see in this book is pretty much what you would see if you want to the Google Books scan of The Crystal Stopper, just in black-and-white printed form.

This includes a scan of the cloth cover, the University of Michigan Libraries watermark on the flyleaf…and "Digitized by Google" in the lower right corner of every page. It is a pity that it has to be there, but that is Google’s condition for the use of its scans. On the bright side, at least it does not block any of the text from being read.

The result is generally quite readable, at least for The Crystal Stopper. You would never mistake it for anything but a reprint of a scan, even if it were not for the Google logo. There are a few pages where a letter or two is cut off from either edge, and many of the pages have a black line along  the edge where the scanned book’s spine was. In some rare cases, such as the one pictured at right, there is actually a bit of the opposite page stuck on.

That being said, there is only one thing about the book that annoys me. It is a minor annoyance, and I would probably not even notice it once I started reading, but all the same it irritates me. As you can see from the picture, the position of the pages is reversed from how they were in the original book—the page on the left was the page on the right in the original, as it shows traces of a page to the left of it across the original book’s spine.

This means that the page numbers at the top. which were meant to be at the outside corners for easy reference, are instead at the inside and harder to see, and the book does not represent an entirely faithful reproduction of the original. They could have easily remedied this by removing the (pointless) scan of the cloth cover, or even just adding a blank page after the Public Domain Reprints and Google Books blurbs, but I suppose it was not a priority.

Why Exact Reprints At All?

I have to wonder: why bother reprinting the book exactly as it was scanned at all? It is the words of the text that matter to the reader—certainly all the people who read e-books don’t care what the original printed page looked like. A book made by taking the Project Gutenberg HTML version of The Crystal Stopper and formatting it into a printable PDF would look more professional, and could take advantage of the advances in typesetting technology that have come about in the last eighty years.

On the other hand, it is probably less resource-intensive to rescale a PDF that already exists rather than create a whole new one from scratch. If it is only having those PDFs available that makes the project even possible at all, there is no point letting the perfect be the enemy of the good. And I have to admit that the reproduction of the original printed pages does give the book a certain veneer of authenticity.

On the whole, I am not sorry I ordered, and this book will be joining the original Arsène Lupin printings on my bookshelf. It is possible I might even order more eventually, now that I know exactly what I am getting.

Apart from the awkwardness of their search system, I find the process of obtaining a reprint from Public Domain Reprints simple and satisfactory. And if the reprint is clearly a reproduction of a Google Books scan, at least they are honest up front about how it will look and advise previewing before ordering. So, if you need a tree-book of a public domain title that is not in print elsewhere, Public Domain Reprints delivers a decent product at a reasonable price.

CAPTCHAs (an overly-cutesy acronym standing for “Completely Automated Public Turing test to tell Computers and Humans Apart”) are those tests that are supposed to verify you are a real human by making you type some distorted letters or numbers. This is meant to keep spambots from being able to register for accounts, make spam posts on forums, sniff out email addresses, or do other things that might be considered harmful.

Two Problems That Solve Each Other

The problem is that spambot software has gotten to the point where it can often correctly parse out even distorted CAPTCHAS unless they are so distorted that humans cannot read them either. Also, users are not fond of being forced to waste time squinting at jumbles of meaningless letters and numbers—especially older users with poor eyesight to begin with.

Meanwhile, volunteers digitizing old public domain texts have known for a long time that optical character recognition (OCR) systems are imperfect. They make lots of transcription mistakes, and the only way to hash them out is to have a human look at the words in question and correct them. This is the rationale behind Distributed Proofing. But it is hard to find people who have the time and inclination to read and correct entire pages of old literature.

But sometimes, two problems will solve each other. Researchers at Carnegie Mellon noticed the similarities between the two situations, and realized it would be possible to harness the “wasted time” dealing with CAPTCHAs as a force for translating mis-scanned words. They built this idea into a system called ReCAPTCHA.

ReCAPTCHA

Under ReCAPTCHA, the OCR program pairs images of a word that it cannot translate with a word that has been verified and asks users to type both of them. If the user got the known word correct, then he is assumed to have gotten the unknown word correct. Once several users give the same answer for an unknown word, the word is considered to be correctly identified.

ReCAPTCHA has proven remarkably resistant to cracking by CAPTCHA-cracking software—for precisely the same reason that the words needed to be passed by humans in the first place; OCR software just cannot read them, but humans can. And most casual users will consider it anything but a waste of their time.

According to the Ars Technica article, ReCAPTCHA is used by over 40,000 sites, and handles over 100 million words per day. It is currently being used to help digitize books for the Internet Archive, and older editions of the New York Times.

(Additional coverage: NPR, Slashdot)

Who says e-reading is the only way to enjoy digitally stored text?

Publishers can now turn digital files into single copies of a professional-quality printed book without requiring a full-fledged production run on a printing press. In the most advanced form, the technology even allows printing at the exact location it is needed.

This is worlds apart from the traditional publishing method of printing centrally and shipping to stores.

Traditional Publishing vs. the Vanity Press

Affordable, consumer-friendly print-on-demand technology has been a sought-after grail of the publishing world for at least ten years.

Traditional publishing has always suffered from a problem called economy of scale. It costs so much to set up a printing operation to create a book that publishers must sell either a huge quantity of inexpensive books or a lesser quantity of expensive books in order to break even.

So publishers have had to pick and choose what books could be published, with significant incentive to publish only the most commercially successful works they possibly could.

Death of the midlist

In recent years, this has lead to the “death of the midlist.” That’s the publishing industry term for books that only sell decently. Some publishers have squeezed out midlist writers in favor of those who consistently produce bestsellers.

Of course, it was always possible to finance your own print run, through so-called “vanity presses.” However, vanity publishing had a bad reputation due to the high per-book cost of small print runs—and due to the large number of shady operators. They would prey upon the author’s gullibility and desire to see his work in print to con him out of a good deal of money to print the book.

In the end the author would be left with the realization that he would have to spend still more money to market it on his own.

So in the old days, vanity publishing was considered feasible only for inferior or so-so writers who had large sums of money burning holes in their pockets. Their vanity led them to want to see the book in print even if it wasn’t good enough for a “real” publisher.

Deciding print run size: Another problem with traditional publishing

Another problem with traditional publishing has been deciding on the right size of print runs. Printing too few books results in lost sales by consumers who lose interest when they are not able to get the book immediately. Printing too many runs up huge warehousing costs, as well as costs associated with destroying excess copies that do not sell.

Warehousing and distribution costs may actually account for considerably more expense than printing costs in the end.

Unhappy returns

Lacking effective crystal ball gazers, publishers are often stuck with lots of unsold books. Due to a clause that has been grandfathered into publishing contracts since time immemorial, publishers usually have to buy back from bookstores all copies that do not sell.

Such bought-back or otherwise unsold books are generally either destroyed or sold at a huge markdown to returned book vendors who turn around and sell them at similarly low prices. These are the books you often see in stacks on tables at Waldenbooks or other bookstores with really low price labels on them.

This was also an obstacle to the growth of book superstores such as Amazon.com. The more titles such a store was to carry, the more warehouse space it would need for books that sold very rarely but still needed to be stocked against the chance that someone eventually would order them.

Enter print-on-demand

However, by the late 1990s, digital printing had advanced enough that professional-quality books could be produced cheaply in smaller numbers, by industrial-strength laser printers. With no type to set, such a printer could produce any one of a whole library’s worth of digitally-stored books on short notice. Or, in other words, print them on demand.

It would still have to be professionally bound and then shipped out like any other book, of course, but compared to the cost of setting up a full-scale print run, the savings were substantial.

The rise of Xlibris, Lulu and Lightning Source

A number of companies were soon set up to take advantage of this industrial-scale print-on-demand technology. Smaller companies such as Xlibris and Lulu deal directly with consumers, while  larger companies like Lightning Source are integrated more directly into the larger publishing industry.

In fact, Lightning Source serves as a considerable part of Amazon.com’s merchandising back end.

Lightning’s print-on-demand capability makes it possible for Amazon to stock more book titles than it can afford to warehouse. When it receives an order for a seldom-requested book, it can simply have Lightning Source print the book out as needed.

Beyond “vanity”

Amazon was not the only one helped by industrial print-on-demand publishing. Thanks to the smaller print-on-demand companies, self-publishing has gone from strictly “vanity” to something that many small publishers or even single individuals can do economically.  For example, Diane Duane will be using Lulu to create printed versions of The Big Meow (see this TeleRead entry) for her subscribers.

Many small presses no longer contract traditional printing for their books but instead send them off to a POD outfit such as Lightning Press.

As Lightning Press offers packages that include automatic listing for sale on Amazon.com, it has become easier than ever to self-publish and promote the book at the same time—perhaps in some cases, too easy. In fact, my own uncle has a self-published print-on-demand book listed at Amazon.com.

Enter Espresso: Beyond the central printing model

But these industrial-scale print-on-demand companies still have one foot in traditional publishing; the books still must be printed by a centrally-located company and then shipped out to the customer.

For a number of years now, companies have been seeking to make something more portable—a consumer-level kiosk that can be placed in libraries and bookstores to produce fast, inexpensive individual copies of books requested by consumers. The process itself is not that difficult. Brewster Kahle of the Internet Archive was able to put everything necessary to create inexpensive printed and bound versions of public-domain books in his Internet Bookmobile van and tour the country with it.

There is also a non-profit organization called Public Domain Reprints that prints and ships copies of public domain ebooks at cost. I have ordered a book from PDR, and will be reviewing it here when it arrives. The problem comes in automating the procedure so it can be done entirely without human intervention.

The On-Demand Espresso Book Machine

But a company called On Demand Books has done it. In 2006, they produced the first model of their Espresso book machine. Dubbed “an ATM for books” by CNN, the Espresso incorporates a computer, two printers, apparatus for bookbinding and trimming, and automation to tie them all together.

A book can go from PDF files on a computer to a finished, professional-quality paperback book in under ten minutes. The machine itself costs $140,000 (though CNN’s article claimed $50,000; perhaps $50,000 is the cost and $140,000 the retail price). It can produce books at a cost of a penny a page.

A sticky problem

But creating the book machine was not as simple as just hooking some equipment together. To create a system that could go from long periods of inactivity to instantly cranking out a book, the On Demand team needed to find a method of gluing the spine to the block (the printed pages) without hot glue. Glue kept hot for too long thickens and loses its effectiveness, and also smells bad.

In the end, they developed a new method of binding that kept the glue at room temperature until needed and then hit it with a burst of ultrasound to heat it up.

The Espresso-Lightning Print agreement

Originally, the Espresso was limited to public-domain titles only, but in April, On Demand entered into a partnership with Lightning Source, the owners of Lightning Print.

The agreement  would give On Demand access to Lightning’s scanning facilities (to create high-quality digital copies of books that they could then reproduce through the Espresso). It would also allow them to use copyrighted material from any of Lightning Source’s publishers who opt to let them.

World Bank site of first completed Espresso machine

The first completed Espresso was deployed in 2006 at the World Bank in Washington DC, amid much fanfare (see also a 33 minute RealVideo recording of the ceremony). It has since been moved to the New Orleans Public Library to replace books destroyed by Hurricane Katrina.

Seven others now exist in such places as the Bibliotheca Alexandria in Alexandria, Egypt; the Internet Archive headquarters in San Francisco, California; and several bookstores or libraries in the US, Canada, and Australia.

In October, some of the machines will start appearing in British bookstores as well.

Still in early stages

While the Espresso offers a potentially bright future for paper book lovers in an e-world, it is still in its early stages.

The existing machines are on the order of prototypes. You see a bunch of off-the-shelf printing and binding equipment jumbled together in an unholy mess of assembly-line robotics that would seem more at home on a factory floor than in a library or bookstore.

They are also costly and time-consuming to build: On Demand has to spend about a month per unit putting them together from scratch. However, On Demand hopes to have a second-generation, mass-producible, prettier-looking unit ready by fall 2008, to begin leasing in 2009.

Implications for e-books and treebooks

If the Espresso is able to be mass-produced, it may not be long before it starts showing up in bookstores, libraries, airports, maybe even coffee shops all over the place—and its effects might filter over to affect e-books as well.

After all, e-book vendors already offer all their wares in digital format. Print publishers that resell their wares through Fictionwise might not want Fictionwise going into direct competition with them. But smaller publishers selling their titles e-only, because they cannot afford even limited industrial POD print runs, might just find this a match made in heaven.

Likewise, if PDF roleplaying game sourcebooks could be economically one-off printed in better-than-Kinko’s quality, it could expand their market as well.

Handy even for the big guys

And even traditional print publishers will not always have the foresight to print enough of their books. Print runs take time, and if a novel turns into an unexpected best-seller, it might be a week or more between the first print run selling out and the second hitting stores.

If the novel could be printed on demand as a stopgap, it might mean selling thousands of copies that would otherwise go unsold as would-be buyers cool down during its unavailability.

It also means that older titles need never go “out of print”—as long as the publisher has a digital copy on hand, a single new paper copy can always be printed out without the cost of setting up a print run or shipping it to the store.

(Of course, this will be a concern to some authors, whose contracts have so-many-years-out-of-print rights revision clauses. But that was an issue caused by e-books as well, and there have been several years since the problem first was noted in which to work that out.)

Consumer-level print-on-demand, as exemplified by the Espresso, is a hybrid of P and E that brings the availability advantages of an e-book to the form factor of a paper book. It it means that people who prefer not to read on a screen need not be left entirely out of the digital revolution. It will be interesting to keep an eye on this situation as it develops.

YouTube Videos

Take a look at these videos of the Espresso in action. The first one is kind of a puff piece, but the second one gets a bit more technical.