Comments on: ‘What Does Google Want Us to Do With All These Free PDF eBooks?’: Problematic downloads http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/ News & views on e-books, libraries, publishing and related topics Sun, 21 Mar 2010 10:51:10 -0500 http://wordpress.org/?v=2.9.2 hourly 1 By: Dagg http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-127704 Dagg Thu, 23 Nov 2006 20:40:58 +0000 http://www.teleread.org/blog/?p=5413#comment-127704 I think DjVU is hands down better solution. I produces smaller files, better quality and has many more advantages for this purpose. Just as one example, if you enlarge PDF document which contaings pictures, moving around page causes flickering (thus pictures are refreshing and re-shaping with each move), wjile in DjVU browser moving around enlarged page is smooth. Some say DjVU as format is not so popular. Who cares, document viewers are not brands like Mercedes or BMW, they are back programs: more invisible to the user, better it is. I think DjVU is hands down better solution. I produces smaller files, better quality and has many more advantages for this purpose. Just as one example, if you enlarge PDF document which contaings pictures, moving around page causes flickering (thus pictures are refreshing and re-shaping with each move), wjile in DjVU browser moving around enlarged page is smooth. Some say DjVU as format is not so popular. Who cares, document viewers are not brands like Mercedes or BMW, they are back programs: more invisible to the user, better it is.

]]> By: Winfried Helge Pelz http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-82738 Winfried Helge Pelz Fri, 08 Sep 2006 16:06:49 +0000 http://www.teleread.org/blog/?p=5413#comment-82738 Just look what Digitized by Google really means: Download as example this book as pdf-file: Eugenio Cappelletti Vocabolario Milanese-Italiano-Francese published 1848. --carefully scanned by Google-- as the first page in the pdf tells you: "This is a digital copy of a book that was preserved for generations on library shelves before it was carefully scanned by Google as part of a project to make the world’s books discoverable online." I have sampled around 100 other books, all abundant with same defect types, although not such extreme high rates. Not a minimum care for any sort of quality control. Very sad. Just look what Digitized by Google really means:

Download as example this book as pdf-file: Eugenio Cappelletti Vocabolario Milanese-Italiano-Francese published 1848.

–carefully scanned by Google–
as the first page in the pdf tells you:
“This is a digital copy of a book that was preserved for generations on library shelves before it was carefully scanned by Google as part of a project
to make the world’s books discoverable online.”

I have sampled around 100 other books, all abundant with same defect types, although not such extreme high rates.

Not a minimum care for any sort of quality control.
Very sad.

]]> By: Bill Janssen http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-82209 Bill Janssen Wed, 06 Sep 2006 19:00:35 +0000 http://www.teleread.org/blog/?p=5413#comment-82209 There's an important post on all this on the Book-People list, pointing out that it's trivial to extract the hi-res page image scans from the PDF files, and discard all the extra Google-added stuff: Subject: [BP] Re: Google Books PDFs From: "Stewart C. Russell" <scruss@gmail.com> Date: Fri, 1 Sep 2006 14:00:00 PDT I've had reasonable success extracting page images using pdfimages (from the xpdf suite: <a href="http://www.foolabs.com/xpdf/" rel="nofollow">http://www.foolabs.com/xpdf/</a>). It's rather slow, but extracts the pages exactly as stored. Each page seems to comprise the main page image (scanned at a high resolution) plus a separate "Digitized by Google" watermark image. These smaller images can easily be discarded. cheers, Stewart -- http://scruss.com/blog/ There’s an important post on all this on the Book-People list, pointing out that it’s trivial to extract the hi-res page image scans from the PDF files, and discard all the extra Google-added stuff:

Subject: [BP] Re: Google Books PDFs
From: “Stewart C. Russell” <scruss@gmail.com>
Date: Fri, 1 Sep 2006 14:00:00 PDT

I’ve had reasonable success extracting page images using pdfimages
(from the xpdf suite: http://www.foolabs.com/xpdf/). It’s rather slow,
but extracts the pages exactly as stored.

Each page seems to comprise the main page image (scanned at a high
resolution) plus a separate “Digitized by Google” watermark image.
These smaller images can easily be discarded.

cheers,
Stewart

–
http://scruss.com/blog/

]]> By: Bill Janssen http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-81615 Bill Janssen Tue, 05 Sep 2006 15:43:04 +0000 http://www.teleread.org/blog/?p=5413#comment-81615 Now you can OCR those PDF files with <a href="http://google-code-updates.blogspot.com/2006/08/announcing-tesseract-ocr.html" rel="nofollow">Tesseract</a>, which has been out for several months, but was just Slashdotted today. Now you can OCR those PDF files with Tesseract, which has been out for several months, but was just Slashdotted today.

]]> By: Howard Bernier http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79257 Howard Bernier Thu, 31 Aug 2006 13:53:09 +0000 http://www.teleread.org/blog/?p=5413#comment-79257 It seems (with limited experience) that when downloading the pdf's; if the size of the file is under 10-12mb, then the download proceeds just great. Larger than that the download stops, and reloading the file will result im problems with the browser. (Explorer, Firefox, Safari all yield the same problem) Anyone have a workaround ? Thanks. It seems (with limited experience) that when downloading the pdf’s; if the size of the file is under 10-12mb, then the download proceeds just great. Larger than that the download stops, and reloading the file will result im problems with the browser. (Explorer, Firefox, Safari all yield the same problem)

Anyone have a workaround ? Thanks.

]]> By: Bill Janssen http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79134 Bill Janssen Thu, 31 Aug 2006 00:56:02 +0000 http://www.teleread.org/blog/?p=5413#comment-79134 That's 10.4.7 Preview, of course. That’s 10.4.7 Preview, of course.

]]> By: Bill Janssen http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79133 Bill Janssen Thu, 31 Aug 2006 00:55:05 +0000 http://www.teleread.org/blog/?p=5413#comment-79133 DjVu, while more efficient (in the past) than other available formats, is still a niche format in terms of support and use. I'd use PDF, instead, myself. I notice that the Google PDF book downloads use the JBIG2 image compression scheme, where similar letter shapes are represented by one canonical glyph, instead of providing the exact (unavoidably distorted) scanned image. Many PDF viewers cannot understand this image compression scheme yet; I tried it with Apple's Mac OS X 10.3.9 Preview, and with pdftotext; the older Preview showed the pages as blank, and the newer pdftotext (part of the xpdf package) gave me a bus-error when trying to de-compress the JBIG2-compressed pages. OS X 0.4.7 Preview seems to work OK with the PDF, though. DjVu, while more efficient (in the past) than other available formats, is still a niche format in terms of support and use. I’d use PDF, instead, myself. I notice that the Google PDF book downloads use the JBIG2 image compression scheme, where similar letter shapes are represented by one canonical glyph, instead of providing the exact (unavoidably distorted) scanned image.

Many PDF viewers cannot understand this image compression scheme yet; I tried it with Apple’s Mac OS X 10.3.9 Preview, and with pdftotext; the older Preview showed the pages as blank, and the newer pdftotext (part of the xpdf package) gave me a bus-error when trying to de-compress the JBIG2-compressed pages. OS X 0.4.7 Preview seems to work OK with the PDF, though.

]]> By: Liviu http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79108 Liviu Wed, 30 Aug 2006 23:11:35 +0000 http://www.teleread.org/blog/?p=5413#comment-79108 I used to like djvu better than pdf in the past, but there is considerable more free software to manipulate pdf's than djvu's, where you need basically to use the linux tools. Also many ocr engines do not (as of yet) recognize djvu, so while I agree that djvu is a better format than pdf from many points of view, it is less practical for now. Anyway you can convert pdf's to djvu freely and easily in Linux with djvudigital (and it works with Colinux too so you can use it on an xp box also). Liviu I used to like djvu better than pdf in the past, but there is considerable more free software to manipulate pdf’s than djvu’s, where you need basically to use the linux tools.

Also many ocr engines do not (as of yet) recognize djvu, so while I agree that djvu is a better format than pdf from many points of view, it is less practical for now.

Anyway you can convert pdf’s to djvu freely and easily in Linux with djvudigital (and it works with Colinux too so you can use it on an xp box also).

Liviu

]]> By: Michael Dillon http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79099 Michael Dillon Wed, 30 Aug 2006 22:21:44 +0000 http://www.teleread.org/blog/?p=5413#comment-79099 One wonders why Google did not use the DJVU format for their books. It was designed for scanned images, works with and without OCR text, and compresses neatly into small files. There is an open source reader called Evince that supports the DJVU format as well as LizardTech's free browser plug-in. The main downside is that you have to think about how to set up your scanning workflow and assemble the bits and pieces yourself rather than buying some commercial book-scanning workflow system. But for a project on Google's scale, it shouldn't have mattered because you only need to figure these details out once. I suspect that the real issue is that Google never thinks much about how to do things. They just throw cash and warm bodies at a problem and hope it will work out. It often doesn't as in the case of Google's ORKUT compared to MySpace. One wonders why Google did not use the DJVU format for
their books. It was designed for scanned images, works with
and without OCR text, and compresses neatly into small files.

There is an open source reader called Evince that supports
the DJVU format as well as LizardTech’s free browser
plug-in.

The main downside is that you have to think about how to
set up your scanning workflow and assemble the bits and
pieces yourself rather than buying some commercial
book-scanning workflow system. But for a project on Google’s
scale, it shouldn’t have mattered because you only need to
figure these details out once.

I suspect that the real issue is that Google never thinks much
about how to do things. They just throw cash and warm bodies
at a problem and hope it will work out. It often doesn’t as in
the case of Google’s ORKUT compared to MySpace.

]]> By: Liviu http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79084 Liviu Wed, 30 Aug 2006 21:42:44 +0000 http://www.teleread.org/blog/?p=5413#comment-79084 As much as I dislike pdf's, I think that Google's action is a step forward. The books are scanned only without ocr'ing so there is not much choice of formats, other than pure pictures. I have not yet run one of my 4 Google downloads through the Abby OCR engine to check the resolution, but for jpg page to screen reading on Nokia 770 they are ok, I extracted one page and checked. The sizes are as expected for picture pdf books (about 7-10M per 300 page book) but they downloaded almost instantaneously on my cable connection. I have heard that they are negotiating with publishers to sell the pdf's of copyrighted books; I have to see prices and drm but it seems a step forward there too. Liviu As much as I dislike pdf’s, I think that Google’s action is a step forward. The books are scanned only without ocr’ing so there is not much choice of formats, other than pure pictures. I have not yet run one of my 4 Google downloads through the Abby OCR engine to check the resolution, but for jpg page to screen reading on Nokia 770 they are ok, I extracted one page and checked.

The sizes are as expected for picture pdf books (about 7-10M per 300 page book) but they downloaded almost instantaneously on my cable connection.

I have heard that they are negotiating with publishers to sell the pdf’s of copyrighted books; I have to see prices and drm but it seems a step forward there too.

Liviu

]]> By: ryanramseyer http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79069 ryanramseyer Wed, 30 Aug 2006 19:37:18 +0000 http://www.teleread.org/blog/?p=5413#comment-79069 Google is to be applauded. Now that they have released the pdf's, we can store these things, read them in any pdf reader, etc. When Sony finally releases the reader, this google trove should provide a textual answer to itunes (at least as far as out of copyright books go.) Google is to be applauded. Now that they have released the pdf’s, we can store these things, read them in any pdf reader, etc.

When Sony finally releases the reader, this google trove should provide a textual answer to itunes (at least as far as out of copyright books go.)

]]> By: Bruce Albrecht http://www.teleread.org/2006/08/30/what-does-google-want-us-to-do-with-all-these-free-pdf-ebooks/comment-page-1/#comment-79068 Bruce Albrecht Wed, 30 Aug 2006 19:36:03 +0000 http://www.teleread.org/blog/?p=5413#comment-79068 I haven't tried it yet, but there's at least one report at Distributed Proofreaders (http://www.pgdp.net) that the PDF books are much better resolution than the page at a time images, and that they are getting much better results OCRing from the PDF. It's possible that Planet PDF was looking at early images, too, as it's my opinion that scans from the last few months are much better than the early scans. They still seem to have a policy of skipping illustrations, though. Google's also been busy. I know there are at least 136,000 books available, up from about 50,000 in February or March. I haven’t tried it yet, but there’s at least one report at Distributed Proofreaders (http://www.pgdp.net) that the PDF books are much better resolution than the page at a time images, and that they are getting much better results OCRing from the PDF. It’s possible that Planet PDF was looking at early images, too, as it’s my opinion that scans from the last few months are much better than the early scans. They still seem to have a policy of skipping illustrations, though.

Google’s also been busy. I know there are at least 136,000 books available, up from about 50,000 in February or March.

]]>