TeleRead: Bring the E-Books Home

Bill, you said: “Proprietary reflow-centric formats (Mobipocket, .LIT, eReader, Sony BBEB, etc.) are on the verge of becoming obsolete.”

Can you explain what you mean here? Why does the new IDPF standards make them obsolete? These proprietary standards have tried to fix an ongoing problem with pdf. Is the basis of your objection that all proprietary document formats which have reflowable features are bad? If so, why? And doesn’t that implicate Adobe too?

Thanks. Actually my question is more of a technical nature. How does IDPF solve the problem of proprietary methods to address the reflowability issue? I don’t see it listed in the use cases/requirements on the Spec page . That’s a problem with rendering, not with source. The other thing that seems to tackle the problem is Section 2.3.2 Single-publication containers, but with alternate renditions. Is that what you mean?

Reflowability is a rendering/css issue, isn’t it? Do the specs have any changes about what CSS/stylesheets that readers should support?

BillJ – the OCF specification *does* say something about what’s inside it!. The spec dictates that using the .EPUB file type implies that OEBPS content is therein. There may be other content, such as a PDF version, and the OCF model may be used to contain other content types, but in the latter case it is not correct to use the .EPUB extension. For example the Adobe Mars project to create an XML-friendly version of PDF, which uses SVG for page descriptions, leverages the IDPF OCF approach but does not use the .EPUB extension.

David,

I have a different perspective. If you accept my thesis that the new IDPF standards (EPUB) delivery on key OpenReader goals, and that there’s no reason for OpenReader to continue promoting a separate “fork” of OEBPS, then OSoft is doing exactly the right thing by de-emphasizing their file format (which is currently proprietary to them), and not promoting a non-starter separate dialect under an OpenReader flag. Hopefully OSoft will support EPUB in the near future.

In other words, this is not IDPF “vs.” OpenReader – we’re all in this together, and share the same goal of promoting digital access to all content based on open standards. OSoft folks and Jon Noring, who’ve been participating in IDPF, are simply acting on this reality, and I think deserve praise, not criticism.

You have in the past expressed personal concerns about IDPF and certain people involved therein. The part of this that’s based on intrinsic suspicion of big corporate interests I confess to some sympathy with, and again I think OpenReader ‘tude can continue to play a valuable role in agitating for progress in new areas such as DRM interoperability. The part of this that’s based on years-ago personal history: well I wasn’t involved, but this was a time when companies were crumbling and eBooks were nearly given up for dead. Bad stuff happened all over – I say “get over it”.

Friends,

it seems that you are discussing the future of the different formats mainly based on the merits and features of standards, standardization procedures and technical arguments, and thereby play in a different arena than the actual game.

1. You know, the terrible tower of eBabel might not be so frightening in the long run as it appears now. Does anyone remember the tower of bitmapBabel? Which has begotten BMP, PNG, GIF, TIFF, TARGA, PSD and a zillion further formats for bitmap images. Many of them pretty much died over the years, some of them evolved and will continue to do so, along with needs of users and applications, market opportunities, market inertia, development trajectories, de facto standards and so on. Perhaps there is some similarity in this respect between textual documents and digital images. And audio formats. And 3D formats. And video formats. Different types of content and application will continue to require different features. Maybe these features are best served in a single multi-purpose format, maybe not. For instance, editing current-generation PDF is a pain in the neck, and mainly for the same reasons that determine that it is not reflowable. But it turns out that this is a desirable feature for documents like office forms, journal papers, train tickets and so on. Perhaps we want to have different families of applications and viewers for different classes of documents? Time will tell.

2. From the perspective of the software industry, a multitude of different formats does not present a problem, but a vast opportunity. As software guys, we should be thankful for that. We should stop complaining but deliver reliable, transparent conversion tools. A few years down the road, users may perceive the differences between Word2009, OpenReader 3.0, OpenOffice+, Media-BBeB, PDF and so on in a similar light as they do now for TIFF, PNG, CDR and PSD – as either necessary and warranted, or translatable by the click of a button.

3. The war will not be fought on the basis of arguments of the proponents of different standards. In the near future, it will depend on pre-existing content, available hardware, dominant players and perhaps a few regrettable glitches produced by software patents and such. OSoft’s promotion of a format (or lack thereof) is certainly important, but it may turn out to be much more important whether Jeff Bezos decides to stick with Mobipocket, or Sony manages to force enough of their adorable little readers down our throats to make BBeB a de facto standard before other hardware enters the field. You know, Ogg Vorbis is much, much nicer than MP3, almost in the same way as OpenReader is way better than BBeB. It simply does not matter.

4. For the next few years, I guess it is safe to say that PDF is here to stay, simply because the majority of documents used by professionals (which may make up the biggest share of paying customers for some time to come) relies on it. The same is true for the dominant office formats. Everything else lies in fate’s hands.

Having said all that, I do not want to sound as grumpy and vitriolic as Teleread on their bad hair days. Lets continue to design and implement the best possible format(s). And: great to have an Adobe guy in this blog too! Looking forward to your contributions!

Well it is nice to be so loved!

Anyway, this comment is not intended to address the “political” issues of OpenReader and OEBPS/OPS (which need discussing at the proper time/venue), but rather to provide some quick answers to other issues brought up in this comment thread.

First, to talk about some “politics” anyway.

I appreciate Bill McCoy’s comments about OpenReader and his eagerness for dialogue. I, too, share this eagerness since we both share common goals. It is true that the technical gap between OpenReader and OEBPS (or I guess the new acronym is OPS) has always been quite small since both specs are quite similar and compatible in their fundamental framework.

As everyone knows, I’ve been a contributor to the IDPF OEBPS Working Group the last year, most of the time as an invited expert. Recently my status changed and I now represent DigitalPulp Publishing, an IDPF member, as David Rothman has made sure the world knows about. DPP appreciates the free publicity!

At times I have been contentious in the OEBPS WG, and proud of it, but not for reasons of sabotaging or slowing down the spec, or because of spite, or ego, or whatever other self-serving emotional motivation some might claim.

Rather, my purpose is best described as “tough-love” — to make sure the new OEBPS spec provides continuity with the past, and that it is forward looking with respect to the future of interactivity, representing a wider range of publication types, and other goals which will benefit, first and foremost, publishers and users, and then the rest of the digital publishing stakeholders.

It has always been my interest to see a future rapport between OEBPS and OpenReader. Bill McCoy talks about forking — well, the worst fork of all is to be blind to the long-term requirements and thereby take the wrong road. Once a road is taken, it is difficult to get back on the right road, especially if we see the ebook industry explode the next few years, as many of us believe it will. For such critical specs like OEBPS/OPS and OpenReader, they have to be thoughtfully designed.

Related to this, and now to address Bill Janssen’s comment more directly (but not specifically), OpenReader, as defined by its core modular spec, the Binder Document specification, still has important innovations that are not yet introduced into OEBPS. Some of the core design goals of OpenReader were to:

1. build a stable basis for the highest level of interactivity including deep-linking into Publications (achieved),
2. represent a wide range of publication types, from purely linear to highly non-linear structures, such as web sites and topic-based representations (achieved), and
3. make it easier for publishers to repurpose their publications and minimize re-editing of file references in content documents, such as for internal links and style sheet assignments (achieved).

Unfortunately, the draft OPS 2.0 has not yet sufficiently addressed these issues, and in my opinion have actually done a few things that will make these goals a little more difficult to achieve. (Inline Islands is the nasty one that throws a monkey wrench into the works — I may post a blog article about this in the near future. It is a solution to a non-problem that few will use. Other than maybe for MathML, I see little wide-scale use of Inline Islands, and MathML can be dealt with in a better and cleaner way. Yet, like an elephant in the living room, Inline Islands requires that solutions to various problems take its presence into account.)

Hopefully OPS 2.5 or 3.0 will do the necessary fixes after proper study of the issues and assessing proposed solutions (of course, I’ll propose the OpenReader innovations, but there certainly may be other equivalent solutions.) Doing this is vitally important to the industry, but which the current OEBPS WG was not interested in doing since their focus was not future-oriented.

Joscha’s comment makes some interesting points. We tend to forget that both OPS and OpenReader are focused on the marketplace 10–15 years down the road. It took the CD audio standard, once it was released and the first equipment sold, about ten years to supplant vinyl analog sources as the dominant music delivery vehicle of the time. It will likewise take a decade for client-side reflowable formats to likewise dominate in the publication arena. (It is helped in that the Web started off with a reflowable, client-side typeset format: HTML. There is a reason why XHTML forms the basis of both OEBPS/OPS and OpenReader, although over time better vocabularies will be used.)

About OSoft’s dotReader, OSoft is working their butts off to get the core engine of their reading system working. It is still their intention to build, or spur the open source community to build, plugins for OpenReader and OPS 2.0 Publications. But they can’t build the plugins until the core is built. It’s like putting a roof on a house before the frame is even built.

As noted at the beginning of this comment, both OPS and OpenReader frameworks are quite compatible, so once a plugin is built for one, the other will naturally fall out pretty quickly. David Rothman does not fully comprehend the difficulties of software development (I do — I was involved with large collaborative programming efforts during my tenure as a staff scientist at three National Labs.)

My final comment is that direct public flogging rarely accomplishes anything positive — it only serves to make the flogger feel good for the moment, and alienates the very people one needs to meet the desired end. Now I am unnecessarily having to pick up the pieces of that public flogging. My week has been fun, to say the least. <smile/>

David, you mention that David Cote dissed OpenReader. Please point to the web page or pages where I can find his negative public statements about OpenReader.

In addition, DPP is truly an innocent bystander in all of this. Prior to my joining them a few weeks ago, they never made any promises to us vis-a-vis OpenReader. But I had been talking about it with them for some time, working up a consensus with them on standards and OpenReader — to build the bridge. Then you publicly flog them, OSoft, and me for no good reason. Smart move. What does it get you? The public reading this knows that you are really frustrated (and I understand that, OpenReader is an uphill battle in the first place, caught in all kinds of Catch-22’s), and now they also know that you don’t handle frustration very well, and will turn on your friends and business associates at the drop of a hat.

What more can I say?

Ok, thanks. So David Cote and DPP did not diss OpenReader by any public statements. Glad to hear that.

As noted before, DPP had no commitment to the OpenReader Consortium at the time, so they have nothing to do with this situation. Dragging them into this discussion is therefore inappropriate, no matter how much you try to spin it differently.

And, in effect, bringing this whole thing up in public as you did a couple weeks ago (or whenever), did grave harm to the OpenReader movement (and don’t say it was already in grave condition because it wasn’t — there’s a difference between having setbacks and grave harm.) It also may have harmed OSoft, whose codebase will form the basis of an OpenReader reading system, and DPP is none too happy, either, and understandably so (although they do appreciate the free publicity and are taking this in stride.)

The result of all of this is that it has become much harder for promoting OpenReader since others, who may have been considering helping out, or simply backing OpenReader, will now have second thoughts, thinking we might viciously turn on them, too, if they don’t do everything exactly to your satisfaction and timetable. Not good.

We are building totally voluntary alliances with other organizations, not marriages. We have to remain flexible when the business environment changes for those we have built alliances with. In the voluntary open source environment we work in, we don’t demand, we negotiate in good faith and adapt as necessary. And we are nice to those who ally with us. Giving public ultimatums (as well as private ultimatums) to alliance partners works against our goals. You did the exact thing which makes OSoft not want to have anything to do with you and possibly not with OpenReader. The public flogging of OSoft, DPP and me had the direct opposite of the intended affect, and reflects negatively on you. There were several things you could have done instead that would have had a much higher probability of success and which would not have left a trail of devastation. And if none of these were successful, one shrugs the shoulders, learns from it, and moves on.

The problem is, David, is that you’ve now so poisoned the playing field for OpenReader, it’s more difficult for companies like OSoft and DPP, and many other organizations, to even consider supporting OpenReader. You publicly flog two companies and one individual (me) for not doing what you want (and in the case of DPP they had made no agreement with OpenReader at that time), and then expect them to turn around and do what you want? And expect other companies and organizations to gladly welcome OpenReader?

Regardless of whether your position is right or not (in my view it is not), the approach you have used (your public blog) to take out your frustrations by giving a public flogging to your friends and business associates is a breach of integrity and decency, and totally counterproductive.

Bill McCoy writes:

the OCF specification *does* say something about what’s inside it!. The spec dictates that using the .EPUB file type implies that OEBPS content is therein.

Bill, thanks for the correction here. Good job. I do hope we don’t have to write .EPUB in all caps (:-)).

Jon Noring writes:

the draft OPS 2.0 has not yet sufficiently addressed these issues, and in my opinion have actually done a few things that will make these goals a little more difficult to achieve. (Inline Islands is the nasty one that throws a monkey wrench into the works — I may post a blog article about this in the near future. It is a solution to a non-problem that few will use. Other than maybe for MathML, I see little wide-scale use of Inline Islands, and MathML can be dealt with in a better and cleaner way. Yet, like an elephant in the living room, Inline Islands requires that solutions to various problems take its presence into account.

Thanks, Jon. I’d like to see more analysis of this type, when you get the chance.

Well, OpenReader is in many respects a vision of what the next-generation reflowable ebook and digital publication format should be. So long as the vision is accomplished reasonably and practically, it doesn’t matter what nameplate it goes under, or who does it.

Ultimately, our goal is to meet the real needs of the users of digital publications — they are our customer. They don’t care about names, but rather function.

I also want to say that I have not enjoyed continuing this conversation, and do not intend to continue it unless there are new accusations. But, I continued to comment to unambiguously note for the public record (since the blog articles plus the associated commentaries will likely be archived at Google and elsewhere for a very long time) that there are two sides to this story (and really 3 or 4 when we include OSoft and DPP).

I saw my integrity being impugned, as were OSoft’s and DPP’s. So when other interested parties in the future come across these published comments by all sides, they will see all points of view and decide for themselves the truth of the matter, as well as assess the integrities of the various involved individuals. Some will study this exchange in order to help answer the question: “Can I work with, and trust, so-and-so?”

Just an aside, David…

Genene Miller Coté is not simply David Coté’s wife who “presides over DPPstore,” she is the CEO of DigitalPulp Publishing.

Your continued attacks at David are misplaced. If you’d like to vilify someone… make sure you’ve done your research first.

There has been no attack on OpenReader, except by you. DPP is a proponent of well-formed standards for eBook creation. However, DPPstore, a retail site, has no interest in promoting ANY standard. Consumers don’t care why products work, just that they do.

-Catherine Hodge, Daughter of “David Coté”, also known as CTO for DigitalPulp Publishing.

Catherine, Mark and Jon…

CATHERINE

Many thanks for your reply. I find it fascinating that your retail e-book site has so aggressively promoted dotReader despite your apparent denial that your family has a business interest in the product. Are you certain that dotReader-specific capabilities aren’t part of your biz plans, especially with Jon involved as your father’s biztechdev guy?

As for “interest” in a more basic sense, I have reproduced the screenshot of the now-deleted promo from the store’s home page. Text read: “DPPSTORE is bringing you innovative fiction and non-fiction in this exciting new format…new in dotReader.” And a DigitalPulp news release even used the word “standard” in an OSoft context: “OSoft’s vision is to create a documentation standard through which publishers, authors, potential authors, and readers can share, collaborate, and exchange information in one common format.”

Catherine, has your site given other “formats,” “standards,” whatever, the same amount of loving attention lately? And why did the super-conspicuous dotReader promo vanish from DPPSTORE’s home page not long after I complained of Jon’s connection with your family? You’re nice people. It’s just that Jon’s new arrangement with your father reduces his detachment in the e-book standards world and his ability to safeguard the integrity of OpenReader.

As for precise corporate jobs, let’s just say: “All in the family.” Since you welcome “research” and have raised “who’s who doing what?” kinds of issues, however, perhaps you can reveal:

1. Jon’s exact relations with the various corporate arms of your family, and who runs those arms.

2. When the relationships started.

3. The financial arrangements.What are the sums and percentages already paid and/or agreed to?

If you and Jon don’t want this on record, for privacy reasons, sure, I’ll understand. But I’m extending the invitation.

Regarding your claim that there has been “no attack on OpenReader, except by you”–well, that would be news to the skeptics. Bill McCoy at the very least raised many questions, albeit more politely than I am. Bowerbird called OpenReader vaporware.

Because of my friendship with Jon and his past credibility, not to mention detailed progress reports from OSoft, I was willing to keep on evangelizing for OpenReader. But when he repeatedly failed to press aggressively enough for OSoft to keep its promises, when OpenReader creation-related activities and rendering capabilities became such small details in OSoft’s plans, and when Jon became a dev guy for David Cote, how could I have not joined the skeptics? I know. Software dev is full of uncertainties. But Mark so far hasn’t even given the standard an “OpenReader’s coming” logo on the home pages of dotReader and OSoft. Read on for why I didn’t insist on a logo all along.

Meanwhile I find it interesting that just about the only people defending OpenReader in a major, persistent way in the TeleBlog are you and Jon—joined by OSoft CEO Mark Carey (with Bill McCoy graciously cutting OpenReader some slack). Perhaps I’m forgetting some people, or other supporters will come out of the woodwork.

MARK

My requests for an OpenReader logo on your home pages began late last year, at least several weeks ago; and in fact you said you’d do one. But you never followed through.

Earlier I was so confident of OSoft’s support of the standard that it didn’t really occur to me to press you on these details. Your company represented itself as more than a little gung ho . See above for why I eventually became skeptical about OpenReader becoming a reality in a timely way. Meanwhile the proprietary dotReader “format” will have a head start.

The confusing language on your dotReader site has not helped, either, and I can understand why even sophisticated techies would feel that dotReader would be OpenReader-capable out of the gate. I know what you meant in the paragraph below, which might even be technically correct; but many a casual reader won’t pick up the nuances:

“The dotReader can convert and render any reasonably structured (non-encrypted) XML document with the appropriate plug-in. What does this mean? We are developing conversion tools that can make a dotReader publication from other document types. We are currently able to convert the OpenReader format, Open Office, many .LIT, and most Mobipocket formatted books. The upcoming beta release will actually contain books that originated in some of these (non-encrypted) formats that were converted into XHTML and rendered in dotReader.”

Notice? “The dotReader can convert and render…” And then three sentences later: “We are currently able to convert the OpenReader format…” So what’s meant by “convert”? Is it happening in the dotReader itself at the moment? No—because a plug-in doesn’t exist. Do you mean a conversion tool? Hand coding? Mark, you’re a very talented guy, and you can write persuasive, factual and clear copy, so I’m disappointed that such misleading gobbledygook went on your site.

I hope it’s obvious now why I’m not happy with either OpenReader or OSoft. Unless I felt that OpenReader had problems well worth airing, just why the devil would I be so public in disagreeing with you and Jon? This isn’t fun for me, either. But I promised publishers and the rest of the world that ThoutReader/dotReader would be linked to an open standard, and if Jon wants to go ahead with the OpenReader standard, then he has major problems to address. The Noring quote in the section ahead does not build much confidence in Jon’s ability to execute standards in a by-the-book way. He’s great as consultant, and I hope that the IDPF will pay plenty of attention to his plea for durable standards, just so they consider his ambitious plans for DigitalPulp and recognize that he is no longer so disinterested. Play to his strengths. Jon still has much to offer.

JON:

You write: “Well, OpenReader is in many respects a vision of what the next-generation reflowable ebook and digital publication format should be. So long as the vision is accomplished reasonably and practically, it doesn’t matter what nameplate it goes under, or who does it.” Wrong! If it happens as a good standard, including hopefully one from the IDPF, then I’ll be very happy. But that’s much different from it happening by way of a proprietary product like dotReader, the creation of just one company.

I’ll trust companies collectively—on standards matters—more than I’ll trust one or a few. Some people may say, “Oh, well, individual companies will eventually make their technology open source, etc.” But I’d rather not see the public forced to rely so much on faith. We go back to the small pond/big pond argument that I used against the IDPF vs. a major OASIS-style effort bring in techies from number of companies. Same applies to OpenReader. While I’m still highly skeptical of the IDPF, I’m even more skeptical of OpenReader—a much smaller pond, with a standard-setting committee of one who happens to be a biztechdev guy for David Cote.

I’m touched by your faith in David C and OSoft, however. This is why it’s hard for me to figure out your motives. I’d hope you were simply being naïve. OSoft and David may mean well, for all I know, but they are profit-driven and not to be confused with philanthropies.

Meanwhile I see a lesson for publishers, nonprofits and others dealing with standards organizations–big and small. Vet them to the hilt and don’t accept “nonproprietary standards” from small groups without a lot of study—not just of the technology but of the business relationships involved. Publishers, nonprofits and others may actually want to go with proprietary standards, regardless of major negatives, instead of problematic solutions from tiny OpenReader-style groups. At least the devils are known. Within the e-book world, of course, nonproprietary HTML-style approaches would be great when appropriate.

As for betraying personal relationships, I hope the OpenReader affair sends a message to anyone with whom I’ll dealing in the future. I’ll value integrity over cronyism. If people want me to bend the rules or go back on my promises—such as the one I gave to the publishers that OpenReader would not just be another occupant of the Tower of eBabel—then count me out.

Thanks,
David