Sony eReader format (LRF) viewer for the Mac, Linux and Windows

2008/01/07

Thanks to Kovid Goyal, the Sony Reader’s very own terrible software (iTunes wannabe) can very soon be decommissioned from my software collection. The libprs500 project, multi-platform solution written in Python works very well and allows to move data between Sony and your Mac / Linux / Windows. The UI is actually pretty decent, considering that it is not native Cocoa app:

picture-8.png

You can also edit metadata, which is very important because of the limited screen space on PRS500. As added benefit, the application gives you a reader for LRF file format that allows reading unprotected LRF books on the Mac/Linux as well in addition to Windows. Here is a snapshot:

picture-9.png

And best for the end: the libprs500 can do format conversion from TXT, HTML, RTF and LIT (haleluja !) as well as PDF – as long as it does not contain too many images. Seems like Christmas arrived twice this winter – and we have still Macworld ahead 🙂

Huge thanks to Kovid who wrote this thing and to Peter who made aware of it …


BOTD: The Reader

2007/09/22

Today’s Blog of the Day is “The Reader”.

Everything about ebooks and readers – all the stuff I like. See at http://wowio.wordpress.com/

Here I discovered that Sony is assumably preparing new version of reader – PRS 505. From the changes that should be included, none is IMHO really so important. The memory capacity of the reader with SD cards (which are reaching 8GB these days) is much bigger than you ever will need. If I should pick some enhancements hat would make a huge difference, it would be:

1) price. Make it accessible, meaning ~ 150 USD (which is about 145 CAD 🙂 – just kidding)

2) content – price and availability of books. The fair price for eBook should not be more than 30-40% of the paperback. Without DRM, of course. Who would want any DRM from the company with 2 rootkit fiasco’s in last 2 years (first and second) ?

3) Software – Sony please stop trying to create these terrible slow knock-off of iTunes, and outsource it to somebody that will write decent client, preferably in Java, so that it runs everywhere. Considering Sony’s software history, making it opensource would help to restore the trust …

Speaking of software, it should provide an easy way how to download and format RSS feed and web pages for offline reading, ideally again something in the way how iTunes handles podcasts.

I do not think that adding WiFi (what many people call for) will work on the eInk device – speed and lack of interactivity would allow to read only very static Web pages, and user input handling would be a big challenge.


Experiment with Sony Reader battery life

2007/09/19

After returning from holidays with completely drained PRS 500 (see field test blog entry), I have decided to test what is the battery life of the fully charged reader without any usage at all. It was easy as I was pretty busy after being away 3 weeks.

The reader was fully charged on 29th of August (24 hrs with wall charger). After that, I did not use it at all, only checked every 2-3 days the status (the reader was switched off except this brief check). As of Wednesday morning, September 19th the battery was drained and the reader switched itself off right away (it is being charged as I type this). This gives us about 3 weeks of available standby time, without any use from page turning, until the fully charged battery is completely drained.

This result allows new angle in looking into the “7500 pageturns story”. Let’s assume that the battery indeed lasts for 7500 turns and let’s factor in the consumption of standby. When the reader is off, every week will take away about one third of the charge – or in other words you will loose battery capacity equivalent to 2500 page turns a week regardless whether you switch it on or not.

If you would turn page every 2 secs, 30 pages a minute, 1800 pages an hour, chances are that you might get closer to the ideal capacity. But if you read a book with a speed of 200 pages per hour (these are small pages) and have have time to read 2 hours a day, during you will “spend” about 7 x 2 x 200 = 2800 page turns – plus the weekly standby tax of 2500 page turns. With this, you will have available about 2200 turns – or less than a week in standby and if you will continue to read, you will have to recharge in about 9-10 days. Which is quite consistent with what I have seen o vacation: the reader died after two and half week (~ 6000 turns) plus about 1500 real pages.

So, the 7500 turns may not be such an impossible number – if you can read 1000+ pages an hour :-).

With the size of the screen, depending on the font and real page count books are anywhere between 500 and 1500 turns long. That means, one full charge is realistically good for 3-4 books.


Field test of Sony eReader

2007/09/17

One of the many gadgets I have packed for the summer vacation was of course Sony Reader and SD card with nice selection of texts. It was fresh and fully charged, so I assumed that I will be well prepared for the gaps in program during the following three weeks ahead of us and for the long flights over Atlantic.

Obviously, the flight was first moment where Reader was used. During the 7 hours, I spent about 3-4 hours reading. We flew through the night, but the overhead spotlight in airplane was more than sufficient and the letters were very well readable. I also enjoyed the almost non-reflexive surface of the screen.

As we were traveling, I did not read much during first week – maybe 4-5 hours in total. Where I really had some leisure time and used it again was Siena (by the swimming pool) and a bit on beach in Tirrenia and Lido di Ostia. For these light conditions, the Reader is probably only electronic device usable – it is perfectly readable in direct sunlight. Few years ago I tried to read during vacation outdoors on Pocket PC – with very dismal results.

As the trip was coming towards the end, so was the battery indicator on the PRS 500. No way I have made 7500 page turns – altogether, it could have been 1000-1500 pages, with some menu searching certainly not more than 2000 turns. The reader was on for about 20 hours (which should not count as it is not supposed to consume any energy in on state). During the two and half weeks and less than 2000 turns the battery was on single bar of four.

Foolishly enough, I have not packed the charger, relying on 3000-4000 pages capacity. What a mistake. I had plan B – using USB cable and charging through computer. I had with me Macbook – without Sony Connect software (which is available in Sony’s tradition for Windows only) – but I assumed that to charge all I need is powered USB port. Mistake number two – it did not work.

What happened was quite weird – the power indicator actually jumped to full after connecting, and stayed there while connected, but it looked like the computer actually drained the battery. After few hours I disconnected the reader, turned it on – and screen went blank. After reconnecting it to USB, it showed the cable indicator, but did not really charge – even when left attached overnight. As result, I was flying back without access to my ebooks. At home, after full recharge from the adapter, it behaved perfectly OK.

I am not sure whether this means that Reader really must be charged only by the wall charger – or whether the USB ports in Macbook did not have enough power for charge. I think I saw the charging via USB work before – but it was my Fujitsu Lifebook, which has the Connect installed. More experiments pending 🙂

Fortunately, we were well equipped and bringing back one suitcase full of boks purchased in my favorite Bratislava bookstore – so I survived :-).

Lessons learned:

1) do not trust Sony’s marketing. The 7500 page turns is an urban legend. I have reader for 8 months now and NEVER came even close to half of that number.
2) Always bring the charger – the battery will discharge even with very light use during 2-3 weeks.

3) Reader is best thing to have for outdoor, bright environments. Reading in limited, artificial light conditions is not so great – but workable. Overhead spotlight in airplane works fine.

4) Reader is good for about 20-25 hours and around 1500-2000 page turns without charge.

It is still not bad – I have no other device running for more than 4-5 hours without needing a recharge – but quite  a disappointment with respect to the high expectations.


Sony eReader – charging problem fixed ?

2007/08/06

With the latest software update (I am now on 1.0.3), one of the issues to be fixed was the “charging indicator behaviour”. Looks like it was indeed the software bug. After upgrade, the indicator jumped to full 4 bars and stayed there. Beforehand, I had usually only 3 bars regardless how long it was charged.

Time will tell – I am taking the reader with me to Italy to get some mileage out of it, during transatlantic flight …


Strange charging behaviour of Sony Reader

2007/06/04

Despite pretty impressive battery life – which is btw *nowhere* in the range of advertised 7500 page turns – over period of few months I have noted that the charging of the reader behaves kind of funny. Sometimes, even after being plugged-in for 24 hours, the battery indicator stubbornly stays on two squares out of four. Other times, right after connecting, it jumps right to full charge.

It is hard to really measure these things as the battery depletion is very slow on reading and I do not really listen to music on this device (that’s what iPods are for). I am not the only person who noticed this behaviour – Steve Gibson and Leo Laporte, who enhanced my favorite podcast Security Now!  to be an unofficial eBook Sony Reader news exchange 🙂 have seen the same thing (search for Sony). Btw, Steve and Leo – thanks for doing the eBooks plugs – I appreciate that a lot …

One possible explanation is that the problem is more in indication, not in charging. The software running the Reader (Linux) may get confused about what is the actual state of the battery – because even if the charge shows one half, it does not seem like the battery was indeed half empty.

Maybe what would help is to create few charge-discharge cycles by running the MP3 player which should use the battery in matter of hours, not days. I will try it out when I get to it – unless the next software update fixes the flaw (if it indeed is a software problem).


Converting eBooks to Sony Reader format

2007/03/22

Since yesterday, I made nice progress in solving my issues with content creation for PRS500 and it’s readability. There are several ways how to proceed:

The simplest is to download Book Designer. It is free for non-commercial use and current version 5.0 Alpha does the job very well. It allows you to load source in text, HTML, Lit, PDF, PalmDoc (prd/prc), rb and few other formats and process them into native LRF format – plus few others I do not really care about. The result is nice, readable LRF file with three sizes, nicely formatted, with metada. As added benefit, because the author is Russian, the program does not assume that English alphabet is the only one in existence and allows to select encoding. The result is quite good – most of the extended characters from Czech/Slovak are there, some are missing and displayed as space (namely ř,ě,ľ …) but it is readable. What is maybe better option is that with English as language and default encoding, the software “downscales” the extended characters to closest English pairs: ř -> r,ě -> e – which results in familiar computer Czech/Slovak. I am very comfortable with option 2, and will work on getting correct font for #1.

If you want to read more about the program go here and here – as long as you can read Russian. I found out that even after 22 years of not using Russian, I can still reasonably well read and understand it …

The program is useful for creating Palmbooks as well as Microsoft Reader Lit book. I did not try that yet. User interface of Book Designer is not exactly Apple-made – extremely technical,  geekish – looking like designed  by engineer for engineers 🙂  – here is how it looks like.  But it is the functionality that counts. Thank you – whoever made this possible :-).

If you want actually understand how the LRF format works and how the book is formatted on very low level, read the format spec and then download the BBeBinder from Google Code. It is C# 2.0 project, which aims to create something similar that BookDesigner – but as opensource, GPL-ed application. It is very early version (0.2) but in the true spirit of opensource, it actually (mostly) works. I have downloaded it and looked inside the code. The solution contains BBeB encoding/decoding library and main program, which was nicely designed with extensibility in mind. Using plugins, it allows to add additional input data formats (currently works well for text files, some HTML and I had mixed results with others).

If both of my projects were not in C# space (which is causing me being slightly over-see-sharped at the moment), I would not mind volunteering few hours into this – to make sure that Central European encoding is handled OK :-).


Sony eBook Reader – software updated

2007/03/21

After few weeks, I have got back to actually using my PRS500 again and I have finished the books loaded on the device. When I have connected the reader, first the CONNECT software as well as the PRS500 downloaded the updates. The firmware in reader is up to version 1.0.02.0130 from 1.0.00.21081 and should bring longer battery life and stability improvements. It better does, because so far the battery life is nowhere in the proclaimed range of 7000 page turns … Time will show.

The desktop software update actually did improve user experience a lot – it went from “terrible” to “almost acceptable”. New version added full screen preview mode and the interaction with CONNECT is much more pleasant – it does not freeze any more, contains “Eject” button to disconnect the Reader and (although it may be subjective feeling) the communication with the device is much faster. The program looks more and more like iTunes 🙂 – but I have no problem with that …

Last thing to do is to verify whether Sony did not install some rootkit, trojan or another pest on my machine. Since the last debacle with rootkit DRM I simply do not trust them any more. Good news is that the update did not require reboot, which lessens the likelyhood of a rootkit …

I  have played a bit with formatting content for the reading. So far I am mostly using TXT books because there is no conversion required. To get best results try avoid hard line breaks – leave long lines – one line per paragraph and separate the paragraphs with empty line. TXT format can be presented in all 3 sizes and usually is quite acceptable. The problems of TXT based books is lack of ANY formatting (beyond paragraph), no metadata (author, year, tags), no images … If your book is not in English, you may have problems with extended characters. So far I have not found good encoding that would present the extended Slovak/Czech characters correctly.

To get any kind of formatting, you need to use PDF, RTF or native LRF formats (the LRF format is also known as BBeB – broadband eBook). The PDF files needs to be specially formatted to be readable on Reader – ideally with the font size 12 to 20 and page size about 8.8 x 13 cm. More detailed instructions on PDF formatting from Sony are here. With PDF, device will allow only two font sizes. The Feedbooks web site offers some preformatted books for Reader.

For RTF, the size or the page does not matter so much, what is important is the font size. Best results are with 16 to 20 points and device alllows three sizes. The reformatting can be done in Word or OpenOffice. Unfortunately, same issue with extended characters is present in RTF rendering as well.

To get the best results, you need to convert eBook from TXT, PDF or RTF to LRF format. The LRF books can have metadata, table of content and nicer formatting with 3 sizes. I started to evaluate few tools to create the LRF content – will get back to it.

Few more good resources on the Net: the discussion forum on MobileRead  as well as Wiki. If you are into Shakespeare, here are his works in PRS500 format



More on Sony Reader

2007/01/27

I have been living with my Sony eReader for almost a month now and managed to read about two full books. If not the time crush in biometric project, I would have probably read much more than just two books – nevertheless, I do have some real life experience with the device.

First – the claimed battery life of 7500 page turns is BS. I have observed real battery life about 15-20 hours, so unless you manage to turn about 500 pages per hour, it just does not add up. I have observed that battery was good for about 1 full book (about 400 letter pages, or about 900 “small” pages on reader), read over 2 weeks period, the battery dropped from full charge to one segment. It is possible that it depends on how reading style – if you are turning pages and reading continuously, it lasts longer. Browsing through books seems to consume considerable energy. I think that 15-20 hours reading time is more than enough – it will keep you busy on flight to Australia :-). Certainly huge improvement against PDA based readers, where the battery life is 4-5 hours max. My 2 years old PocketPC is now down to about 3.5 hrs. I am still using it because a) I have lots of books in PalmDoc and CHM formats and b) I like to read in bed …

I have looked at the content of the SD card as it is in file system. The Reader does not seem to do any conversion for the files involved – the copied PDF and RTF files are binary equal to the original files. The structure of the SD card is:

\Sony Reader\
\Sony Reader\books
\Sony Reader\database

The books folder contains all files copied into SD. The database contains single XML file cache.xml, which holds directory of the books. I do not know yet whether the file indeed a directory or what it says – a cache. Simple experiment to do is to copy few files and test of Reader will find them and updates the cache. The book metadata has following format as:

<text author="AUTHOR NAME" page="0" part="0" scale="0" sourceid="37" id="269">
date="Sat, 30 Dec 2006 22:42:12 GMT" mime="text/plain"
path="Sony Reader/books/FILENAME.txt" size="339623"
title="BOOK TITLE">
<layouts>
<layout part="0" scale="0">
...

I am not sure how the one or more encoded binary chunks – layouts (for various scales) work. The encoded jpeg -thumbnail is on the hand pretty obvious.

The source of the information appearing in the metadata depends on the source format. For PDF and RTF files, Connect software seems to access the document properties, for TXT files I did not figure it out yet.

So to get good content on the Reader requires

a) get the metadata right and
b) convert the content into best readable format.

Which is, unfortunately the proprietary format BBeB. The readibility of the BBeB is far the best, it offer nicer text, better sizing and overall much better reading experience. The good news is, that it is possible to create content in this format and first few tools are appearing on the net. I will try them out and post the results here as soon as I am done. Until that time, you can download free ebooks (Project Gutenberg and others) on Manybooks.net directly in the eReader format.

This site is very good source of information on the eReader and things around it.


Sony eReader – first impressions

2007/01/03

I have got my eReader on 30th December, so after few days of playing with it, here are first impressions. I managed to read one book and played with it quite a bit, but I am still learning.

The quality of the display and “electronic paper” look and feel is great. It actually looks like paper, not like screen. The non-glossy texture surface has same visibility from every angle and it really reads like paper. I have no problem with the speed or lack of it – the page turn takes about 1-2 seconds, switching to full black and then redrawing is kind of cool when the page text “emerges” from the page. The picture is perfect, steady, does not strain the eyes, perfectly readable in sunlight. It is the best feature of the reader.

Disclaimer: I have not tried the Sony eBook store, did not buy any DRM-protected eBooks and have no intention doing so. Actually I am going to avoid it like a plague. The prices of their eBooks are ridiculous (e.g. Michael Crichton Next costs $15.96 in ebook format compared to $25-$27 in paperback). If you think about economics, production costs and actual author’s cut – this is pure greed. I only hope that the market will teach Sony a lesson and force them to get real – why would sane customer buy ebook which is locked to a new-to-market device with uncertain future ? Especially from company with such disastrous record when it comes to customer’s right protection – anybody wanna rootkit ? Fortunately, there is enough content available without it.

I have inserted SD card (thanks, Sony for not forcing us to go with otherwise useless Memory Stick) and loaded about 200 books in text, RTF and PDF format. The loading works only with supplied CONNECT software, so if you hoped (as I did) that you will copy the files on the notebook on the card and insert it, just forget about it. The CONNECT seems to be the only way how to get content on the device.

The same way as the screen of the reader is great, the CONNECT software is terrible. It tries to look like iTunes, but it fails in every aspect. It is sluggish on P4-3.2 GHz with 1 GB RAM (XP-Pro, do not blame Vista ;-)). It seems to “freeze” or lock up without visual indication while copying – and then it moves on. Uses 60-70-80% of CPU time. SO after about half an hour, I managed to have some content and started reading.

The absolutely best results are with TXT books, as long as you do not use hard line breaks, because – despite the many CPU cycles consumed – CONNECT did not bother reformat the paragraph and device obviously did not either. The device allows 3 or 4 sizes of the font. It does not allow to actually set the font, but the default font is OK.

Reading PDF files – which was one of the three main reasons I bought the device (next to large paper-like screen readable outdoors and long battery life) – was not so great. Imagine the lettersize page of PDF shrunk to the size of small paperback – less then half of the original. The characters are minuscule, very thin and pretty hard to read. Unlike with text, you have only two size options with PDF – bad and worse. Bad displays too small letters with no page margins, the worse adds some empty space around and makes the letters even smaller. Ouch. The only (partial) workaround is to switch the screen to landscape (by holding the size button for 5 seconds). In landscape the width of the page makes the characters almost well readable – for the price of seeing half of the page at once. I also noticed that one or two bottom lines in landscape appear to be “grayed” – probably software bug ?

The menu system looked strange at first, but then I got used to it. The device has limited bookmarking and navigation capability – you can store bookmarks, but you cannot search or jump to a given page. Partial compensation for it is history feature that remembers last pages per each book. If your books’ file names are long and start with the author name, and you have several books from the same author, the book list display will be pretty useless – because all you will see is the first 15-20 character of the file name. There is also no option of annotating or marking anything in the text. I guess I can understand why. Combination of touch screen and e-ink is probably very hard problem to solve and features like search or annotate require user input.

The device offers option of viewing pictures and play MP3 (unprotected). The images look pretty good for the B&W screen – the grayscale is quite nice. I did not try the MP3 -IMHO the user interface is too impractical for real usage as player when you are used to iPod, and of course CONNECT is a bad joke compared to iTunes …

If the above sounds like this is pretty terrible device, and I am sorry I spent about $450 CDN ($349 US plus Florida sales tax) – it is not true at all. I am very happy to have it, I actually like it a lot and use it as much as time permits. True, I had very high expectations, and except of the screen, Sony did not quite deliver. But it is version one – and not a bad version one. Let’s hope there will be better firmware and most importantly better desktop software sometimes soon.

Here is my wishlist of changes and improvements (in case somebody listens):
– reformat TXT files during conversion to remove hard line breaks
– “reflow” the PDF files to make shorter lines and readable font size
– selectable font type and more font sizes
– allow bold/italic for font selection
– allow transfer files on SD card without CONNECT software – at least for TXT files
– network capabilities – ideally wireless or at least Bluetooth – allowing either to access it as drive or (ideally) to use it as client to access network files
– more intuitive navigation with “breadcrumbs” – showing where you are
– show long filenames in two lines
– thesaurus (for us non-native speakers of English)
– paging buttons on both sides of the screen, not only on the left
– text-to-voice (like Apple Macbook has)
– clock, showing you the time in case you dive into book too deep 🙂

In the meantime, if you are preparing the content for reading yourself, you can do several things to improve your reading experience: Format your text files and DOC files to avoid hard breaks and use only paragraphs. If you can generate PDF, try format and “print” it for custom page paper sizes – e.g. half-letter. This way the characters will be large enough to be actually readable.

Happy reading.