Update: It is now possible to buy an official subscription to The Guardian and The Observer. The rest of this post is now largely of historical interest if you just want The Guardian on your Kindle, but I’ve left the rest of the content unchanged for people who are interested in how I generated my unofficial version.
If you just want a copy of today’s copy of The Guardian or The Observer for you Kindle, you can download one from this automatically generated page. This post describes the script that generates that generates the file and the motivation for it.
Since moving to Switzerland, I’ve found that I really miss being able to get The Guardian in the morning on my way into work. Unfortunately, reading the website on a phone (or any other device) is no substitute if you’re relying on data over the mobile phone networks – one really wants all the articles cached for fast navigation through the paper. The solution for this should be my shiny new Kindle, but sadly subscriptions to The Guardian aren’t available in the Kindle store. (There are many other papers available.) Fortunately, The Guardian has an excellent API for accessing its content, and the lovely interface produced by Phil Gyford for reading the paper in a cleaner interface suggested that I could similarly generate a bare bones version of the paper for my Kindle. I believe that this is permitted under the terms and conditions of the Guardian Open Platform, since I’m (a) including the advertisement linked to from each article, (b) linking back to the original article and (c) acknowledging that the data is supplied by that service. If I’ve misunderstood, and in fact this is not allowed, please let me know.
To generate a book in the Kindle’s preferred format, you have to generate a .opf file describing the contents of the book, which refers to other files describing its text, images, structure, etc. Then you can run a binary called “kindlegen” to generate a .mobi archive from those files that will work on your Kindle. (The samples in the kindlegen archive and the Amazon Kindle Publishing Guidelines are quite sufficient to figure out how to do this.) My script to generate the .opf and supporting files is far from elegant, but I’m very happy with the results that it produces – it’s a really lovely reading experience. You can use the normal page forward / back buttons to go from page to page, while the left and right buttons on the five-way skip to the next article’s headline. This means you can skip quickly through the articles that you aren’t interested in, but each article you do want to read is presented very clearly on the amazing eInk display:
There are a few articles for which the API won’t return the text, saying that rights for redistribution are not available – I’m still including the other metadata for these article and the link to the original article, so that you know what’s missing:
At the end of each article is the advertisement image that’s included – this is to comply with the requirements of the Guardian Open Platform:
You can download the generated .mobi file for today’s Guardian (or The Observer on Sunday) from this page:
A Kindle version of today’s Guardian or Observer
You can bookmark that page in your Kindle’s web browser. Then, whenever you select the bookmark and then “Reload”, then it’ll be refreshed with a link to that day’s generated edition of the newspaper for your Kindle, which you can download straight from that page.
If you’re interested in this project, or have any comments or suggestions, you can contact me by email at:
… or leave a comment below. The script for generating this version of The Guardian is available at github.
Great job :)
It’d be really nice if this were offered as a web service where people can register their kindle usernames and have a daily cron job run that creates a guardian mobipocket file and e-mails it to everyone who registers via their @kindle address. Resulting in anyone interested getting the Guardian delivered directly to their kindle each day and removing the onerous requirement of each user needing to apply for their own guardian open platform key.
Mike: Thanks for the suggestion. The problem I see with that is that even after compression the .mobi files are several megabytes in size, which at $0.99 per megabyte could be a significant cost. I’m reluctant to set up a web service that would end up costing people an unpredictable amount of money.
My real hope is that someone at the Guardian notices this and sets up an official Kindle version, of course. :)
An official Kindle version is now available: http://www.amazon.co.uk/The-Guardian-and-the-Observer/dp/B004MME3M8/ref=sr_1_2?ie=UTF8&m=A3TVV12T0I6NSM&s=digital-text&qid=1310129212&sr=1-2
Great, thanks for letting me know! I’ve subscribed to that now, and it seems to be very nicely done. I’ll update my various pages with pointers to the official version.
Hi, I might be really wrong on this but I’ve seen elsewhere that it’s possible to get Calibre setup on you PC to pull together the guardian articles (I don’t know how well it does this) or probably even load it with the download you offer, and it will automatically send it out daily to your Kindle email address. In my case if I could get it working, I would be picking up that email over wifi therefore for free.
Will: is there some way to say that you only want to receive things sent to your Kindle email address when you’re on wireless LAN? (I haven’t tried sending anything to it in case I get charged.)
Is this the sort of thing you’re looking for? http://blog.garethj.com/2011/01/21/free-daily-news-to-a-kindle/
HI Mark, yes I think there is an email address you use (http://www.amazon.com/gp/help/customer/display.html?nodeId=200375630) such as name@free.kindle.com – I think you need to set it up following instructions at that site. It’s not explained massively well. My Kindle is wifi only which as far as I know doesn’t get any charges applied (not using 3G). Presumably if you sent it to the free email it will use either 3G or wifi hopefully for free. And there may be a network ‘order of preference’ type setting so you can turn 3G off or something. But for me this would work very well in updating my Kindle with content before leaving the house of a morning.
Thank you, man! Been looking a long time for this.
Enjoy and thank you, thank you, thank you!!
I can’t understand why is The Guardian the only major newspaper missing in the Kindle. I would gladly subscribe…
Best.
This is amazing!
Would love to see a Blind [Carbon] Copy (BC) mailinglist, where we could leave our “name”@FREE.kindle.com addresses (for free delivery). We have to add the @longair address to the Kindle approved e-mail list though, so that you are authorized to send attachments to our Kindle.
Ingmar – just what I’m looking for as well. Have you found anything yet?
Have to leave my laptop on at the moment to do it at the moment, but would be great if someone set something up…..
i have just bought a kindle for the sole purpose of being able to download the guardian every day when i am out of the u.k. on trips. so it is a huge disappointment to find that the guardian is the one newspaper that cannot be downloaded to the kindle.
please make the guardian available to kindle users as i would gladly subscribe.
n.b. downloading the guardian onto a kindle is more environmentally friendly than buying the tree paper.
roisin o’hara: Just to make it completely clear, I don’t work for the Guardian, so it would be better to complain to the newspaper about that directly rather than here. What this page does gives you, however, is a link you can bookmark in your Kindle’s web browser that on reloading each day will have a link to a version of that day’s Guardian with much of the content you would find in the newspaper itself.
Guys, you can have any newspaper on your Kindle automatically and for free – as long as they publish article summaries in RSS. There are plenty of sites that let you build your RSS list then send every new article to your e-reader. And guess what: some of them even send you full articles, not just the RSS summaries (one that I know of is http://www.tomykindle.com). Personally I don’t read anything that’s longer than 10-20 lines on a monitor – I just sent it to my Kindle to read later. On a toilet, presumably :D
Thanks a lot for this Mark.
I also hope the Guardian provides a subscription service soon… I haven’t gotten my Kindle yet, but this was my primary worry in getting one… And you have just solved it!
Cheers.
Hi Mark,
This is great! Thanks very much!
Just to be clear: does this method require you to have a Guardian digital subscription (I currently have one) and is the content the same as the print edition?
Guardian not working on Kindle has been the only reason I was holding off buying one! (as the “read on kindle” option from the on-line edition is truly awful). If this works I will order my Kindle today!
Thanks so much for this!
GK
Gammakid: thanks for you kind remarks. To answer your questions:
* No, you don’t need a Guardian digital subscription.
* The content isn’t the same as the print edition because some proportion of the articles are not available via the API for free – I guess this is usually for content that is sourced from another news provider whose terms and conditions don’t allow the articles to be made available for free. This is sometimes frustrating, but I don’t mind too much. In addition, the pictures are only small thumbnails, and you don’t get the crosswords, puzzles and lovely typesetting of the digital edition.
What do you mean by the “read on kindle option from the on-line edition”?
Hi Mark,
Thanks for the reply,
Sorry if that was unclear, by “on-line edition” I meant the digital (subscriber) edition, which has the option to read off-line and export to any e-reader format. Until very recently the formatting of this was dreadful. It seems like they have adressed this now (possibly in the last few days in fact) but there still seems to be no table of contents to help navigate the articles. I’ve only tried this on the desktop version of the kindle however, and although it’s an improvement on their previous offering it still doesn’t seem as nice to use as the versions via your link or from Caliber.
So if there is any way to gauge the level of content that is excluded in the API content it would be useful in making a decision as to which path to stick with. What type of content would they typically source from other providers?
GK
Gammakid: thanks for explaining about the digital subscriber Kindle version – I didn’t know that existed.
You can get a sense of how much is missing on any given day by going to Phil Gyford’s “Today’s Guardian” and flicking through the articles with the right arrow key looking for those whose text is just “Redistribution rights for the article body are unavailable.”
My generated Kindle version uses the same source data, so the same articles should be unavailable in both.
The other thing to bear in mind, of course, is that currently my version isn’t delivered automatically, as the periodicals you would buy from Amazon are. I’m working on an option to have it delivered to @free.kindle.com email addresses but that’s not done yet. On the other hand, I think this version is a very nice reading experience.
Thanks for the link, it seems like all the “main” articles are there, with bits missing mainly from the comments/world news sections…
The digital edition of the Guardian is also not delivered through Amazon, it comes from http://newspaperdirect.com/ and must be downloaded each day manually, so the convenience is about the same but without the cost of monthly subscription :)
I presume you are also aware of the version available for download through Caliber? This is also free but can be auto delivered to your Kindle through the Caliber software, i’m unsure if the articles are the same as your version however…? (i’ll have a check over the next days)
Anyway, great work! i’m off to order a Kindle! :)
GK
Great Work!
I’m trying to do some thing with the periodical format mobi file. But the output is always in book format. What version of kindlegen are you using? Could you please sent me a copy?
Thank you very much.