It turns out that if you want to reformat a partition on an external USB drive so that it’s encrypted, this is just a matter of doing the following:

sudo luksformat -t ext4 /dev/partitiondevice

…. where /dev/partitiondevice is the device for the drive partition you want to overwrite.  Obviously, this will destroy everything that was previously on that partition.

I like to use a proper filesystem for USB mass storage devices, but if you leave out the -t ext4 then the default is to use VFAT.

When you next plug in that drive, you’ll be prompted to enter the password that you picked when creating the partition – if you type that correctly, the drive will be mounted and usable.  (If you mistype it, you’re not given another chance to enter the password, so you’ll need to go to the command line and do: gvfs-mount -d /dev/partitiondevice to try again.)

One small thing is that the mount point in /media will be based on a UUID by default, but if you set the ext4 partition label, it’ll be mounted under that name in /media/ instead.  To do this, starting from when your disk is mounted, you can run mount without parameters to find the unencrypted device name and then unmount it and change the label:

$ umount /dev/mapper/udisks-luks-uuid-b7bbb2c8-etc
$ e2label /dev/mapper/udisks-luks-uuid-b7bbb2c8-etc topsekrit

If you unplug and plug in the disk again, it should be mounted on /media/topsekrit

If you just want a copy of today’s copy of The Guardian or The Observer for you Kindle, you can download one from this automatically generated page.  This post describes the script that generates that generates the file and the motivation for it.

Since moving to Switzerland, I’ve found that I really miss being able to get The Guardian in the morning on my way into work.  Unfortunately, reading the website on a phone (or any other device) is no substitute if you’re relying on data over the mobile phone networks – one really wants all the articles cached for fast navigation through the paper.  The solution for this should be my shiny new Kindle, but sadly subscriptions to The Guardian aren’t available in the Kindle store.  (There are many other papers available.)  Fortunately, The Guardian has an excellent API for accessing its content, and the lovely interface produced by Phil Gyford for reading the paper in a cleaner interface suggested that I could similarly generate a bare bones version of the paper for my Kindle.  I believe that this is permitted under the terms and conditions of the Guardian Open Platform, since I’m (a) including the advertisement linked to from each article, (b) linking back to the original article and (c) acknowledging that the data is supplied by that service.  If I’ve misunderstood, and in fact this is not allowed, please let me know.

To generate a book in the Kindle’s preferred format, you have to generate a .opf file describing the contents of the book, which refers to other files describing its text, images, structure, etc.  Then you can run a binary called “kindlegen” to generate a .mobi archive from those files that will work on your Kindle.  (The samples in the kindlegen archive and the Amazon Kindle Publishing Guidelines are quite sufficient to figure out how to do this.)  My script to generate the .opf and supporting files is far from elegant, but I’m very happy with the results that it produces – it’s a really lovely reading experience.  You can use the normal page forward / back buttons to go from page to page, while the left and right buttons on the five-way skip to the next article’s headline.  This means you can skip quickly through the articles that you aren’t interested in, but each article you do want to read is presented very clearly on the amazing eInk display:

There are a few articles for which the API won’t return the text, saying that rights for redistribution are not available – I’m still including the other metadata for these article and the link to the original article, so that you know what’s missing:

At the end of each article is the advertisement image that’s included – this is to comply with the requirements of the Guardian Open Platform:

You can download the generated .mobi file for today’s Guardian (or The Observer on Sunday) from this page:

A Kindle version of today’s Guardian or Observer

You can bookmark that page in your Kindle’s web browser. Then, whenever you select the bookmark and then “Reload”, then it’ll be refreshed with a link to that day’s generated edition of the newspaper for your Kindle, which you can download straight from that page.

If you’re interested in this project, or have any comments or suggestions, you can contact me by email at:

… or leave a comment below.  The script for generating this version of The Guardian is available at github.

• Hosting large amounts of data on a generic shell account is typically quite expensive.  Flickr‘s “pro” account subscription is a very good deal in comparison: as long as each photo is beneath 20 megabytes in size, you can upload as many as you like for $24.95 a year.
• The community aspect of sites like Flickr is very encouraging – it’s lovely to have random people say nice things about your photographs, and occasionally have people use them in articles, etc.

(Some people are put off from using Flickr by the appearance of the site, but its API means that there are plenty of alternative front-ends for viewing or presenting your photos, such as flickriver.)

The slight problem with switching to hosting on Flickr was that previously I’d indexed all my photos by the MD5sum of the original image, so several of my pages had links or inline images that pointed to an MD5sum-based URL on the old site.  It occurred to me that it might be useful in general to have “machine tags” on each photo with a hash or checksum of the image, so that, for example:

• You can simply check which photos have already been uploaded.
• You can find URLs for all the different image sizes, etc. based on the content of the file.

Unfortunately, I hadn’t done this when uploading the files in the first place, so had to write a script (flickr-checksum-tags.py) which takes the slightly extraordinary step of downloading the original version of every photo that doesn’t have the checksum tags to a temporary file, hashing each file, adding the tags and deleting the temporary file.  This add tags for the MD5sum and the SHA1sum, using a namespace and keys suggested in this discussion, where someone suggests taking the same approach. These tags are of the form:

  checksum:md5=c629c63f8508cfd1a5e6ba6b4b3253a8
  checksum:sha1=df44fc771660fbe7a2d6b2e284ae61e9ed3e377c

The same script can return URLs for a given checksum:

  # ./flickr-checksum-tags.py -m c629c63f8508cfd1a5e6ba6b4b3253a8 --short
  > http://flic.kr/p/7oQxqK
  # ./flickr-checksum-tags.py -m c629c63f8508cfd1a5e6ba6b4b3253a8 -p
  > [... the Flickr photo page URL, which WordPress insists on turning into an image ...]
  # ./flickr-checksum-tags.py -m c629c63f8508cfd1a5e6ba6b4b3253a8 --size=b
  > http://farm3.static.flickr.com/2552/4196574615_491c6387f8_b.jpg

The repository also has a script to pick out files that haven’t been uploaded, and a simple uploader script which will upload an image and add the checksum tags.  The scripts are based on the very useful Python flickrapi module and you’ll need to put your Flickr API key and secret in ~/.flickr-api

Anyway, these have been useful for me so maybe of some interest to someone out there…

Use pdflatex for Output

There are various different options for generating output PDF output in LyX, but it will save you trouble if you do everything using pdflatex in the first place. (I think this is the upshot of the slightly unclear advice in the FAQ on the subject.) This turned out to be particularly important because when your document is 50000 words long and has over 100 figures, the other methods take over 10 minutes to generate a PDF; pdflatex would finish in a couple of seconds. If you take this advice then you have to change the Document -> Settings -> Document Class option to pdfTeX, or you get some surprising errors. Also, I would strongly recommend that you only use PNG files for bitmap images and PDF for vector graphics. (PNG is obviously sensible, but in the case of vector graphics I found PS and EPS files unexpectedly awkward in terms of getting the orientation and clipping right.)

Incorrect Colours in Bitmap Graphics

I came across a bizarre problem where the colours would be slightly wrong for certain PNG files that I include in the document. (I suppose I should say “colors” too, just for the sake of searchers using American English.) This turned out to be a problem with full-colour PNG images with transparency (i.e. RGBA images), which my notes say is discussed further in these posts. Setting the PDF version as suggested in the first of those posts didn’t help me at all, so I had to convert all my RGBA PNG files to RGB. If you want to check for these files you can use file(1), something like:

find . -type f -iname '*.png' -print0 | xargs -n 1 -0 file | egrep RGBA

… and I fixed them by feeding the filenames (one per line) to a script like:

#!/bin/sh

set -e

while [ $# -ne 0 ]
do
 t=`mktemp`
 convert "$1" png24:"$t" && mv "$t" "$1"
 shift
done

Obviously you need imagemagick installed for the “convert” command.

Footnotes

By default there is no extra vertical space to separate footnotes, but I much prefer there to be a small gap. To do that, add to the following line to the document preamble:

\setlength{\footnotesep}{12pt}

Captions

By default, the formatting of caption text in floated figures looks very similar to the main body text. Somewhere on the web I found the recommendation to use the “caption” package to change this, e.g. by adding the following to the preamble:

\usepackage[margin=10pt,font=small,labelfont=bf,labelsep=endash]{caption}

Fitting Tables Onto Pages

Making tables fit on the page is annoying – just changing the text size often doesn’t reduce the overall size or causes a horrible font to be used. Resizing the whole table is the best way I found. Before the table (either in a float or in the normal flow of the document) I added the following in ERT:

 \resizebox{\textwidth}{!}{%

and then immediately after the table added, again in ERT:

 }

This scales the table such that the width of the table fits the page width.

Suppressing Pages Numbers For Full Page Figures

If you want to use a whole page for a floated figure, the page number can overlap with the figure or just look odd.  However, second tip here: http://wiki.lyx.org/FAQ/UnfloatingFigureOnEmptyPage works well to remove page numbers from all-page figures. To summarize, add the following to the preamble:

\usepackage{floatpag}
\floatpagestyle{plain} % Default page style for pages with only floats

Then, in ERT before the figure (but still in the float) add:

\thisfloatpagestyle{empty}
\vspace{-\headsep}

… and similarly, after the figure but above the caption add:

\vspace{0.3cm}

… or you may find the caption too close to the graphics.

Better On-Screen Fonts in PDFs

As explained in the second question in this mini-FAQ on generating PDFs from LyX you should use the outline font version of Computer Modern instead of the bitmapped versions. For me, this boiled down to going to Document -> Settings -> Fonts and setting the Roman font option to “AE (Almost European)”.

You can further improve the rendering of text in your output by using microtype. Just add

\usepackage{microtype}

… to the preamble. (These suggestions only apply if you’re using the pdflatex workflow as suggested above.)

Adding Footnotes in Tables

You might notice that adding footnotes in table cells doesn’t work. One answer to this is manually add them in ERT with \footnotemark and \footnotetext, possibly adjusting the counter as described in that FAQ entry.