Looking Out To Sea

It looks like the recent series of non-events relating to climate research and global warming have legitimised/mobilised the denialist factions of the Conservative party:

Most Conservative MPs, including at least six members of the shadow cabinet, are sceptical about their party’s continued focus on climate change policies, it has been claimed.

The recent furore around “Climategate” has hardened the views of Tory MPs, many of whom were already unconvinced by the scientific consensus, and has led to increasing calls for the issue to be pushed down the priority list.

I’m beginning to think that Climategate should be re-labelled “the media’s global warming hoax” because I’ve yet to see any article about the story that even approximately tells what we know. Every story serves to reiterate the general sense that there was falsification, manipulation of data or deletion of data series or correspondence — none of which is true.

As time goes on each “smoking gun” gets put into clear context and shown to be nothing of the sort, but the aura of suspicion — all these hundreds of stories about dishonesty — doesn’t disappear. It’s just now part of common knowledge that the scientists at the University of East Anglia’s Climate Research Unit (CRU) were doing bad science, even if every individual claim made about the research is untrue.

Which is how so many of the comments on this interview with Phil Jones (director of the CRU) are so disgusting:

I thought of killing myself, says climate scandal professor Phil Jones

Why didn’t you? Because you are a coward! Just like the rest of your free loading, elitist,socialist hack buddies living off free money from socialist governments to which you sell your soul and junk science!!! Go to hell and take your ilk with you! Miserable lowlife scum!

and

Jones, Mann and Hansen should all lose their degrees for what they have done to science. They have taken over $100 billion, ruined every new science book for two generations, poisoned science education with environmental BS and political correctness. These guys should be pushing brooms and driving cabs.

This is exactly the behaviour exhibited by the anti-vaccination crowd recently, when the GMC finally found Andrew Wakefield to have acted unethically — this from the pen of famous US anti-vaccination campaigner Jenny McCarthy:

The retraction from The Lancet was a response to a ruling from England’s General Medical Council, a kangaroo court where public health officials in the pocket of vaccine makers served as judge and jury.

Appeals to conspiracy and perceived political consequences without an attempt to grapple with the science. Every one of those commenters on the interview with Phil Jones knew that he was a lying, mendacious bastard, though there is no evidence of it at all; and every single one of Andrew Wakefield’s supporters knew there was a link between MMR and autism when they gathered to support him at the GMC hearing:

We have come here to show him we believe in him. There has been a witch-hunt against him when all he was trying to do was help people.

Of course, there was no evidence of that either. There has been very little soul-searching on the part of the media since the GMC verdict — indeed, plenty of “bad doctor done bad!” stories but very little admission of their own part in the sordid tale. I don’t hold out much hope for a change of tack over their coverage of Climategate and the later “scandals”, regarding the Himalayan glaciers or Amazon rain forests. From what I have read so far, the opinion pieces continue to lecture scientists on the benefits of self-criticism and humility. Irony is dead.

Some of you might remember a TV broadcast from a couple years ago called The Great Global Warming Swindle. The central thesis, that cosmic rays are the central cause of global warming, has been long disproved. (To make the film-makers’ case more appealing they, uh, “omitted” the last 30 years of data.) Two of the interviewees filed official complaints with Ofcom because their views were misrepresented and their scientific findings distorted in order to show the opposite effect. The producer has previous record on this point and it’s a wonder anyone wants to work with him at all.

I bring all this up to mention that I saw the Ofcom summary by accident the other day:

However, whilst Ofcom is required by the 2003 Act to set standards to ensure that news programmes are reported with “due accuracy” there is no such requirement for other types of programming, including factual programmes of this type.

You heard it here first — factual programmes do not have to be factual.

It seems documentaries, or programmes which look like documentaries, do not have to hew to anything we might call reality. Graphs, figures and statistics can be pulled out of the producer’s… hat and this wouldn’t matter.

The remainder of the ruling makes for some quite depressing reading. You can get away with whatever you want if you introduce your detractors as “the orthodoxy”, mention that they represent a “distortion of a whole area of science” and that they are conspiring to “invok[e] the threat of climatic disaster, to hinder vital industrial progress in the developing world”. Because despite all that you are letting the opposing view have a say. The excuses can stretch even further if your programme is viewed as being “polemical”, as if unsubstantiated nonsense is its own rightness.

Totally unrelated to the above, the same document also contains other rulings, the last of which quite amused me. It was regarding a complaint against subscription-only SportXXXGirls, in which the female presenters “perform[ed] explicit sexual acts” and “invited viewers to contact them for ‘adult chat’ via a premium rate text service”. The complaint was that the “live chat” was a repeat from the week before, which wasn’t obvious (unless you’d seen the previous screening, I guess…). I can only imagine how often they get complaints like this — I don’t know how many people consider complaining about subscription porn channels — but the result was that “Ofcom viewed the recordings supplied and noted that the material shown on the 10 February 2008 was a repeat of that shown on 3 February 2008”. What a strange job…

After however many pages and words and months I finished reading the last volume of Neal Stephenson’s Baroque Cycle, The System of the World. Back in August I was effusive with my praise of part one but never got round to saying anything when I finished part two (The Confusion). So let this entry mark as my feelings on both.

Brilliant! Read them now!

Ahem. The two books that comprise The Confusion are the intertwined tale of piracy on the high seas, and political intrigue in the court of Versailles. They follow on from the events of Quicksilver though many other characters drop out and others become more important. It’s in The Confusion that many of the important links to Cryptonomicon become obvious — many of the family names which appear in that latter book first make an important appearance here — the Hacklhebers, Gotos and so on. It’s also in The Confusion that Stephenson lets his nerdy side really come to the fore. The chapter about the creation of phosphorous from urine was exciting and informative and hugely enjoyable — all that from chemistry, you ask, but it really was damn good reading.

The events which unfold at the very start of Quicksilver aren’t mentioned again for the remainder of Quicksilver and the entirety of The Confusion. All that is basically flashback. The thread is picked up again in The System of the World as one party attempts to reconcile the deep philosophical rift between Newton and Leibniz while another attempts to sabotage Newton’s work at the Royal Mint. Again we’ve got political intrigue combined with stunning action sequences that would overwhelm the audience of most heist movies.

Rather unusually for Neal Stephenson he manages to end everything too. Maybe given 2500 pages of writing he has finally worked himself into the right position to pull it all together. His other books have disappointed me on the last page, but I was very thankful that such an epic series ended right.

Make that investment! Read these books!

We’ve had so much success with recipes in the last few years that it’s sometimes easy to forget that some recipes are useless through and through. If you’re not very familiar with the general idea then it’s easy to get dragged far from the path by a instructions that are confusing or just downright wrong.

That’s what happened to me last week when I tried Scrummy Chocolate Swirl Shortbread from the Green & Blacks Chocolate Recipes book. I couldn’t really remember what the process for shortbread was so I just followed the instructions and ended up with something useless quite demoralising. I later checked Delia and James Martin’s respective recipes for shortbread and confirmed that the recipe in the Chocolate book is utter bobbins. Compare:

• Cream sugar into butter, then add flour.
• Mix dry ingredients then rub in butter.

The first one gets you a stiff dough, the second ones gets you breadcrumbs. And at that point there’s not much you can really do to pull it back — it’s not easy to decrumb crumbs.

I’ve just tried again, ignoring the mixing process they suggested in favour of Delia Smith’s instructions, and they seem much better. They’re cooling at the moment. Now I get to revisit the original recipe and mark it up for future occasions. I’m not sure if I should write proper instructions, or just score the whole thing out with “Wrong! Consult Delia!”. That would be more satisfying.

Tomorrow (that is, 30 January 2010) there are going to be a number of demonstrations/protests outside branches of Boots, under the general name of the 10:23 campaign. At 10:23, a bunch of not-very-brave people will be “overdosing” on homeopathic pills. I think the plan is that each person takes a bottle full. This will, of course, have no downsides whatsoever because there is nothing in it.

The point in this case is not to point out the stupidity of homeopathy to the people in the street, though it will no doubt do that. It’s to make the point that Boots sell these things — have whole aisles devoted to these little white pills — even though they admit there is no evidence for their effectiveness. There are even Boots-branded homeopathic pills on the shelves! And at the same time they sell you something useless, they want you to know that their pharmacists are trustworthy enough to dispense medicines with active ingredients, and give advice about these medicines.

I leave you with James “the Amazing!” Randi, to explain the absurd details of homeopathy in his wonderful way:

About a fortnight ago now I went into a chiropody clinic for some medical attention on my big toe. I’ve had a bothersome ingrown toenail for quite some time and eventually found a doctor who pointed me in the right direction for treatment. Thank you Dr Reiff-Musgrove.

I received a couple of injections to numb the area and then they went at my foot with little bolt-cutters. It was fascinating to watch (though not if you’re a bit squeamish) — cutting down into the nail to the nail bed and then using a poky-stick to fill the resulting wound with phenol to inhibit nail regrowth. With luck I should have a mostly-normal nail when it all heals, but without risk of the same thing recurring.

For a few days I had a huge cartoonish bandage wrapped round my big toe which meant wearing a sandal on that foot. This is a very cold way to dress in winter. More recently the toe has been redressed with neater coverings which allow me to wear shoes again. I’m still trying to wear shoes as little as possible though to reduce pressure on the affected area. Which ultimately means that I wear the dreaded socks-with-sandals übernerd combination at home in the evening. (You can ignore this image and continue to think of me in a velvet smoking jacket and slippers if that makes the nightmares go away.)

So far I’ve been delighted by the sudden loss of background pain, which I’ve been growing accustomed to for several years now. The site of the operation still looks pretty horrible but it’s not painful. I can walk again to some degree, though I think it will be some time before I can don a pair of walking boots and go into the hills. I had intended to document the healing process with the camera but as the toe spends most of its time in bandages that’s proven quite difficult.

I look forward to doing all the usual feet related activities — like buying shoes that fit, ceilidhing without pain and wearing sandals without fear of frightening small children.

Taken from the man page for strncmp:

       The following sections are informative.
 
EXAMPLES
       None.
 
APPLICATION USAGE
       None.
 
RATIONALE
       None.
 
FUTURE DIRECTIONS
       None.

I hope that was a joke.

I posted a short guide yesterday to recovering photographs from corrupted memory cards. There were a few interesting technical points in that post that I glossed over for the sake of the explanation, but discussion on Facebook has prompted me to try filling in those gaps. Let’s have a look.

File systems

How is stuff written to and read from a disk, and why would it suddenly stop working?

Reading and writing disks

Picture a disk as a big sheet of grid paper. Each little square in the grid can hold a single letter, and that little square has a location, like the grid reference. We can read and write a single letter at a time by knowing its grid reference. Most things which are written on the paper will take up many little squares, because most useful information is more than one letter long — words, sentences, paragraphs and so on are stored as long sequences of individual letters. This doesn’t really pose a problem because our sheet of paper is easily big enough to hold all the sentences we could ever write. It is also possible to store many different sequences of text on one sheet of paper, so that books sit alongside essays alongside love letters alongside formal complaints. The only problem this does leave us with is organising and retrieving the information, since a gridded sheet where each box contains a single letter will start to look more like a wordsearch after a while.

If we have been sensible we will write our texts onto the great sheet of paper in a consistent manner, such as left-to-right, and starting a new row one line down whenever we fill up the previous row. The system of organisation we use is arbitrary — we could just as easily use right-to-left, write vertically or move up a row instead of down at the end of each line. It doesn’t really matter as long as we are consistent. (This should be obvious from the history of written language!) We can then identify the beginning of a piece of text by knowing the grid reference of the first letter; and we can retrieve the whole text if we know how long the text is, since it was written in a consistent direction. If we do this we should easily be able to organise all the information we have by keeping track of the starting location and length of all the sequences of letters we write.

The disk, like the gridded paper, is full of little slots where data can be stored; and each slot has a location called its “address”. Data can be stored on disk in long sequences of adjacent slots. To get the sense of the original data we only need to know the starting address and to read the correct number of entries from that point onwards. All we need to extract a file from disk is two numbers — the address and the length. From this we can reconstruct the original data.

Now we have many long essays and books and letters stored on the hypothetical sheet of paper, and these long sequences of text can be reached via the much shorter pairs of numbers we mentioned, representing address and length. But where are these numbers kept? The one location that doesn’t require any intelligence to find on a grid of paper is the start, which we shall call address 0 (zero). How can we use this to our advantage? We could write a list, starting at address 0, which contains the pairs of numbers needed to find all the other sequences of letters! It will never be possible to “lose” the sequence at the top corner of the sheet of paper so we will always be able to access this contents listing for the rest of the page.

But think again: if the sheet of paper is covered in many sequences of letters, it’s very conceivable that one of those sequences will contain map references or temperatures or other lists of numbers that could be mistaken for the pairs of numbers which we use to find the texts themselves. If a sequence of pairs of numbers starts at address 0 how do we know when it has ended and a different sequence of numbers has started? We know the numbered pairs start at 0 but we don’t know how long they continue after that. The solution is to enter the length of the first sequence at address zero itself. It then becomes possible to start at the top of the page and read off a list of starting locations for every piece of information on that page, without fear of accidentally reading data which isn’t relevant.

The organisation of real disks and memory cards is done along these lines, and the system of addresses and references which comprise the implementations are called a file system.

(Bonus question: Imagine that one of the sequences of text buried somewhere in our piece of paper was a list of location/length pairs just like the one at address zero. The location/length of this secondary list would be mentioned by the primary one, but the pairs of numbers it contain would not be mentioned. What use would this be? Can you think of a common instance of this sublist in computer file systems?)

Disk corruption and file recovery

In order to reuse a disk people often “format” it, making a clean slate. The formatting process generally doesn’t affect the data on disk, only the contents listings which help to make sense of it. We can format our grid of paper by altering the contents listing which we store as the first sequence. It’s not difficult — all we have to do is change the number at address zero, the one that represents the length of the contents listing, to a zero. From now on, reading that sheet of paper will immediately show that this sheet contains no data!

Quick and reliable access to the text written on the vast hypothetical sheet of paper hinges on the reliability of the contents listing at the top of the page. It allows us to make sense of the lines and lines of individual letters that follow. If this gets corrupted or erased in any way, our sheet of paper is turned from a useful store into a tedious wordsearch. And as you can see with the formatting example, it takes only a tiny change to render the file listing “wrong”.

So, if through chance or misadventure your disk becomes unreadable in this way, you will not be able to read the data straight off. But it’s all still there, somewhere. The process of file recovery is one of trawling through the disk looking for interesting sequences of text, like a word search. Thankfully many files such as images or videos are organised like a file system in miniature — they contain useful header and file length info which aids identification. For example, JPEG images, which is how digital cameras commonly store their images, start all their files with a particular number. File recovery programs can trawl through the disk looking for known sequences such as this and attempting to pull useful data from the mess. In many cases it works very well.

Well, I’ve just done my good deed for the day by pulling some photographs off a corrupted memory card. Helen’s camera stopped talking to its SD card on Boxing Day and the photos from Christmas Day were assumed lost. The solution was quite painless:

1. Copy the contents of the memory card to a file for safekeeping. The file ends up taking up as much space as the original disk does but it’s generally much nicer to take one copy and use it rather than risk further degradation if you’re worried that the memory card itself is dying.

   $ dd if=/dev/sdd of=memcard.img

   The card reader was named /dev/sdd from the viewpoint of my machine but this will no doubt depend on what kind of reader and how many devices you use. Adjust for your own circumstances. Typing dmesg into a terminal just after you’ve inserted the card in the reader will probably give you usable info — this is what I saw, in case it helps:

   [177907.297698] sd 5:0:0:1: Attached scsi generic sg4 type 0
   [177907.321601] sd 5:0:0:1: [sdd] Attached SCSI removable disk
   [177922.175874] sd 5:0:0:1: [sdd] 3994624 512-byte logical blocks: (2.04 GB/1.90 GiB)
   [177922.177838] sd 5:0:0:1: [sdd] Assuming drive cache: write through
   [177922.183593] sd 5:0:0:1: [sdd] Assuming drive cache: write through
   [177922.183608]  sdd: unknown partition table

   You can see it found the disk, named it sdd but couldn’t read any partition data from it. This is just as we expected because it got corrupted somehow.

2. Install file recovery tools. This installs a disk recovery program and something that will trawl through disks looking for lost files. It looks like a direct port of a DOS/Windows app because the help commmand lists all CLI flags as being prefixed with a / instead of --. But it still works well.

   $ sudo aptitude install testdisk

3. Run the newly installed photorec from the above suite. Any found files will be put into a newly-created directory called recovered. If this dir already exists it’ll create a new one with a numbered suffix rather than reusing it. I don’t know why.

   $ sudo photorec -d recovered memcard.img

   We were only expecting a few dozen photographs because the card had been emptied just before Christmas. In the end it pulled over 300 JPEGs and a video from the memory card — a nifty tool if you accidentally reformat a disk!

The Christmas photos were saved and Helen will no doubt be posting them soon. Stay tuned.

We got to see James Cameron’s new epic on Tuesday night, Avatar in 3D.

The three-dimensional aspect was very well done — much better than Beowulf in 3D which was hampered by terrible graphics. It looked fabulous and, who knows, maybe the added depth helped to make the action clearer.

Unfortunately the plot was fairly boring and the characters were about as shallow as it’s possible to get. In fact, it was largely the characters from Aliens — including Sigourney Weaver herself — fighting the Corporation and their unethical ways. They had a knuckle-head military guy, who appears to be a subtle blend of Duke Nukem and Kilgore from Apocalypse Now. No, I tell lie, there was nothing subtle there at all. They had a sleazy company guy and they had a mining operation set up to gather Unobtainium. I couldn’t help but snigger that they actually called their precious metal unobtainium. I wonder if they use it to make MacGuffins?