Data has long been the staple of mash-ups, news articles, bar charts and…just about anything really. It’s just another word for information, preferably neatly arranged in nice rows so we can make a colourful map of MPs’ expenses. Or a word cloud of Obama’s inauguration speech. Hurray! These pretty toys have given journalists a reason to live again!

Occasionally though, the toys achieve something useful, and since Wordle the New York Times’ revolutionary Faces of the Dead in 2006, data + journalism has gone mainstream. It’s largely taken off in the US where the launch of data.gov earlier this year prompted calls in the UK for the British Government to do the same. The idea being, simply, more transparency in government. You can look at air quality in California. Or average wage information in Utah.

The Telegraph’s MPs’ expenses coverage has proven beyond all reasonable doubt that transparency is a) needed and b) achievable through data. And so, yanked by the American dog-lead, the British government will indeed follow suit. Maybe. Hopefully.

Speaking at Open Tech 2009, Whitehall munchkins John Sheridan and Richard Stirling laid down the first draft for a UK version of data.gov. The only data available to them was traffic logs, but they hope to sell the idea to civil servants nonetheless. Currently the pair are looking for ways to spin this to Whitehall. “It works with most of the big departments if we say we can make public services better, and that we can do this by making information about a service available,” said Richard Stirling wryly.

Except – and even data enthusiasts immersed in numbers cannot fail to have noticed this – the UK is in the worst recession since 1991 and Brown can barely find the funds to let young people actually have the jobs they want. Let alone funding a database for some XKCD-shirted nerds to plot 10 years’ worth of traffic data on Many Eyes.

So it’s unlikely we’ll be using a British open databank any time this year, but they’re trying.

Why should you care?

Because although all of this is happening centrally in Whitehall at the moment, eventually local authorities will have to publish their stats. Don’t hold your breath though, even the US has only managed to get two states on board so far. Do you know how much of a pain in the arse these things are to get hold of usually? I wanted to find out the mobile phone expenditure of one local council, and had to wait three weeks for a reply. And that’s not even a difficult Freedom of Information request. It’s the same sort of thing as PlanningAlerts – because information is freely available, it’s a lot easier for the public to kick up a fuss about a planning application. Or traffic logs. If that’s your thing.

What next?

It’ll happen, eventually. The downside is obviously it will cost the taxpayer money just when the government is looking to cut back everywhere but yes, it may lead to a general improvement in services. Largely by you spotting something dubious on your local council’s databank and then shouting at them about it.

The political response is hilarious, at first glance. Tragic, at the second. The immediate response to the leaks has been ‘whodunnit?’ because, rather conveniently, revealing whether Phil Woolas bought his wife a shirt with public money is information ‘held in confidence’. You can see the letter hinting at a police investigation at Guido Fawkes’ blog.

Publicly, MP’s sought to defend themselves over individual claims while spectacularly missing the point. It’s no longer about the details, but the disillusioning whole. It’s the snouts-in-the-trough mentality while the rest of us are being lectured to cut down our costs of living.

Meanwhile, the opposition has remained strangely quiet because…well, they’re all in on it. David Cameron has been quick to issue a general condemnation and assure the public the Tories’ expenses will surface sooner or later, but it’s not going to be pretty either.

Stuff to read and listen to:

8/5 - Telegraph: the original splash and follow-up material.

BBC Today programme: Harriet Harman floundering. Unfortunately I can’t find a clip of presenter Evan Davis reading the watery Tory response, saying that it was a ‘live issue’ and they wanted to see ‘how it would play out’, but that was gold too.

BBC online – MPs bite back, in a classic example of missing the point, and a nice summary of what is actually within the rules.

And finally…

Major kudos to journalist/Freedom of Information campaigner/superhero Heather Brooke. Tireless in her efforts to achieve greater political openness, including MPs expenses, she has been credited by Guido Fawkes for her part in this. She even inspired this humble FOI request….

Update: Even my ethics tutor Roy Greenslade acknowledges what a total legend she is.

Oh alright. I’ll summarise it here.

Since May last year, the government has been bleating about a central database in which to store information about how we communicate. Primarily it’s a counter-terrorism measure but it’s um, also quite a big privacy infringement. So under the Interception Modernisation Programme (which really, when you think about it, literally means ‘better ways of snooping’) the government is proposing ways to effectively ‘save lives’ while not spying.

The database isn’t going to happen any more, it says. Which is peculiar, because it then goes on to outline exactly how it would go about creating a data store, if it was allowed to. Which it isn’t.

Nothing’s law yet. Hence this consultation report where you – because that will be your data eventually – get to raise objections.

It’s an educational document. By the second page I am graced with such insights as: Communications data is information about a communication. And Jacqui Smith’s inappropriately beaming face on the introduction which, for your own safety, I have replaced with serious cat.

This is the bit where she says the database isn’t happening. But, in case you were interested, she tells you how it would work anyway on p25.

The next bit’s quite important though. And for all the paranoia, it’s not as though anyone will actually read your racist jokes/cybersex/ascii art.

Communications data does not include the content of a call or the content of any other communications event, such as an email. This consultation does not propose changing the law to collect or store the content of any communication.

No, but it will try and get everything but. Mostly by coercing service providers into doing it for them.

As it stands, VoIP services like Skype currently don’t hold that much information about you. Or at least, not enough to be of any use. Indeed, Skype isn’t actually a UK service provider, so it’d be down to ISP’s like BT, Virgin etc to actually collect the times and destinations of your Skype calls. They’d also have to arrange it nicely so that the authorities aren’t flailing around uselessly with numbers. This, aside from the database or doing nothing, is the ‘middle way’. And here, quietly tucked away, is the £2bn cost estimate to compensate the private sector.

What next?

UK service providers will take a look over these suggestions and respond. This isn’t good news for them; they’ll be bearing the brunt of the costs for a lot of extra work, and potentially compromising the privacy of their customers. Equally, the intelligence agencies are disappointed by this seemingly flimsy resolution. Not having a central database makes piecing together data about one individual more difficult. In short, no one has won.

I will give a cookie to anyone who can spot the grammatical mistake in all three seconds of my ferocious grilling. Video by Shona Ghosh/PA.