11 December 2009

Visualising Open Data

One of the heartening trends in openness recently has been the increasing, if belated, release of non-personal government data around the world. Even the UK is waking up to the fact that transparency is not just good democracy, but is good economics too, since it can stimulate all kinds of innovation based on mashups of the underlying data.

That's the good news. The bad news is that the more such data we have, the harder it is to understand what it means. Fortunately, there is a well-developed branch of computing that tries to deal with this problem: visualisation. That is, turning the reams of ungraspable numbers into striking images that can be taken in at a glance.

Of course, the problem here is that someone has to spend time and effort taking the numbers and turning them into useful visualisations. Enter the Open Knowledge Foundation, which today launches the self-explanatory site “Where Does My Money Go? - analysing and Visualising UK Public Spending” (disclaimer: I have recently joined the OKFN's Advisory Board, but had nothing to do with this latest project.)

Here's what the press release has to say about the new site:

Now more than ever, UK taxpayers will be wondering where public funds are being spent - not least because of the long shadow cast by the financial crisis and last week’s announcements of an estimated £850 billion price tag for bailing out UK banks. Yesterday’s pre-budget report also raises questions about spending cutbacks and how public money is being allocated across different key areas.

However, closing the loop between ordinary citizens and the paper-trail of government receipts is no mean feat. Relevant documents and datasets are scattered around numerous government websites - and, once located, spending figures often require background knowledge to interpret and can be hard put into context. In the UK there is no equivalent to the US Federal Funding Accountability and Transparency Act, which requires official bodies to publish figures on spending in a single place. There were proposals for similar legislation in 2007, but these were never approved.

On Friday 11th December the Open Knowledge Foundation will launch a free interactive online tool for showing where UK public spending goes. The Where Does My Money Go? project allows the public to explore data on UK public spending over the past 6 years in an intuitive way using an array of maps, timelines and graphs. By means of the tool, anyone can make sense of information on public spending in ways which were not previously possible.

There's currently a prototype, and a list of the datasets currently analysed available as a Google Docs spreadsheet. There are some really cool interactive visualisations, but I can't point you to any of them because they are hidden within a Flash-based black box – one of the big problems with this benighted technology. Once HTML5 is finalised it will presumably be possible to move everything to this open format, which would be rather more appropriate for a site dedicated to open data.

That notwithstanding, it's great to see the flood of information being tamed in this way; I hope it's the forerunner of many more like it (other than its dependence on Flash, of course) as governments around the world continue to release more of their data hoards. Meanwhile, do take it for a spin and pass on any suggestions you have that might improve it.

Follow me @glynmoody on Twitter or identi.ca.


Anonymous said...

Tip for a story: exFAT.

I'm quite worried about the situation, since there is no official way to use it beyond windows and patents for it are pending - one more reason for MS to drive software patents thru in Europe..


Glyn Moody said...


Rufus Pollock said...

Glynn: thanks for your notice and comments on the Where Does My Money Go prototype. Just wanted to respond on one point re. our use of Flash (you're not the only one to have raised this!).

For rich internet applications, the choice is between Flash and Javascript. Of the two, Flash is generally regarded as the industry standard, due to its flexibility, powerful capabilities, and browser penetration. There are other options (Java, SVG, Silverlight etc) but they don't have the same market coverage.

Our priority was to visualize the data we had as effectively as we could. We chose to use Flash because it is faster for rapid prototyping, and we can guarantee access for 99% of users [1]. The application would not be as complete, or work as smoothly, had we used Javascript, and given our limited budget and timeframe, we could not have supported IE6 [2].

You're quite right that it's impossible to deep-link into the application (something we ourselves discussed quite a bit while developing the prototype). This is entirely correct criticism, and one we hope to address in future versions. We're also keen to release an open API which will allow users to create their own visualizations on their favorite platform.

[1] http://www.adobe.com/products/player_census/flashplayer/version_penetration.html

[2] http://about.digg.com/blog/much-ado-about-ie6

Glyn Moody said...

@Rufus: thanks for the response.

I accept that Flash is the de facto standard: not quite sure that makes it an *industry* standard.

And as for Javascript's lack of support for IE6, that sounds like a feature, not a bug, to me...

In any case, I look forward to future versions, which I'm sure will address this issue, not least once HTML5 is finalised.