03 June 2010

Why "Naked Transparency" Has No Clothes

Although I have a great deal of time (and respect) for Lawrence Lessig, I think his article "Against Transparency" is fundamentally misguided. And for the same reason I think these concerns are overblown, too:

The coming wave of transparency could transform this in a hugely positive way, using open data on costs, opportunities and performance to become a much more creative, cost-effective and agile institution, mindful of the money it spends and the results it achieves, and ensuring individuals are accountable for their work.

But it might make things worse, frightening senior managers into becoming more guarded, taking fewer ‘risks’ with even small amounts of money, and focusing on the process to the detriment of the outcome. It may also make public service less attractive not only for those with something to hide, but for effective people who don’t want to spend their time fending off misinterpretations of their decisions and personal value for money in the media. And to mirror Lessig’s point, it may push confidence in public administration over a cliff, in revealing evidence of wrongdoing which in fact is nothing of the sort.

First of all, I think we already have a data point on such radical transparency. Open source is conducted totally in the open, with all decisions being subject to challenge and justification. That manifestly works, for all its "naked transparency".

Now, politics is plainly different in certain key respects, not least because hackers are different from politicians, and there has been a culture of *anti*-openness among the latter. But I think that is already changing, as David Cameron's latest billet doux to opening up indicates:

the release of the datasets specified in the Coalition Programme is just the beginning of the transparency process. In advance of introducing any necessary legislation to effect our Right to Data proposals, public requests to departments for the release of government datasets should be handled in line with the principles underpinning those proposals: a presumption in favour of transparency, with all published data licensed for free reuse.

Now, I am not so naive as to believe that all will be sweetness and light when it comes to opening up government; nor do I think that open goverment is "done": this is the beginning or the journey, not the end. But it is undeniable that a sea change has occurred: openness is (almost) the presumption. And the closer we move to that state, the more readily politicians will work within that context, and more natural transparency - even of the naked kind - will become.

Moreover, shying away from such full-throated openness because of concerns that it might frighten the horses is a sure way to ensure that we *don't* complete this journey. Which is why I think concerns about "naked transparency" are not just wrong, but dangerous, since they threaten to scupper the whole project by starting to carve out dangerous exceptions right at its heart.

Follow me @glynmoody on Twitter or identi.ca.


Andrew said...

I'm sorry if this is getting a bit tedious, Glyn, but I agree with you. Again.

Transparency is only likely to cause the problems that Lessig suggests in a culture where no failure is tolerated. There are many organisations that already possess this characteristic. Many would describe the civil service as such an organisation. If you have such a culture, you have a big problem already. It both makes people very wary about making decisions, and also leads to a prevalence of spin.

Any scientist knows that no experiment is a failure: results that are not as expected are often as valuable (and may be more so) as those which confirm a hypothesis, so the initiator of such an experiment should not be vilified as failing. True transparency will reveal that *everyone* has "failed" at some point or another, and with any luck, after a while, the binary obsession of the media with "success" and "failure" will be seen as over-simplistic (actually, I'm probably being over-charitable to much of the media, who are only interested in failure).

More transparency will lead to the ability to cull, or to change projects, which are not working properly, and much more quickly than they otherwise would be. There are sections of the media for whom "experiment" is a dirty word.

The whole idea of a limited liability company was developed to encourage entrepreneurs to take risks (and in fact has its origins in Roman times). Risk taking is good. Continuing to compound a mistake is bad.

What is even worse, is making the same mistake that someone else has made, becuase there is no access to the knowledge that derived from making the mistake the first time round. Ben Goldacre has much to say about the vast sums wasted by the pharmaceutical industry in re-trialling the same compounds which have already been discovered to be ineffective by a company's competitors, because the knowledge of failure is a trade secret.

After all, every time a genetic mutation is passed on to another generation, a risk is being taken. With no mutations, there would be no evolution. But the vast majority of mutations are detrimental, providing no adaptive advantage.

Applying a bit of intelligence to which risks should be taken is the sort of intelligent design I can live with. But to apply that intelligence requires transparency.

Glyn Moody said...

@Andrew: great analysis - thanks (and I can live with the agreement....)

Jeremy Benett said...

Hi Glyn,

You and Lawrence Lessig are tacking an important area, and I side with you on this.

It's much deeper than pure mechanics of making information open. It represents a deep shift in our cultural attitudes. For that reason our new government will have to climb a huge mountain to make its openness policy really work.

For centuries we have been able to maintain a public persona and a private persona, a public morality and a private morality.

Now we see the two coming together. In the 1950's the prime minister (Churchill) could be serious ill, and no one knew about it. Now an unguarded remark caught on a microphone can seriously hamper your election prospects.

As Andrew points out we need to be more realistic about risk. In fact more generally we need to be realistic about other people.

At present we are in an intermediate state. We can know almost everything about anyone through technology, but it does require some effort, so it is only applied to the "great and the good". Thus we can hound a Cabinet Minister for not following expenses rules, while we ourselves have not been adverse to fiddling our own claims.

True openness will only become widespread, when it is seen not to have a cost. That in turn will only happen when it applies equally to everyone.

Arthur C Clarke and Stephen Baxter's novel "The Light of Other Days" is an interesting analysis of what such true openness might mean. I actually found it a rather dissatisfying book, but I think that only underlines just how monumental a philosophical issue this is.

But back to today. We have to start somewhere, and like you, I think the Open Source model has a lot to offer.

Glyn Moody said...

@Jeremy: thanks for the interesting points.

Exciting times...