The State of the Database State

A recurrent theme in these posts – and throughout Computerworld UK – has been the rise of vast, unnecessary and ultimately doomed databases in the UK.

But those stories have been largely sporadic and anecdotal; what has been lacking has been a consolidated, coherent and compelling analysis of what is going on in this area – what is wrong, and how we can fix it.

That analysis has just arrived in the form of the Database State report, commissioned by the Joseph Rowntree Foundation from the Foundation for Information Policy Research (FIPR).

On Open Enterprise blog.

Why Barclays Are Barking

The little brouhaha concerning the Guardian and Barclays Bank is a wonderful object lesson in how the Internet changes everything. Once those super-secret documents were put up for even a few seconds, the game was over: taking them down from the Guardian afterwards really is the proverbial closing of the stable door after the horse has bolted.

Inevitably, a copy has made its way to Wikileaks; inevitably that link is being exposed all over the place, which has led to the site being overloaded (do make a donation if you can: I've given my widow's mite). Barclays Bank can apply for as many injunctions as they like, the judge can - and probably will - huff and puff as much as he/she likes, but the game's over: this stuff is out.

And quite right too: these documents either show the bank engaged in something dodgy, in which case they should be published, or they don't, in which case there's no problem in them being public anyway, since the bank is asking for serious scads of public dosh, and is effectively being part-nationalised.

But even if it weren't, it would be folly to try to keep them secret now: it would only ensure that even more people write about them, and point to them, and maybe even read them. The rules have changed.

Тaking the War against Terror to a New Level...

..of utter, inane stupidity. Here's the grand summing-up of Brown's "new level":

Terrorism threatens the rights that all in this country should hold dear, including the most fundamental human right of all - the right to life. We know that terrorists will keep on trying to strike and that protecting Britain against this threat remains our most important job.

That tired old Blairite trope: the "right to life" as the "the most fundamental human right of all". Except that it's not a *right*: do I have a right to life when I'm suffering from a terminal disease? Do I have a right to life when I'm 123 years old? Do I have a right to life when the Sun explodes? "Right to life": an idiotic meme, which certainly has no "right to life".

What he should have said is this:

This government threatens the rights that all in this country should hold dear, including the most fundamental human right of all - freedom. We know that this government will keep on trying to strike and that protecting Britain against this threat remains your most important job.

RMS "Broke into Microsoft and Stole Software"...

...that, at least, is what this deranged story in the Komsomolskaya Pravda newspaper claims:

Ричард Столлман. В 1990 году он объявил крестовый поход против компании Microsoft и ряда других китов компьютерного бизнеса. Он взламывал сайты, где предлагалось купить новое программное оборудование. И потом раздавал его народу бесплатно.

[Via Google Translate: Richard Stollman. In 1990, he announced a crusade against Microsoft and several other whale computer business. He cracks the sites where the proposed purchase new software. And then handed out to the people free.]

Not quite sure why the newspaper has the word "Pravda" - truth - in its title given the utter incorrectness of this from just about every viewpoint. (Via Stargrave's blog.)

Of Blacklists and Barbra Streisand

In the wake of news that Australia's blacklist has been leaked, I came across these interesting comments:

"Because this is a secret that has been leaked, everyone will be after it.”

“Every Australian will want to know what they were not they were considered so irresponsible to not leave alone.”

Guys said the leakage is proof that the list will be continually leaked if the Internet content filters are enforced, which he said will completely undermine its effectiveness.

Of course, the classic Streisand effect applied to blacklists: as soon as you create one, everyone wants to know whats on it, and some will manage to do so despite the blacklists - thus ensuring that those sites will get far more traffic than if no blacklist had been created.

So blacklists are actually one of the most foolish ways of trying to censor: wonder how long it will take the authorities to work this one out?

GNU/Linux does "Fashion" Robotics

I'd not paid much attention to the Japanese "fashion" robot, until I came across this:

Japan's National Institute of Advanced Industrial Science and Technology (AIST) has demonstrated a Linux-based humanoid robot that will perform in a fashion show next week. The HRP-4C runs the robotics-focused hard real-time ART-Linux distro, which was released this week for Linux 2.6xx under GPL.

The HRP-4C robot and the open-source ART-Linux distro (see more farther below) were developed by AIST's Human Robotics Group (HRG). ART (Advanced Real-Time) Linux has been used in a variety of humanoid robot prototypes from the Japanese government-backed HRG/AIST, says the group. The newest HRP-4C model announced earlier this week has been a hit on YouTube (see below). Designed to look like a young Japanese woman, the robot stands (and walks) about five feet, two inches (158 centimeters), and weighs about 95 pounds (43 kilograms).

Inevitable, really: why would you use anything else?

Ballmer: GNU/Linux Will Win on Netbooks

Here's what he said:

"The economy is helpful. Paying an extra $500 for a computer in this environment -- same piece of hardware -- paying $500 more to get a logo on it? I think that's a more challenging proposition for the average person than it used to be."

I think this is a very frank analysis of the problem for Microsoft: after all, who's going to pay extra money just to get the Windows logo on a netbook, when they can get the same features for less with free software...?

TeachingOpenSource.org

One of the signs of a healthy ecosystem is that it is constantly expanding into new niches. Here's a new angle on opennes I hadn't come across before - a site devoted to the *teaching* of open source coding skills:

Open Source is becoming a dominant development model in the software industy. The next generation of software developers, computer scientists, system administrators, analysts, and build engineers need to understand Open Source and must be able to work efficiently within Open Source communities.

This is a neutral collaboration point for professors, institutions, communities, and companies to come together and make the teaching of Open Source a global success.

It already has its own planet of associated blogs, alongside a host of other useful content.

Mirror, Mirror, on the Wall...

...who is the secure-est browser of them all? The answer may surprise you...

On Open Enterprise blog.

Coming to an ID Card Near You: Your DNA

One of the many disgraceful aspects about the disgraceful ID card programme is the reluctance of the UK government to make key documents available. For such a momentous change in the relationship of government to governed, it is critically important that a full debate about all the issues be conducted; but without key details of the scheme, that is made more difficult – which is presumably why the UK government has resisted the publication of the so-called “Gateway reviews” so long.

Finally, though, we have gained the right to see these somewhat outdated documents. Despite their age, and the unnecessary redactions, some useful new information has come to light, which more than justifies the long battle to gain access.

On Open Enterprise blog.

Welcome to the New (Networked) News

There's some fine writing coming out of the current newspaper crisis. Here's some more, from one of the my favourite thinkers, Yochai Benkler. He's replying here to an earlier article in The New Republic; two paragraphs in particular caught my attention:

Critics of online media raise concerns about the ease with which gossip and unsubstantiated claims can be propagated on the Net. However, on the Net we have all learned to read with a grain of salt between our teeth, like Russians drinking tea through a sugar cube. The traditional media, to the contrary, commanded respect and imposed authority. It was precisely this respect and authority that made The New York Times' reporting on weapons of mass destruction in Iraq so instrumental in legitimating the lies that the Bush administration used to lead this country to war. Two weeks ago and then last Friday, The Washington Post was still allowing George Will to make false claims about the analysis of a scientific study of global sea ice levels without batting an eyelid, reflecting the long-standing obfuscation of the scientific consensus on the causes of climate change by newspapers that, in the name of balanced reporting, reported the controversy rather than the actual scientific consensus. On some of these, the greatest challenges of our time, newspapers have failed us. The question then, on the background of this mixed record is whether the system that will replace the mass mediated public sphere can do at least as well.

Absolutely: newspaper have their virtues, but as Benkler says, they certainly have their vices too. So criticising potential weakness in nascent news forms is perilously close to pots calling the kettle black.

This other point also struck a chord (well, it would do, wouldn't it?):

Like other information goods, the production model of news is shifting from an industrial model--be it the monopoly city paper, IBM in its monopoly heyday, or Microsoft, or Britannica--to a networked model that integrates a wider range of practices into the production system: market and nonmarket, large scale and small, for profit and nonprofit, organized and individual. We already see the early elements of how news reporting and opinion will be provided in the networked public sphere.

In other words, welcome to the new news: it's the future.

German Court Says Data Retention is "Invalid"

As part of a global conspiracy of Glyns, Glyn Wintle has kindly pointed me to this very interesting decision from those fun-loving German judges in Wiesbaden:

As the first German court, the Administrative Court of Wiesbaden has found the blanket recording of the entire population's telephone, mobile phone, e-mail and Internet usage (known as data retention) disproportionate.

The decision published today by the Working Group on Data Retention (decision of 27.02.2009, file 6 K 1045/08.WI) reads: "The court is of the opinion that data retention violates the fundamental right to privacy. It is not necessary in a democratic society. The individual does not provoke the interference but can be intimidated by the risks of abuse and the feeling of being under surveillance [...] The directive [on data retention] does not respect the principle of proportionality guaranteed in Article 8 ECHR, which is why it is invalid."

Now, IANAL, and certainly not a German one, but it seems likely to me that the Administrative Court is not the highest authority in the land (which would be something like the Federal Constitutional Court), so there's probably lots of to-ing and fro-ing still to come on this before a definitive decision is reached. But it's certainly a good start since the that judgment is in tune with commonsense: that data retention is disproportionate and violates privacy.

It's *Not* The 15th Birthday of Linux – and Why That Matters

Last week, I wondered whether I'd gone back in time. Everywhere I went online – on news sites, blogs and Twitter – people were celebrating the 15th birthday of Linux, it seemed. “How is this possible?” I asked myself. “Since Linux was started in 1991, that must mean we are in 2006: have I fallen through a wormhole into the past?”

On Linux Journal.

Follow me on Twitter @glynmoody

Keep Calm and Carry On: Freedom is in Peril

The Guardian has a nice story about the unexpected success of the "Keep Calm and Carry On" poster. But what struck me was the following:

This was the third in a series. The first, designed to stiffen public resolve ahead of likely gas attacks and bombing raids, was printed in a run of more than a million and read: Your Courage, Your Cheerfulness, Your Resolution Will Bring Us Victory. The second, identically styled, stated: Freedom Is In Peril.

How prescient they all were.

Facebook Users of the World, Unite!

Maybe this is the way to get the huddled masses roused up against the insane and disproportionate Interception Modernisation Programme:

The U.K. government is considering the mass surveillance and retention of all user communications on social-networking sites, including Facebook, MySpace, and Bebo.

Vernon Coaker the U.K. Home Office security minister, on Monday said the EU Data Retention Directive, under which Internet service providers must store communications data for 12 months, does not go far enough. Communications such as those on social-networking sites and via instant-messaging services could also be monitored, he said.

"Social-networking sites such as MySpace or Bebo are not covered by the directive," said Coaker, speaking at a meeting of the House of Commons Fourth Delegated Legislation Committee. "That is one reason why the government (is) looking at what we should do about the Intercept(ion) Modernisation Programme, because there are certain aspects of communications which are not covered by the directive."

Monitoring Face, MySpace, Bebo: just think of the embarrassing/illegal things you mentioned there. Now, do you really want a bunch of control freaks sifting through *that* little lot?

Home Office Utterly Clueless on Pornography

What a bunch of incompetent, arrogant fools:

The Home Office has admitted that it has been trying to force ISPs to subscribe to the Internet Watch Foundation's (IWF) blacklist, even though it doesn't know what the organisation does.

Speaking exclusively to Computer Shopper, a Home Office spokesman thought the IWF deletes illegal websites and doesn't look at the content they rate.

He also revealed that the government's measures to ensure that the IWF is blocking illegal content only consist of "meeting with the IWF fairly regularly for updates on how they're doing."

Against the background of countries like Australia secretly blocking Wikileaks, this use of unappointed censors that are never questioned or even checked by any kind of review body is really getting dire. When will these politicians come to their senses?

Indian Opposition, *and* Indian Government

I wrote recently about the main opposition party in India getting behind open source; it seems that the Indian government has also seen the light:

E-mail system of the Prime Minister's Office was under the grip of a computer virus for three months last year forcing officials to replace the software.

The technical glitch plagued the e-mail communication system of the PMO, which was based on the Microsoft Outlook Express, from February to April in 2008.

Although the extent of damage was uncertain, the PMO said that most of the e-mails addressed to it were not received.

The problem was detected only in late April after which the Microsoft Outlook Express email software was discontinued and replaced by another software - Squirrel mail.

"Squirrel mail" is presumably the open source Squirrelmail: "Webmail for nuts". (Via The Reg.)

The Russians Are Coming...

...well, the well-known Russian distro ALT Linux is getting active in that well-known hotbed of free software, Brazil:

Компания ALT Linux и OpenGO (Ventox Boundless Brasil) объявляют об открытии представительства ALT Linux в Бразилии.

Такой шаг со стороны ALT Linux обусловлен стремлением собрать вокруг репозитория Sisyphus максимально широкий круг разработчиков и является частью стратегии по расширению рынка за счет других стран.

Теперь продукты и услуги ALT Linux доступны и в Латинской Америке

Внимание к ALT Linux со стороны Латинской Америки наблюдается уже давно. Оно выражает как интерес к нашим продуктам и опыту внедрений с одной стороны, так и привлекательность открытой архитектуры с другой.

Вместе с португальской версией сайта, ALT Linux Brasil открывает свой интернет-магазин, в котором может будет купить локализованные для Латинской Америки дистрибутивы ALT Linux и услуги по технической поддержке дистрибутивов. Деятельность представительства будет ориентирована на работу с государственными учреждениями и учебными заведениями, а также с представителями бизнеса, которым будут оказываться услуги технической поддержки, внедрения и обучения.

[Via Google Translate: The company ALT Linux and OpenGO (Ventox Boundless Brasil) announce the opening of the representation of ALT Linux in Brazil.

Such a move by the ALT Linux is due to a desire to gather around the repository Sisyphus the widest range of developers and is part of a strategy to expand the market by other countries.

Now, products and services are available, and ALT Linux in Latin America

Attention to ALT Linux from Latin America, there has been a long time ago. It is as interested in our products and experience in implementation on the one hand, and the attractiveness of the open architecture on the other.

Together with the Portuguese version of the site, ALT Linux Brasil opens its online store, which can be purchased localized for Latin America ALT Linux distributions and support services to the distributions. Activities will focus on the representation of working with government agencies and educational institutions, as well as with representatives of business, which will provide technical support services, implementation and training.]

Open Enterprise Interview: Mike Olson, Cloudera

Yesterday, I wrote about the launch of the open source company Cloudera. It's always hard to tell whether startups will flourish, but among the most critical factors for survival are the skills of the management team. The fact that less than three hours after I sent out some questions about Cloudera to Mike Olson, one of the company's founders, I had the answers back would seem to augur well in this respect.

Olson explains the background to the company, and to Hadoop, the software it is based on: what it does, and why business might want to use it; he talks about his company's services and business model, and why he thinks cloud computing is neither a threat nor an opportunity for open source.

On Open Enterprise blog.

EU Telecoms Promote "Legitimate" Content

Interesting initiative from the European Telecommunications Network Operators' Association (ETNO):

ETNO is launching a new online content web site today, to raise awareness of attractive online offers put on the market by its members throughout Europe to download music, films or watch TV. ETNO members believe that offering a wide choice of online services is the best way to promote a legitimate use of the Internet and fight against illicit file-sharing.

"The rapidly growing choice of legitimate online content services illustrates the increasing cooperation between e-communications providers and content owners in order to respond to consumer demand for price-worthy, secure and user-friendly services”, says Michael Bartholomew, ETNO Director.

The new ETNO web site gives a non-exhaustive overview of services available including IP TV, video on demand or music downloads, offered by ETNO members through different platforms and devices to meet user’s demands.

"User-demand for content is the basis of our actions. ETNO members develop and promote business models for content online offers, including music, films and TV. This list will of course need to be continuously updated,” says Patrik Hiselius, TeliaSonera, Chair of ETNO’s Content Working Group.

Increasing choice of legitimate content online and raising awareness among users are the best instruments to fight against illicit file sharing.

“Illicit file sharing represents a major burden for all stakeholders, including internet service providers. Education is key. Users should not be unreasonably criminalised or stigmatised. Through this new web site, ETNO members show their commitment to play their part and cooperate with rightsholders under the existing legal framework, in a scenario where choice and availability for the consumer, and rights and privacy for the citizen are all fully guaranteed”, added Bartholomew.

ETNO calls on policy makers and stakeholders to work together in order to ensure the wide availability of legitimate content offerings and to enable new creative market-driven business models to emerge.

This isn't perfect - I have problems with this "illicit file sharing", and the phrasing of "users should not be unreasonably criminalised or stigmatised", but what's interesting is that it shows an awareness of the broader issues, and of the fact that customers have rights as well as holders of intellectual monopolies. It suggests to me that the telecoms companies are beginning to understand that things are changing, and are beginning to change their own stance in response too.

OpenStreetMap Passes 100,000 Users

Although many people have heard of OpenStreetMap, not everyone realises just how damn successful and important it is becoming. Here's one indication:

Sometime during Monday 16th March 2009 OpenStreetMap gained it’s 100,000′th registered user account!

Moreover, the graph on the page linked to above also shows that there are now over one billion track points on its maps.

I predict that soon people will be talking about OpenStreetMap as a demonstration of how openness works in the same awed tones as they do now about GNU/Linux.

Yet More Open Source for South Africa

Yesterday I wrote about India's opposition backing open source more fully, and here's a good update on what's happening in South Africa:

“The move to open source software has not been as fast as we would have liked, but we are now entering a new era. In the past, open source deployments were mostly spontaneous and ad-hoc. We now have a more systematic approach.” In years past many government departments pursued their own open source migrations, usually in isolation from one another, and with varying degrees of success.

...

Now, says Webb, the State IT Agency (Sita) is assuming the role of paving the way for OSS migration by finalising standards and conducting pilot projects to make it easier for all to implement open source software successfully. ... Webb also says that Sita expects all government department websites to be running on open source software “very soon”.

There's clearly a pattern emerging here around the world, as governments their loyal opposition first experiment with open source, and then commit to it whole heartedly.

Opening Minds about Closed Source

One of the most exciting experiences in blogging is when a post catches fire - metaphorically, of course. Often it happens when you least expect it, as is the case with my rant about Science Commons working with Microsoft, which was thrown off in a fit of pique, without any hope that anybody would pay much attention to it.

Fortunately, it *was* picked up by Bill Hooker, who somehow managed to agree and disagree with me in a long and thoughtful post. That formed a bridge for the idea into the scientific community, where Peter Murray-Rust begged to differ with its thesis.

Given all this healthy scepticism, I was delighted to find that Peter Sefton is not only on my side, but has strengthened my general point by fleshing it out with some details:

Looking at the example here and reading Pablo’s Blog I share Glyn Moody’s concern. They show a chunk of custom XML which gets embedded in a word document. This custom XML is an insidious trick in my opinion as it makes documents non-interoperable. As soon as you use custom XML via Word 2007 you are guaranteeing that information will be lost when you share documents with OpenOffice.org users and potentially users of earlier versions of Word.

He also makes some practical suggestions about how the open world can work with Microsoft:

In conclusion I offer this: I would consider getting our team working with Microsoft (actually I’m actively courting them as they are doing some good work in the eResearch space) but it would be on the basis that:

* The product (eg a document) of the code must be interoperable with open software. In our case this means Word must produce stuff that can be used in and round tripped with OpenOffice.org and with earlier versions, and Mac versions of Microsoft’s products. (This is not as simple as it could be when we have to deal with stuff like Sun refusing to implement import and preservation for data stored in Word fields as used by applications like EndNote.)

The NLM add-in is an odd one here, as on one level it does qualify in that it spits out XML, but the intent is to create Word-only authoring so that rules it out – not that we have been asked to work on that project other than to comment, I am merely using it as an example.

* The code must be open source and as portable as possible. Of course if it is interface code it will only work with Microsoft’s toll-access software but at least others can read the code and re-implement elsewhere. If it’s not interface code then it must be written in a portable language and/or framework.

Great stuff.

Update: Peter has written more on the subject.

India's Main Opposition Party Backs Open Source

As I've noted before, you can tell open source has entered the mainstream when political parties try to outbid each other in establishing their open credentials. Further evidence of this trend now comes from India, where The Bharatiya Janata Party (BJP), the largest opposition party, has released its “IT vision”, document which includes a healthy chunk of openness....

On Open Enterprise blog.

Why Music Companies are Doomed Regardless

One of the favourite tropes in the music industry is that they'd all be rolling in it like the good old days if it weren't for those nasty people downloading music for free. Here's a perceptive analysis that explains why that isn't so:

the newspaper industry is in the same death spiral as the recording industry, without the lawbreaking that’s commonly blamed for the recording industry’s troubles. And it seems to me that this poses a philosophical challenge to DeLong’s theory that the problem is a lack of respect for “property rights.” The decline of the newspapers is clearly a story of technological progress producing increased competition and entrepreneurship—precisely the sort of thing libertarians normally celebrate. The news business has gotten far more competitive over the last decade, and we’re now seeing a normal shake-out where the least efficient firms go out of business.

I think the fact that this is happening in an industry without a piracy problem should give us second thoughts about blaming the decline of other copyright industries on BitTorrent. The newspaper example suggests that even if we could completely shut down peer-to-peer networks, we should still expect the recording industry to decline over time as consumers gravitate toward more efficient and convenient sources of music. Piracy obviously accelerates the process, but the underlying problem is simply this: the recording industry’s core competence, pressing 1s and 0s on plastic disks and shipping them to retail stores, is rapidly becoming pointless, just as the newspaper industry’s core competence of pressing ink on newsprint and dropping them on doorsteps is becoming obsolete. Not surprisingly, when a technology becomes obsolete, firms who specialize in exploiting that technology go out of business.

Gotcha.