Companies Developing Crowd Analysis Programs To Detect 'Abnormalities' In Behavior And Match Faces Against Giant Databases

One of the reasons that the total surveillance programs of the NSA and GCHQ are possible is that computers continue to become more powerful and cheaper, allowing ever-more complex analyses to be conducted, including those that were simply not feasible before. Here's another example of the kind of large-scale monitoring that is now possible, as reported by Nikkei Asian Review: 

On Techdirt.

Europe's 'Database Right' Could Throttle Open Data Moves There

One of the more benighted moves by the European Union was the introduction of a special kind of copyright for databases in 1996: not for their contents, but for their compilation. This means that even if the contents are in the public domain, the database may not be. Thanks to a recent court judgment in France, this "database right" now threatens to become a real danger for the burgeoning open data movement in Europe (original in French). 

On Techdirt.

Classic Function Creep As EU Police May Gain Access To Asylum Seekers Fingerprint Database

As Techdirt readers well know, one of the problems with measures brought in for "exceptional situations" -- be it fighting terrorism or tackling child pornography -- is that once in place, they have a habit of being applied more generally. A case in point is the blocking of Newzbin2 by BT in the UK. That was possible because BT had already installed its "Cleanfeed" system to block child pornography: once in place, this "specialized" censorship system could easily be deployed to block quite different sites. 

On Techdirt.

Swine Flu in the Nude

This is what the virus really looks like:

1 atgaaggcaa tactagtagt tctgctatat acatttgcaa ccgcaaatgc agacacatta
61 tgtataggtt atcatgcgaa caattcaaca gacactgtag acacagtact agaaaagaat
121 gtaacagtaa cacactctgt taaccttcta gaagacaagc ataacgggaa actatgcaaa
181 ctaagagggg tagccccatt gcatttgggt aaatgtaaca ttgctggctg gatcctggga
241 aatccagagt gtgaatcact ctccacagca agctcatggt cctacattgt ggaaacatct
301 agttcagaca atggaacgtg ttacccagga gatttcatcg attatgagga gctaagagag
361 caattgagct cagtgtcatc atttgaaagg tttgagatat tccccaagac aagttcatgg
421 cccaatcatg actcgaacaa aggtgtaacg gcagcatgtc ctcatgctgg agcaaaaagc
481 ttctacaaaa atttaatatg gctagttaaa aaaggaaatt catacccaaa gctcagcaaa
541 tcctacatta atgataaagg gaaagaagtc ctcgtgctat ggggcattca ccatccatct
601 actagtgctg accaacaaag tctctatcag aatgcagatg catatgtttt tgtggggtca
661 tcaagataca gcaagaagtt caagccggaa atagcaataa gacccaaagt gagggatcaa
721 gaagggagaa tgaactatta ctggacacta gtagagccgg gagacaaaat aacattcgaa
781 gcaactggaa atctagtggt accgagatat gcattcgcaa tggaaagaaa tgctggatct
841 ggtattatca tttcagatac accagtccac gattgcaata caacttgtca gacacccaag
901 ggtgctataa acaccagcct cccatttcag aatatacatc cgatcacaat tggaaaatgt
961 ccaaaatatg taaaaagcac aaaattgaga ctggccacag gattgaggaa tgtcccgtct
1021 attcaatcta gaggcctatt tggggccatt gccggtttca ttgaaggggg gtggacaggg
1081 atggtagatg gatggtacgg ttatcaccat caaaatgagc aggggtcagg atatgcagcc
1141 gacctgaaga gcacacagaa tgccattgac gaaattacta acaaagtaaa ttctgttatt
1201 gaaaagatga atacacagtt cacagcagta ggtaaagagt tcaaccacct ggaaaaaaga
1261 atagagaatt taaataaaaa agttgatgat ggtttcctgg acatttggac ttacaatgcc
1321 gaactgttgg ttctattgga aaatgaaaga actttggact accacgattc aaatgtgaag
1381 aacttatatg aaaaggtaag aagccagcta aaaaacaatg ccaaggaaat tggaaacggc
1441 tgctttgaat tttaccacaa atgcgataac acgtgcatgg aaagtgtcaa aaatgggact
1501 tatgactacc caaaatactc agaggaagca aaattaaaca gagaagaaat agatggggta
1561 aaactggaat caacaaggat ttaccagatt ttggcgatct attcaactgt cgccagttca
1621 ttggtactgg tagtctccct gggggcaatc agtttctgga tgtgctctaa tgggtctcta
1681 cagtgtagaa tatgtattta a

Amazing what a few As, Cs, Gs and Ts can do.... (Via Common Knowledge.)

ID Card Database *Already* Breached

That's almost before it's come into existence:

The breaches of the Customer Information System (CIS), which is run by the Department of Work and Pensions, were revealed in a DWP memo to housing benefit and council tax benefit staff on 15 January.

CIS is designed to give local authorities access to citizens' data, including HMRC tax-credit information. In 2006, it was decided that the ID card project would use CIS for biographical information, to avoid having to create a new, monolithic database of the UK's inhabitants.

In the DWP memo, the government department said that desktop access to CIS had helped to "significantly improve service delivery" to citizens, but noted that a series of checks had identified that some local-authority staff were committing serious security breaches using the system.

What makes it even more risible is the following comment:

"The breaches were not necessarily someone purposely going on there and checking something they shouldn't," the DWP spokesperson said. "They could be inadvertently clicking on information."

Yes, that will be a good excuse, won't it: honest guv, I just inadvertently clicked on Gordon Brown's ID card information....

And then, of course, there is the canonical "white is black", "up is down", "bad is good" bit of spin:

The DWP's spokesperson did not respond to a request to describe how it might be possible to break these rules by inadvertently clicking on information in the CIS database, but did claim the number of breaches revealed in the memo showed the system was secure.

And presumably it will use the increasing number of breaches to prove the increasing security of the system in the future.

Ingres Paints a Rosy Picture

If you have a good memory, you might recall a 2003 research paper from Goldman Sachs called “Fear the Penguin”....

On Open Enterprise blog.

Where China Leads...

...can Jacqui be far behind?

All visitors to internet cafés in Beijing will be required to have their photographs taken in a stringent new control on the public use of cyberspace.

...

According to the latest rules, by mid-December all internet cafés in the main 14 city districts must install cameras to record the identities of their web surfers, who must by law be 18 or over. There are more than 250 million internet users in China, approximately 10 times more than there were in 2000.

...

All photographs and scanned identity cards will be entered into a city-wide database run by the Cultural Law Enforcement Taskforce. The details will be available in any internet café.

Well, if it's got a centralised database, Labour's bound to want one to add to its growing collection....

IDiotic or What?

The chief executive of the Identity and Passport Service has said the ID cards database will not be completely secure.

James Hall said on Thursday that, after a string of high-profile data breaches in the past year, people should be concerned about the security of their personal information held by the government.

"You would rightly be concerned about the integrity and security of the information held about you," said Hall in a speech at the Homeland & Border Security Conference 2008 in London. "The issue has been heightened by recent events. I won't stand in front of you and say there will never ever be a breach of information."

Oh, that's alright, then.

To Open DB2, or Not To Open DB2: That is the Question

Interesting:

IBM is positive about the possibility of bringing out its DB2 database-management software under an open-source licence.

While the computing giant has no immediate plans to open-source DB2, market conditions may make it unavoidable, according to Chris Livesey, IBM's UK director of information management software.

Are Closed Source Databases Doomed?

On Open Enterprise blog.

"Open Source Does Not Mean Free": Huh?

Here's an interesting little to do:

Open Source does not mean Free: Why we are declaring a license for the community database

...

Very shortly you will notice an important change to our GPLv3 Resource site [at http://gpl3.palamida.com/]. This week's events have led to the decision to add a Creative Commons License (CCL) to the site to ensure that recent blatant plagiarism of our database contents by a newly launched GPLv3 site will be duly credited and/or cease. After two days of intense investigation, we have confirmed that most of our database has been copied directly – word for word and misspelling for misspelling, with very few original additions to our initial work. We feel that that this secondary site does a disservice to the open source community that has for many months diligently contributed data to our database, assisted in correcting discrepancies, and supported the accurate and timely tracking of GPLv2 and v3 conversations and conversions. It has always been the aim of Palamida to run our Resource Site like an open source project – encouraging collaboration, edits, transparency and commentary – so we understand that our data has always been free for re-distribution. However, we did not anticipate the entirety of our database being re-copied and re-packaged as original information without appropriately referencing Palamida as the source. We are disappointed to have to add any sort of copyright but have chosen an open source license in hopes of continuing the spirit of the resource.

Well, I hate to break it to you chaps, but if your original licence allowed the database to be copied (and I don't know if that's the case, but let's assume it is) it's a bit unfair to complain when someone, er, copies it. If you want credit - which is a perfectly reasonable thing to want - make sure the licence reflects that. If you don't want people to copy it, fine, but then it ain't "like an open source project".

Basically, sharing means sharing - and open source *does* mean free (subject to complying with the licence.) (Via C|net.)

Open-Mouthed...

...I am, if this "sea-change" turns out to be true (a sceptic of the UK Government writes):

The way the government makes its vast amounts of data available to the public could be about to change.

It has decided to make access to a database of UK laws completely free for the public to access and re-use.

It marks a "sea-change" in the way government information becomes available to the public, a senior civil servant has told the BBC News website.

Please, please, please, please, please.

ID'ing Reality

The truth begins to sink in:

The government has abandoned plans for a giant new computer system to run the national identity cards scheme.

Instead of a single multi-billion pound system, information will be held on three existing, separate databases.

Well, that's a start. Just as hopeful is the statement:

Home Secretary John Reid denied this was a "U-turn" saying it would save cash, boost efficiency and cut fraud.

So, presumably cancelling the whole thing would also not be a "U-turn", since it too "would save cash, boost efficiency and cut fraud"....

Open Access to the Origins of Language

New Scientist reports:

Linguists are calling for an online public database, similar to the human genome project, that would allow researchers to collaboratively share different studies of language impairment.

By gathering together studies of developmental disorders that cause communication impairments – such as autism or Down’s syndrome – they hope to provide new clues about the origins of language.

Aside from the interesting nature of the project, what is striking is that the key element is not creating new knowledge, but consolidating it in a database, allowing higher-level knowledge to emerge. Clearly, for this to work in an optimal way, all the data and papers need to be open access. Whether it will be, assuming the project goes ahead, remains to be seen.

Update: Wow, the original article behind the NS story is not behind the usual paywall. So from this I can read:

We close by illustrating how systematic analyses within and between disorders, suitably informed by evolutionary theory—and ideally facilitated by the creation of an open-access database—could provide new insights into language evolution.

If Laws Were Secret

This sounds like something straight out of Brazil. The UK Government is rolling out a database of UK laws, and it looks like the people who have already paid for it - the UK public - will have to pay again to access it.

First they make the laws pay-per-view, then they make them secret....

Pervasive but not Persuasive

Further proof, if any were needed, that open source business operates under different rules. Here's a letter from CEO of Pervasive, a company set up to "with the goal of helping accelerate the transition from the traditional, high cost database licensing model to the open-source, high-value model", specifically to PostgreSQL. But something funny happened along the way:

While we always knew that PostgreSQL is a solid product with advanced database capabilities and that it has a very real opportunity to shake up the high-end database market, we underestimated the high level of quality support and expertise already available within the PostgreSQL community. In this environment, we found that the opportunity for Pervasive Software to meaningfully increase adoption of PostgreSQL by providing an alternative source for support and services was quite limited.

In other words, the PostgreSQL community can look after itself, thank you very much. Just as can all other flourishing free software projects. Which is why, ultimately, they will prevail, because there are immune to the fads and fashions of the business world. (Via Matthew Aslett.)

A Study in Official Openness

It is probably hard for those outside the UK to appreciate the extent of the secrecy that has pervaded public life here for centuries. The clearest manifestation of this is the pernicious Official Secrets Act, which makes pretty much anything a secret if the Government says it is.

Against this presumption that the public has no right to know anything, the passage of the Freedom of Information Act in 2000 was a major milestone, and credit must be given to the current Government for finally making it a reality. This is especially the case since it is clear that the information released by Act is proving a major embarrassment at times, thanks to both an increasingly demanding public and a commendably independent commissioner, Richard Thomas.

As the foreword to his first Annual Report makes clear, he is acutely aware of the central position that his department occupies in today's world, where there is an inevitable tension between his two main tasks: promoting openness and protecting privacy:

Never before has the threat of intrusion to people’s privacy been such a risk. It is no wonder that the public now ranks protecting personal information as the third most important social concern. As technology develops in a globalised 24/7 culture, power increases to build comprehensive insights into daily lives. As internet shopping, smart card technology and joined-up e-government initiatives reduce costs, respond to customers’ demands and improve public services, more and more information is accumulated about us. According to one estimate, information about the average working adult is stored on some 700 databases. New information is added every day. Much of this will be confidential material which we do not want others to see or use unless we say so. There are obvious risks that information is matched with the wrong person or security is breached. The risks increase substantially as information is shared from one database to another, or access granted to another group of users. Real damage can arise when things go wrong – careers and personal relationships can be jeopardised by inaccurate information. Identity theft can involve substantial financial loss and loss of personal autonomy.

The vast majority of information that is held on adults, and increasingly on children, serves a useful purpose and is well intentioned. But everyone recognises that there must be limits. Data protection provides the framework. It raises questions about where lines should be drawn. What is acceptable and what is unacceptable? What safeguards are needed? What is the right balance between public protection and private life? How long, for example, should phone and internet traffic records be retained for access by police and intelligence services fighting terrorism? Whose DNA should be held, and for how long, to help solve crime? What safeguards are needed for commercial internet-based tracking services which leave no hiding place?

All power to Mr Thomas' elbow.

O Happy, Happy Digital Code

My book Digital Code of Life was partly about the battle to keep genomic and other bioinformatics information open. So it's good to see the very first public genomic database, now EMBL, spreading its wings and mutating into FELICS (Free European Life-science Information and Computational Services) with even more bioinformatics goodies freely available (thanks to a little help from the Swiss Institute for Bioinformatics, the University of Cologne, Germany, and the European Patent Office).

The Science of Open Source

The OpenScience Project is interesting. As its About page explains:

The OpenScience project is dedicated to writing and releasing free and Open Source scientific software. We are a group of scientists, mathematicians and engineers who want to encourage a collaborative environment in which science can be pursued by anyone who is inspired to discover something new about the natural world.

But beyond this canonical openness to all, there is another, very important reason why scientific software should be open source. With proprietary software, you simply have to take on trust that the output has been derived correctly from the inputs. But this black-box approach is really anathema to science, which is about examining and checking every assumption along the way from input to output. In some sense, proprietary scientific software is an oxymoron.

The project supports open source scientific software in two ways. It has a useful list of such programs, broken down by category (and it's striking how bioinformatics towers over them all); in addition, those behind the site also write applications themselves.

What caught my eye in particular was a posting asking an important question: "How can people make money from open source scientific software?" There have been two more postings so far, exploring various ways in which free applications can be used as the basis of a commercial offering: Sell Hardware and Sell Services. I don't know what the last one will say - it's looking at dual licensing as a way to resolve the dilemma - but the other two have not been able to offer much hope, and overall, I'm not optimistic.

The problem goes to the root of why open source works: it requires lots of users doing roughly the same thing, so that a single piece of free code can satisfy their needs and feed off their comments to get better (if you want the full half-hour argument, read Rebel Code).

That's why the most successful open source projects deliver core computing infrastructure: operating system, Web server, email server, DNS server, databases etc. The same is true on the client-side: the big winners have been Firefox, OpenOffice.org, The GIMP, Audacity etc. - each serving a very big end-user group. Niche projects do exist, but they don't have the vigour of the larger ones, and they certainly can't create an ecosystem big enough to allow companies to make money (as they do with GNU/Linux, Apache, Sendmail, MySQL etc.)

Against this background, I just can't see much hope for commercial scientific open source software. But I think there is an alternative. Because this open software is inherently better for science - thanks to its transparency - it could be argued that funding bodies should make it as much of a priority as more traditional areas.

The big benefit of this approach is that it is cumulative: once the software has been funded to a certain level by one body, there is no reason why another should't pick up the baton and pay for further development. This would allow costs to be shared, along with the code.

Of course, this approach would take a major change of mindset in certain quarters; but since open source and the other opens are already doing that elsewhere, there's no reason why they shouldn't achieve it in this domain too.