Why Scientific Software Wants To Be Free

Not sure if I missed this earlier, but it strikes me as a hugely important issue that deserves a wider audience whether or not it is brand new:

Astronomical software is now a fact of daily life for all hands-on members of our community. Purpose-built software for data reduction and modeling tasks becomes ever more critical as we handle larger amounts of data and simulations. However, the writing of astronomical software is unglamorous, the rewards are not always clear, and there are structural disincentives to releasing software publicly and to embedding it in the scientific literature, which can lead to significant duplication of effort and an incomplete scientific record.

We identify some of these structural disincentives and suggest a variety of approaches to address them, with the goals of raising the quality of astronomical software, improving the lot of scientist-authors, and providing benefits to the entire community, analogous to the benefits provided by open access to large survey and simulation datasets. Our aim is to open a conversation on how to move forward.

We advocate that: (1) the astronomical community consider software as an integral and fundable part of facility construction and science programs; (2) that software release be considered as integral to the open and reproducible scientific process as are publication and data release; (3) that we adopt technologies and repositories for releasing and collaboration on software that have worked for open-source software; (4) that we seek structural incentives to make the release of software and related publications easier for scientist-authors; (5) that we consider new ways of funding the development of grass-roots software; (6) and that we rethink our values to acknowledge that astronomical software development is not just a technical endeavor, but a fundamental part of our scientific practice.

Leaving aside the obvious and welcome element of calling for an open source approach (and, presumably, open source release if possible), there is deeper issue here: the fact that astronomy - and by extension, all science - is increasingly bound up with software, and that software is no longer an incidental factor in its practice.

A consequence of this is that as software moves ever-closer to the heart of the scientific process, so the need to release that code under free software licences increases. First, so that others can examine it for flaws and/or reproduce the results it produces. And secondly, so that other scientists can build on that code, just as they build on its results. In other words, it is becoming evident that open source is indispensable for *all* science, and not just the kind that proudly preclaims itself open.

Crowdsourcing an Astronomy Commons

This is a fab use of pooled images combined with automation:

Flickr hosts a wide range of beautiful images, but a new project built on top of Flickr's API only focuses on photos of the night sky from amateur astronomers. The Astrometry.net project constantly scans the Astrometry Flickr group for new images to catalog and to add to its open-source sky survey. At the same time, this project also provides a more direct service to the amateur astronomers, as it also analyzes each image and returns a high-quality description of the photo's contents.

The Astrometry group currently has over 400 members, and as Christoper Stumm, a member of the Astrometry.net team, told the Flickr Code blog, the back-end software uses geometric hashing to exactly pinpoint and describe the objects in the images. When you submit an image to the Flickr pool, the robot will not just respond with a comment that contains an exact description of what you see in the image, but it will also annotate the image automatically.

What I'd like to see is something similar for terrestrial images, to build up a huge mosaic of everything, everywhere.

Open Access Astro-Observatory Runs GNU/Linux

Montegancedo Observatory is the first free open access astronomical observatory in the world. It is located in Building 6 of the School of Computing. The dome is equipped with a computer-automated, robotized 10” telescope, and several computers operating as a web applications server. The observatory also links and broadcasts images and videos captured by the webcams arranged around the dome... All servers run on GNU/Linux systems.

Providing open access to facilities, mediated by the Internet; providing (presumably) open access to the results in real-time; using free software to run the whole thing: this is the future of science. (Via Open Access News.)

Google Reaches for the Stars

One of the most important shifts in science at the moment is towards dealing with the digital deluge. Whether in the field of genomics, particle physics or astronomy, science is starting to produce data in not just gigabytes, or even terabytes, but petabytes, exabytes and beyond (zettabytes, yottabytes, etc.).

Take the Large Synoptic Survey Telescope, for starters:

The Large Synoptic Survey Telescope (LSST) is a proposed ground-based 8.4-meter, 10 square-degree-field telescope that will provide digital imaging of faint astronomical objects across the entire sky, night after night. In a relentless campaign of 15 second exposures, LSST will cover the available sky every three nights, opening a movie-like window on objects that change or move on rapid timescales: exploding supernovae, potentially hazardous near-Earth asteroids, and distant Kuiper Belt Objects. The superb images from the LSST will also be used to trace billions of remote galaxies and measure the distortions in their shapes produced by lumps of Dark Matter, providing multiple tests of the mysterious Dark Energy.

How much data?

Over 30 thousand gigabytes (30TB) of images will be generated every night during the decade-long LSST sky survey.

Or for those of you without calculators, that's 10x365x30x1,000,000,000,000 bytes, roughly 100 petabytes. And where there's data, there's also information; and where there's information...there's Google:

Google has joined a group of nineteen universities and national labs that are building the Large Synoptic Survey Telescope (LSST).

...

"Partnering with Google will significantly enhance our ability to convert LSST data to knowledge," said University of California, Davis, Professor and LSST Director J. Anthony Tyson. "LSST will change the way we observe the universe by mapping the visible sky deeply, rapidly, and continuously. It will open entirely new windows on our universe, yielding discoveries in a variety of areas of astronomy and fundamental physics. Innovations in data management will play a central role."

(Via C|net.)