The IT Department in Cloud Cuckoo Land

On Open Enterprise blog.

Textbook Enterprise Open Source

There's no more powerful argument in favour of using GNU/Linux in an enterprise context than big names that are already doing so. Google and Amazon are the obvious ones, but we can now add PayPal to the list:

PayPal is currently processing $1,571 worth of transactions per second in 17 different currencies on about 4,000 servers running Red Hat Linux.

The article also gives some very concrete advantages of running a GNU/Linux-based grid in this way:

As PayPal grows it's much easier to grow the grid with Intel (NSDQ: INTC)-based servers than it would be to upgrade a mainframe, he said. In a mainframe environment, the cost to increase capacity a planned 15% or 20% "is enormous. It could be in the tens of millions to do a step increase. In [PayPal's] world, we add hundreds of servers in the course of a couple of nights and the cost is in the thousands, not millions," he said.

PayPal takes Red Hat Enterprise Linux and strips out all features unnecessary to its business, then adds proprietary extensions around security. Another virtue of the grid is that PayPal's 800 engineers can all get a copy of that customized system on their development desktops, run tests on their raw software as they work, and develop to PayPal's needs faster because they're working in the target environment. That's harder to do when the core of the data center consists of large Unix symmetrical multiprocessing boxes or mainframes. In neither case is it cheap to install duplicates for developers, he said.

Getting Organised for Grids

I've always liked the idea of grids - creating virtual supercomputers by hooking up often geographically distant systems. It goes almost without saying that free software rules the grid roost, notably in the form of the Globus Toolkit. Now there's a new meeting place for open source gridders, with the easy-to-remember URL grid.org:

Grid.org is intended to provide a single location where open-source cluster and grid information can be aggregated and where community members can exchange information, experiences, and ideas related to the complete open source cluster software stack. In particular, but not exclusively, Grid.org provides a community where users of Cluster Express, and the various open source components it comprises, can interact with each other and with the source code.

This has only just been (re-)launched, so content and community are still slightly thin on the ground, although it is already good on Cluster Express:

Cluster Express 3.0 is open source cluster management software that integrates best-of-breed open source technologies to provide everything one needs to run technical and advanced computing applications on a cluster.

The Other Grid God: Open Source

As I was browsing through Lxer.com, my eye caught this rather wonderful headline: "Grid god to head up Chicago computing institute". The story explains that Ian Foster, one of the pioneers in the area of grid computing (and the grid god in question), is moving to the Computation Institute (great name - horrible Web site).

Grid computing refers to the seamless linking together across the Internet of physically separate computers to form a huge, virtual computer. It's an idea that I've been following for some time, not least because it's yet another area where free software trounces proprietary solutions.

The most popular toolkit for building grids comes from the Globus Alliance, and this is by far the best place to turn to find out about the subject. For example, there's a particularly good introduction to grid computing's background and the latest developments.

The section dealing with grid architecture notes that there is currently a convergence between grid computing and the whole idea of Web services. This is only logical, since one of the benefits of having a grid is that you can access Web services across it in a completely transparent way to create powerful virtual applications running on massive virtual hardware.

The Globus Alliance site is packed with other resources, including a FAQ, a huge list of research papers on grids and related topics, information about the Globus Toolkit, which lets you create grids, and the software itself.

Open source's leading position in the grid computing world complements a similar success in the related field of supercomputing. As this chart shows, over 50% of the top 500 supercomputers in the world run GNU/Linux; significantly, Microsoft Windows does not even appear on the chart.

This total domination of top-end computing - be it grids or supercomputers - by open source is one of the facts that Microsoft somehow omits to tell us in its "Get The Facts" campaign.