An artist’s impression of LiveWN as a pre-packaged grid computing product, designed by member of the LiveWN team, John Kouvakis. Image courtesy of LiveWN

What if grid nodes were as simple to run as music on a compact disk player?

What if a new cluster involved zero installation and only minimal—if any—configuration?

What if setting up that new cluster took only a few minutes?

If all this were true, we would be able to readily exploit the idle time of publicly funded computers, such as those in libraries, universities and institutes. We could share resources among arbitrary users, Internet Service Provider members or Open Source Software projects. We could rapidly multiply computing power for urgent tasks. We could quickly and easily train novice users.

So, if you think it is impossible to run gLite worker nodes within virtual machines, or over dial-up lines with low-speed and ad-hoc connections, or behind firewalls and private networks, think again. All of these features are now feasible with a single solution.

Grid on a stick

LiveWN was prototyped in 2006 as a grid-bootable CD supporting diskless, easy-to-deploy worker nodes and requiring virtually zero administration upon deployment.

The system mixed key technologies including LiveCD (a self-booting Linux CD), gLite (the grid middleware stack), OpenVPN (virtual private network) and OpenAFS (open distributed filesystem), proving that the initial concept was correct and workable and efficient enough to dynamically provision for worker nodes and user interfaces, without having to resort to a hard-disk installation. This “grid distribution” was simple enough to run even from a USB stick.

The gLiteDVD includes software such as rendering tool PovRay, for on-demand grid-enabled video rendering farms. Image courtesy of PovRay

During testing, it became clear that there was a need for two solutions: a CD format called LiveWN; and a DVD format called gLiteDVD.

LiveWN has a tiny footprint, ensuring maximum compatibility with as many systems as possible, including low-RAM, virtual machines and netboot. It implements a combined worker node/user interface.

gLiteDVD has a much larger footprint and it is a superset of LiveWN. On top of LiveWN, it includes a full desktop environment and some extra scientific software, including tools from high energy physics project ATLAS.

gLiteDVD also includes WINE (an open source implementation of the Windows API), video and rendering tools such as PovRay (for on-demand grid-enabled video rendering farms), transcoding and ray-tracing applications, Octave, VLC, Python and other software.

For some users gLiteDVD’s added capacities make it the best choice; others will find the simplicity and compatibility of LiveWN suits them better.

What comes next?

We are now in the process of setting up a network of Points of Presence, or POPs, where a POP in practice corresponds to a set of computing elements, virtual private networks and Andrew File System servers. This is necessary, as scavenged resources (worker nodes) have to be dynamically assigned to clusters (computing elements). We still need to run some larger scale experiments in order to tune scalability and robustness aspects of our implementation, as well as investigate potential “worker node roaming” capabilities.

Further developments this year will involve Internet Protocol Version 6, as well as configuration management (based on a Content Delivery Network, Service Location Protocol, etc),  incorporation of non-standard grid users (exploiting GridShib and OpenID),  accounting and credit control (using DGAS and similar systems), benchmarking and optimal resource allocation (using LMbench), checkpointing and overall reliability improvement (easier said than done), automatic load-balancing, and fail-over in case of faults (unknown territory).

What makes this technology really stand out from other scavenging techniques is that it is a true grid solution: it can dynamically collect resources that belong and live within multiple administrative domains, without breaking the multi-federation grid computing model. This is its primary advantage when compared to other solutions.

The LiveWN and gLiteDVD technology is wishware—freeware “restricted” by wishes—developed by the National Technical University of Athens, Greece, with NTUA student Giannis Kouretis contributing much of the R&D work. The technology is already being tested by the Greek National Research and Education Network GRNET, and is being considered by other universities.

Fotis Georgatos, Research associate, National Technical University of Athens and GRNET grid trainer

A new release of the gLiteDVD disk was recently released and will be demonstrated during EGEE '07 in Budapest next week.