Opinion - What clouds and grids can learn from each other
During the past 10 years, hundreds of grid projects have come and gone, passing away after funding ran dry. Most didn’t have a realistic strategy for sustainability, let alone a viable business model for their infrastructures, tools, applications or services. Often, the only asset left after the project’s end was the hands-on expertise gained by those involved, which is certainly valuable in the long run, but doesn’t justify the effort and funding. So far, in my opinion, grids didn’t keep up to their full promise.
What went wrong?
Sure, grids, by their very nature, are complex to design, build and maintain; and applications are cumbersome to run. It might take another 10 years of trial and error (and re-writing grid middleware?) to navigate the labyrinth of new technologies and paradigms, such as Utility Computing, Autonomic Computing, ASP, SOA, SOI, SaaS, PaaS, HaaS, Outsourcing, Hosting, Virtualization, Web 2.0 and Mashups . . .
And obviously, grids followed a natural curve: technology trigger, inflated expectations, trough of disillusionment, slope of enlightenment, and–finally–plateau of productivity. Some technologies disappear completely. For me, however, we are now facing a potential renaissance, triggered by the appearance of . . . clouds.
Cloud computing, as offered by companies like Sun, IBM, Amazon, Google, CloudCamp, and others, will soon become an important component of R&D, adding a new, ‘external’ dimension of flexibility by enhancing one’s ‘home’ resource capacity when needed Existing businesses will use them during peak demand, service providers will host their applications on them and provide Software as a Service (SaaS), and start-ups will integrate them without needing to buy resources upfront.
While grids provide the ‘plumbing’ to enable access to distributed resources, clouds denote service on a pay-per-use basis. Grids stand out because of their flexible, dynamic, feature-rich resources; they are thus innately complex. This complexity, however, must be hidden from the end-user (e.g. in the form of a cloud) if we want acceptance of this infrastructure.
Cloud applications will likely follow similar stages as grid-enabling ones. Just as challenging, though, are the cultural, mental, legal, and political aspects. It is difficult to imagine users easily entrusting their corporate assets and sensitive data to cloud service providers. Today, the status of clouds seems to be similar to that of grids in the early 2000s: a few simple and well-suited application run on clouds, but more complex and demanding applications will face many barriers to overcome.
Sure, clouds are easier to deploy than grids: Clouds are user-friendly, service oriented, on-demand. But if clouds aim to ‘replace’ grids, they face similar challenges. Even a single grid service running in a cloud image (or using a service in the cloud) will quickly face the roadblocks we know from grids today.
The role of DEISA
One promising solution might be the Distributed European Infrastructure for Supercomputing Applications, or DEISA, which is moving from a grid to a cloud. DEISA Extreme Computing Initiative, or DECI, is successfully offering millions of supercomputing hours to the European e-Science community and helping scientists gain new scientific insights.
Why is DECI successful? Several reasons, in my opinion: it has a targeted focus on specific, long-running, supercomputing applications; many applications run on one single system; it has user-friendly access to resources through DESHL and UNICORE; it gives consortium partners full autonomy; and there is an application task force (ATASKF) that helps users plug in to the supercomputing infrastructure. If all this were to stay, DEISA would have some real staying power. And then, we might have a DEISA Cloud which will become an external HPC node within your grid application workflow.
* We should lower our expectations in the first place.
The good news is: clouds will help grids mature, and grids will help clouds avoid some teething troubles. Clouds will teach grids that to be widely accepted and sustainable, grids must be simple, user-friendly, service oriented, scalable, on-demand, with simple applications. And grids teach clouds that by raising the bar of expectations too high, they could easily plunge into the trough of disillusionment.
With this sea-change ahead of us, there will be a continuing need for support of the work of the Open Grid Forum (OGF). Certainly, existing processes can be agreed upon and developed that are streamlined, shortened, and more efficient outside it. But only an agreed-upon set of standards will enable e-infrastructures and grid applications to be easily built from different components, and move towards an agile, federated service platform. Only standards, such as those developed in OGF, will guarantee interoperability of technology components best suited for your applications–and thus reduce dependency on proprietary building blocks, keep costs under control, and increase research and business flexibility.
Wolfgang Gentzsch, DEISA Project, Duke University, and Board of Directors, Open Grid Forum