Can clouds transform science?

A diagram showing different cloud modes with hybridhigh-performance computing and gridmodels forsupporting real-world science and engineering applications.From left to right:HPC in the Cloud;HPC plus Cloud; andHPC as a Cloud.Image courtesy Manish Parashar.

Wolfgang Gentzsch, an independent high-performance computing consultant and chairman of the ISC Cloud Conference series, interviewed Manish Parashar, a speaker for the upcoming ISC Cloud’12 event, which will be held on 24-25 September in Manheim, Germany. Manish Parashar is professor of electrical and computer engineering at Rutgers University, and founding director of the Rutgers Discovery Informatics Institute (RDI2), and of the National Science Foundation Cloud and Autonomic Computing Center (CAC).

Manish, computational and data-enabled science and engineering (CDS&E) has traditionally used dedicated platforms and specialized high-end systems – why should clouds be considered for them?

Clouds are already having a significant impact by enabling enterprises to outsource IT on demand. Clouds enable the outsourcing of many of the tedious aspects of research, such as deploying, configuring, and managing infrastructure, and thus enable scientists to focus on science.

Cloud services improve productivity, facilitate the sharing of research results, and enable the reproducibility of associated computations. This has democratized access to computational and data resources for researchers who don’t have adequate local infrastructure.

What role do you see for clouds in computational and data-enabled science and engineering (CDS&E)?

While clouds can provide many of the same advantages to CDS&E that they provide enterprise IT, it is critical to also look beyond these benefits, and understand application formulations and usage modes that are meaningful in a hybrid infrastructure  with grids, high-performance computing, and clouds.

Cloud computing can support CDS&E applications by providing infrastructure; when, for example, local infrastructure is not available or is insufficient. It can also supplement existing infrastructure to provide additional capacity or complementary capabilities to meet heterogeneous or dynamic needs. Clouds can serve as accelerators, or provide resilience to scientific workflows by moving execution to alternative resources when a failure occurs. Cloud abstraction can alleviate some of the challenges that scientific applications face in current high-performance computing environments.

Which aspects of computational and data-enabled science and engineering (CDS&E) and clouds does your research address?

Our research has been exploring cloud concepts and supporting infrastructure for CDS&E. Our experiments include a broad set of federated resources spanning existing grids, high-performance computers, clusters, public clouds, and applications. We have been using the CometCloud framework, developed at Rutgers, to explore application formulations and hybrid infrastructure usage modes that are meaningful for CDS&E application workflows.

This includes three usage modes: ‘HPC in the cloud’, where researchers outsource entire applications to current public and - or - private cloud platforms; ‘HPC plus cloud’, which focuses on exploring scenarios where clouds can complement grids and high-performance computers to support science and engineering application workflows; and ‘HPC as a cloud’, that’s focused on parallel resources using elastic on-demand cloud abstractions.

Can you tell me a little bit about CometCloud?

An image of Manish Parashar next to Excalibur, theRutgers high-performance computer. Image courtesy isc-events.

CometCloud is an autonomic computing engine that enables a dynamic and on-demand federation of clouds and grids, as well as deployment and execution of applications on federated environments. It supports diverse parallel infrastructures, clouds and grids, enabling on-demand integration of public and private clouds. And it provides autonomic cloudbursts, i.e. a solution for extreme requirements and spikes in demand. The CometCloud programming layer provides platforms for application development and management. It supports MapReduce, Workflow, and Master-Worker, or Bag-of-Tasks.

What do you see as the key research challenges for computational and data-enabled science and engineering (CDS&E)  and clouds?

First, algorithms and application formulations can benefit from elasticity and hybrid cloud, grid, and high-performance computing usage modes. Second, programming abstractions and systems can enable CDS&E applications to simply and effectively take advantage of the elastic access to resources and services. Third, middleware stacks and management policies will be able to support the new CDS&E application formulations and services.

Note, that these are in addition to core cloud challenges such as programming models, security, standardization, etc. These are discussed in more details in our recent white paper on Cloud Paradigms and Practices for CDS&E.

Do you have any concluding remarks for readers?

Aggressive cloud computing technology development has resulted in many classes of cloud services that provide attractive solutions for a range of business applications. It is also clear that there are real benefits in using clouds as part of a hybrid cyber infrastructure for supporting CDS&E, e.g. simplifying the deployment of applications and the management of their execution, improving efficiency, productivity, and providing more attractive cost to performance ratios. Furthermore, clouds can support new classes of algorithms and enable new application formulations, which can potentially revolutionize CDS&E research and education.

More details about Manish’s research can be seen during his presentation at 10:30 am on 25 September during the ISC Cloud conference, in Mannheim, Germany. You can see the event program hereor register to the conference through this link.