Feature - A tool for reaching the cloud

There are a many different clound providers to choose from. Image courtesy UCAR

Utility computing is a way for potential clients — whether they are a business or an academic researcher — to match the cost of their computing and storage needs to their demand for these resources. Suppose you need the resources of 1,000 computers, but for only one day per week. Using on-demand utility computing such as clouds allows you to purchase the resources just for the one day, which can result in substantial cost-savings.

Faced with the challenge of developing dynamic, service-focused infrastructures for the media and financial sectors — where data is often subject to rapid peaks and troughs in demand (consider how quickly breaking news develops and how traders on Wall Street react to it) — the Belfast e-Science Center BeSC found that to best exploit the benefits of cloud technology daily, the user needs the option of using different resource providers to accommodate the data.

However, as cloud computing has become more popular, there has been a dramatic increase in the number of resource providers, all of which have a different resource usage model with a different application programming interface (API). This presents a problem for clients who want to run applications using multiple resource providers, because applications have to be tailored to each individual provider and their API.

Image copyright Belfast e-Science Center

Simplifying the process

In an attempt to simplify the allocation, management and discarding of on-demand resources from a multitude of providers, BeSC, as part of the UK’s National Grid Service (NGS), has developed an abstraction layer that supports a provider-independent API.

BeSC have their own service hosting cloud (SHC), which offers an on-demand service and application hosting environment to both academic and commercial users with high-performance computing needs. In an attempt to encourage and facilitate the adoption of cloud technology, academic users can take advantage of applications and services hosted on the SHC for free.

In contrast to the norm, the API which BeSC has developed is designed to work on more than just their own SHC, but also to be compatible with common utility providers such as the Amazon and Flexiscale.

The PRISM media service cloud, which supports access to all the content on the BBC from different media devices, employs this system. It relies on a core infrastructure of partner clouds and, if possible, data is routed to the core datacenters at BeSC, the BBC and British Telecom. But when demand is high, or part of the core infrastructure fails, the system automatically fails over to another utility resource provider, such as the Amazon EC2.

The Belfast e-Science Center is currently investigating extending their API framework in order to make it compatible with more utility providers.

—Terry Harmer and Peter Wright, BeSC. Adapted by Seth Bell, iSGTW. A version of this article previously appeared in the NGS newsletter.