Smooth upgrade to XSEDE's network

The Extreme Science and Engineering Discovery Environment, XSEDE, has transitioned its network backbone infrastructure to use the FrameNet service of National LambdaRail. This change, a switch from older technology used by TeraGrid, provides a more flexible infrastructure and saves substantial costs for connecting XSEDE service providers.

"We've eliminated the need for the TeraGrid-owned capital equipment, making a more direct interconnect between XSEDE sites. This will save hundreds of thousands of dollars in equipment cost, as well as tens of thousands of dollars in annual maintenance," said Linda Winkler, a researcher at Argonne National Laboratory who leads the XSEDE networking group.

The NLR FrameNet service is part of the nationwide optical-fiber infrastructure of NLR, a consortium of more than 280 U.S. universities and private and government laboratories that provides high-performance network services for research and education. XSEDE replaced SONET/OC192 circuits with 10-GigE (10 gigabits per second Ethernet) connectivity to FrameNet for all core XSEDE service provider sites; XSEDE has a dedicated 10-GigE circuit from NLR between backbone nodes in Chicago and Denver.

As with the predecessor TeraGrid service, all XSEDE participants have full site-to-site connectivity —including broadcast capability — between all the service provider sites.

The XSEDE networking group, led by Winkler, spent the past few months switching the backbone infrastructure of the TeraGrid to the newer, more cost-effective technology. Each of the core XSEDE sites did the switch at a separate time to minimize impact to users.

"As far as I know, nobody noticed that we did the change, and in my estimation that's a job well done," Winkler said.

The changed infrastructure provides flexibility to easily extend the XSEDE network as additional sites are included.

"FrameNet has the advantages of a private network along with a rich set of connectivity options," Winkler added. "The footprint of FrameNet nodes across the country makes it feasible to extend the XSEDE network backbone to other cities and to add capacity to existing sites."

In coming months, XSEDE networking staff will implement performance monitoring services that will make it easier for researchers to obtain optimum network performance. In addition, networking staff will research alternative connection strategies, including the ability to take advantage of dynamic circuits on demand.

The eight core XSEDE service-provider sites directly affected by the network transition are: Indiana University, National Center for Supercomputing Applications, National Institute for Computational Sciences, National Center for Atmospheric Research, Pittsburgh Supercomputing Center, Purdue University, San Diego Supercomputer Center, and Texas Advanced Computing Center.

A version of this article first appeared on the XSEDE website.