Breakthrough simulation: A supercell producing a long-track EF5 tornado

Long-track EF5 tornadoes, like the one that touched down in 2011 in Joplin, Missouri, US, are the longest-lived and most destructive of their kind. Most have wind speeds that reach more than 200 miles (322 kilometers) per hour and create major devastation. These tornadoes are also the most rare, which makes studying them all the more difficult.

Using Blue Waters at the National Center for Supercomputing Applications (NCSA) at the University of Illinois, US, scientists have created the first simulation of a long-track EF5 tornado — including the supercell thunderstorm that spawned the tornado. “There isn't a lot of research published on this type of work in numerical simulation," says Leigh Orf, associate professor of atmospheric scienceat Central Michigan University, US.

“We’re trying to capture several events at once. We’re simulating the entire thunderstorm and want the tornado to form naturally, live on the ground for a long time, and then decay just like a real tornado would in a supercell," explains Orf. "This requires tremendous computational resources. The Blue Waters environment is crucial to the development, execution, data management, visualization, and post-processing work we are doing.”

Tornadoes can occur whenever and wherever conditions are right. According to the US National Oceanic and Atmospheric Administration, the current average lead-time for tornado warnings is 13 minutes. “Our overarching goal,” says Orf, “is to be better able to predict tornadoes, so that people do not get caught in the path of these storms.”

For the simulation, Orf used the CM1 cloud model developed by George Bryan at the US National Center for Atmospheric Research — specifically with massively parallel architectures in mind. “We’re using a hybrid MPI/OpenMP model, and it scales very well; we have run it with up to 200,000 cores. We are hoping to reproduce the simulation at even higher resolution,” says Orf.

Because the air in and around tornadoes moves so fast, frequently saving data is required to study what’s happening. This results in a lot of Input/Output (I/O) — one of Orf’s biggest challenges. “CM1 did not come out of the box with an I/O option that worked well on Blue Waters, so I spent a lot of time learning about hierarchical data format (HDF) and different ways to organize data. We now have things situated so that I/O does not choke us when we’re saving data every two model-seconds.”

Visualization is also a challenge. To study three-dimensional atmospheric phenomena, you have to be able to look at and observe it. “When you start producing hundreds of terabytes of data, you can be overwhelmed quickly,” notes Orf. “There are a lot of grid points; with help from Rob Sisneros at NCSA, I wrote a plug-in to interface with the data output, and it actually works pretty well.”

The EF5 storm that Orf and his colleagues are simulating has a 65-mile (104-kilometer) path. The model was initialized with the environment that surrounded the Calumet-El Reno-Piedmont-Guthrie Tornado of May 24, 2011, a long-track EF5, and a cloud was initialized within this environment and grew, following the laws of physics. “The simulated tornado is actually on the ground for almost the same length of time as the observed tornado,” says Orf. “In fact, it is still on the ground when I stop the model, but it is weakening.”

This simulation followed several previous simulations where different parameters were explored, including model resolution, surface treatments, how cloud and precipitation is handled, as well as the way the cloud was initialized. “We tried a lot of different things before we got the long-track EF5,” Orf explains. “However, this is a friction-free run; we turned off friction and it led to a good result. We are currently exploring simulations where we allow for the inclusion of surface friction, which is more physically realistic.” The group is using the volume rendering capabilities of the VisIt tool on Blue Waters. “You can sit at your workstation at home, connect to Blue Waters, and it will spawn jobs into a queue and stream data back to the screen.”

Orf hopes to increase overall grid resolution, especially near the ground, and eventually do friction correctly. “Even with the grid that we have now, I've managed to turn friction on and have the storm move over a region of gently increasing surface roughness, and we still get an EF5 tornado. It even looks a little more physically realistic than the one we have now, but it’s still a work in progress.”