Accelerate physics with your own computer

A single dipole magnet in the LHC tunnel. Image courtesy CERN.

The interest around the Large Hadron Collider (LHC) at CERN is largely focused on what physicists will discover just after the beams of high-speed protons smash together. But few people spare a thought for those physicists who coordinate and manage such a complex maneuver in the first place, who force the protons to conform, pack them into a neat bunch, and send them on their way through a 27 kilometer-long circular tunnel without losing a single one along the way.

Protons are just tiny little particles – much much smaller even than the air molecules that we breathe. So, it’s easy to underestimate how much power they have when they’re in the LHC and travelling close to the speed of light. In fact, if the protons went off course they could cause some serious damage, it would be similar to “an aircraft carrier smashing into the wall,” says Lenny Rivkin, a professor of accelerator physics at École Polytechnique Fédérale de Lausanne (EPFL).

The superconducting magnets would be seriously damaged and the machine would be inoperable. Billions would be lost, and so would our chances of testing the theories of physics developed over the last few decades.

It’s no easy task, getting the protons to conform. One problem with these beams of protons is that their behavior can become chaotic. As with the ‘butterfly effect’ – often stated as ‘the flap of a butterfly’s wings in Brazil can set off a tornado in Texas’ – this means that even the slightest change in the conditions around the bunch of protons could result in a drastically different outcome.

“Chaotic motion cannot be easily predicted by theory. Since at least the 1960s, physicists have been trying to predict it,” says Frank Schmidt, a senior scientist in the accelerator division at CERN. “This is quite hard to get your head around. You have a formula for the motion of a proton after one revolution in the LHC, but the formula gives an unpredictable result when applied for several hundred thousand revolutions.”

There is one aspect of the protons’ behavior that seems a very lucky fact: there is a relatively long time between a proton becoming unstable – or “jittery” as Schmidt describes it – and it completely losing its course and flying off and hitting the wall. And so, physicists can still work with the proton when it is in the ‘metastable’ state.

Big science requires big computing

In parallel to a maybe fruitless or at least extremely difficult search for a theory that could predict chaotic motion, accelerator physicists at CERN try to design and understand the machine down to the finest detail, and then use computer models to simulate the protons’ motion over and over and over again, looking for any signs of instability.

The physicists know the machine well; the position, tilt and imperfections of each of the LHC’s 1232 dipole magnets, used to curve the path of the beam, is known down to the submillimeter, says Massimo Giovannozzi, leader of the LHC Commissioning and Upgrade section of the  accelerator physics group at CERN. Never mind the fact that each dipole magnet is 15 metres long and weighs 35 tonnes.

But there is a limit to what these accelerator physicists can know about each piece of the LHC, and limits on what they can correct for when they find imperfections. For example “the field quality of the magnets can only be known and improved up to a certain level,” Giovannozzi says. And they find out more and more precise details about the LHC as it continues to run today, “we will never stop doing simulations on the current machine, benchmarking our simulations against dedicated experiments probing the protons’ dynamics or looking for things that come into play at higher energies.”

The last of 1746 superconducting magnets is lowered into the LHC tunnel via a specially constructed pit at  26 April 2007. This 15-m long dipole magnet is one of 1232 dipoles positioned around the 27-km circumference of the collider. Dipole magnets produce a magnetic field that bends the particle beams around the circular accelerator, and volunteers helped physicists to design the stable beam coniditions. Image courtesy CERN.

The computer code they use for simulating the particles as they fly around the LHC, called SixTrack, was first developed back in the 1980s by Frank Schmidt at CERN, but it is still only an approximation of what is actually going on in the LHC. It looks at 60 protons at a time, which is only a small sample of the 1.5x10¹¹ protons in an actual bunch. And it simulates a million revolutions in the LHC, which corresponds to a few seconds of real time for a bunch of protons, but, in reality, a bunch of protons can spend up to 10 hours in the LHC.

They run simulations of protons in the machine over and over and over again, with slightly different initial conditions or a different sample of protons from the bunch, and different machine configuration to take into account the imperfect knowledge of some properties of the machine, to look for any signs of instability. Running these calculations could take “almost infinite computing power” says Schmidt.

In 2004, during preparations for the construction of the LHC Eric Mcintosh had the idea, inspired by SETI@home, to let anyone donate their computing power to CERN. He first established a way of obtaining identical results, bit for bit, from different computers. “This challenge is the work of a lifetime for me," says McIntosh, “and it is also a test of the correct operation of the computer."

The LHC@home project was then launched as a volunteer computing project using BOINC. Volunteers from around the world simply downloaded BOINC, which would run SixTrack simulations on their home and office computers whenever they weren’t in use.

One of the most crucial studies done with LHC@Home was a study of the effect of the two proton beams on each other as they approach collision. To perform the analysis of this so-called ‘parasitic beam-beam effect’, the group had access to only 100 PCs at CERN, and in 2005 when they ported it to LHC@Home, they got the equivalent of 72,000 PCs, running continuously for the six months of the study.

When the LHC doubles its energy to 7 TeV, which is scheduled for 2014-2015, these kinds of beam-beam interactions become critical, and without LHC@Home the accelerator physicists “would have had to fly blind” says Schmidt.

The true test of the simulations came when the LHC started operating. It started in September 2008, and in March 2010 it ramped up to half the design energy, thus reaching 3.5 TeV. “Up until now, we haven’t observed any particle losses that could be due to unstable behaviour, which is in agreement with the results of our simulations,” says Giovannozzi “a fantastic qualitative confirmation of our past simulation campaigns!”

New magnets planned for 2020

Now the current machine is operating even better than expected, which is also partly due to the performance of the lower-energy accelerators that are feeding protons to the LHC. Nevertheless, accelerator physicists have started to plan for the next stage of the machine, an upgrade to the magnets in 2020. Stronger magnets will be used to decrease the beam size ever further, and increase the chance of collisions inside the detectors. It’s essentially like a new machine, and the accelerator physicists have to do all new calculations. The proton-proton interactions will also become crucial to the stability of the beam.

“All these changes need to be simulated very carefully, and volunteer computing provides the resources that no one has,” says Rivkin.

Over the last year, Igor Zacharov, from EPFL, in close collaboration with McIntosh (who is retired now, but still active) have revived the LHC@Home beam dynamics study project so that anyone who wants to help design the next stage of the LHC can now volunteer their own computer (the program only runs while the computer is idle). There are already a few thousand users, and the physicists are getting “several results per minute,” Zacharov says.

“Still, we would like to recruit 10 times more for this resource to have major impact on the LHC upgrade."

Left: Physical space image of a particle for a stable orbit. Each point represents the particle position after one revolution around the accelerator. Middle: Physical space image of a particle for an unstable orbit. For unstable orbits, points tend to fill out the entire phase space ('phase space' is the physical position as well as momentum). At some point, unstable particles completely leave their orbit and they are lost. Right: the red area represents initial conditions that are stable up to 100,000 revolution around the LHC. The blue circles represent unstable initial conditions.