Virtual atom smasher in LHC@Home

“When I learned that if successful, [Test4Theory] would allow BOINC users to participate in physics experiments being done at CERN - and ultimately perhaps even the search for the Higgs Boson itself - I jumped at the chance to be a part of it,” said Tony De Bari.

De Bari, who is from New Jersey, USA, joined the project in February 2011, after finding out about it from forum posts online and emailing the developers to ask for an invitation code. He now has three computers, at work and at home, contributing time to the LHC@Home Test4Theory project.

“Even though I do not have a formal physics background, I am a self-proclaimed ‘science geek’ with a particular interest in physics. While pursuing my computer science degree, I took as many elective physics courses as time would permit, and I try to read as much about the subject as I can," he said.

The developers hope their LHC@Home software will be used for several research projects in the future. Test4Theory, the first project, has been in the alpha testing phase since October 2010, and now has more than 100 volunteers just like De Bari. Even at this early stage, volunteers have already provided about 10% of the total computing resources currently available to theoretical physicists at CERN, according to Anton Karneyeu, one of the developers on the project who also works on the CMS experiment.

Sharing computing power

CERN may be the epicenter of the Worldwide LHC Computing Grid (WLCG), one of the largest distributing computing networks in the world - containing about 250,000 processing cores distributed across 36 countries – but the resource is almost exclusively for the use of the experiments (ATLAS, CMS, ALICE and LHCb), which pump data out of their detectors at about 300 MB per second.

The computational power available to the theoretical physicists, by comparison, is minor: about 50 theoretical physicists at CERN share a small computing cluster.

“I think we can get 10,000 or maybe more people participating in Test4Theory. This would make a real difference to the resources available to theoretical physicists,” said Peter Skands, the theoretical physicist who, together with the developers, came up with the idea for Test4Theory about a year ago.

Image courtesy Wikicommons

Generating random events

Skands uses Monte Carlo simulations to determine the rate at which specific events should occur in the experiments. Monte Carlo simulations, named after the area in Monaco famous for its casinos, generate explicit random numbers. For example, you could use a Monte Carlo simulation to generate a string of one million dice rolls, and for each roll you know you would get a number between 1 and 6, but it could also be random and independent of the other rolls.

Similarly, in relativistic quantum field theory - the theory that combines quantum mechanics with special relativity and describes the particle physics being explored at the LHC - the particles are governed by probabilities, the probability of being in a certain state or the probability of interacting in a certain way.

The Monte Carlo simulations generate specific events randomly, based on the probabilities that define the interactions. The simulation can also be extended to include hypothetical new laws of nature, and results compared to data, to find which hypothesis fits best.

If a particular event is extremely rare – like rolling a die a million times and getting a one every single time – then a lot of simulations are required in order to find it. And “generating these random events to find that one-in-a-million, exceptionally rare one requires a lot of computing power,” said Skands.

Extremely rare events in particle collisions

Unfortunately for physicists, it is the rare events that are the most interesting in the study of particle interactions. For example, in April this year, Fermilab’s Tevatron in Illinois, USA, reported a mysterious bump in their data that, as claimed at the time, could point to new elementary particles or even a new force of nature.

In a proton-proton (grey circle, 'p') collision, the color force is mediated by gluons (curly lines). Gluons can produce a quark and an anti-quark (b and anti-b). Unlike the electrically neutral photons that mediate the electric force, gluons actually carry a color charge themselves and so they can interact with other gluons, as illustrated by the lower diagram. This makes quantum chromodynamics much more difficult to simulate and to analyze than quantum electrodynamics. Image courtesy Peter Skands and Jacqui Hayes.

What the proton-antiproton collisions had produced is called a W-jet-jet (Wjj) event: two jets of lightweight particles, called hadrons, and a heavy-force-carrying particle called the W boson (about 85 times heavier than the proton). However, extremely rare events in quantum chromodynamics can mimic such a signal, and must be accurately understood to determine if there is really a signal of something new in the data. The physicists are still studying these events, and the jury is still out.

Another event modeled with computationally expensive Monte Carlo simulations is the generation of beauty and anti-beauty quarks after a proton-proton collision, as detected in the LHCb experiment; LHCb is designed to study the asymmetry between matter and anti-matter apparent in the universe. After LHCb detects and records the quark pairs, the physicists compare these measurements to what is predicted by theory.

The CERN 'virtual machine'

To help with these simulations, volunteers must first download VirtualBox, which is free software from Oracle to allow a virtual machine to run on their home computer. The virtual machine is a ‘computer within a computer’, complete with its own operating system.

The VirtualBox system allows someone with a Mac operating system or Windows on their home computers to run a Scientific Linux virtual machine (VM) called CernVM, which runs the software that almost all the experiments and theorists at CERN use. This virtual machine can then communicate directly with the CERN experiments and theoreticians using software called Co-Pilot, developed by Artem Harutyunyan at CERN.

“The best thing about this set-up is that the physicists themselves don’t have to port their programs or understand BOINC. They see all the volunteer machines as part of a Volunteer Cloud. It can support all the LHC experiments as well as the theorists," said Ben Segal, one of the lead developers behind LHC@Home Test4Theory.

How to get involved in LHC@Home's Test4Theory

Volunteers can download a package, compressed to about 200 MB, which consists of BOINC software, a CERN virtual machine and some ‘wrapping’ code that acts as the interface between the home computer and the virtual machine.

The wrapping code, previously developed by several CERN students and currently coordinated by Daniel Lombraña González, from the Citizen Cyberscience Centre at CERN, then opens up the virtual machine on the home computer and runs CERN jobs, typically event simulations, via the Co-Pilot link for 24 hours at a time – some jobs may take only and hour or two, and occasionally a single job may take up the entire 24 hours.

Though computing some of the events may have taken several hours, it ends up taking only few minutes to transfer the final files back to CERN. Every 24 hours the volunteers receive BOINC credit for their work.

New opportunities for BOINC

The use of virtual machines is new for BOINC projects. The wrapping code will likely be used for future projects at CERN and elsewhere, said Lombraña González.

The volunteers were encouraged to actively participate in this part of the project as well. “The most exciting thing for me is that for the first time in my distributed computing ‘life’, I have the opportunity to help build a project from the ground up, and to help take the BOINC platform into areas where it couldn't go before,” said De Bari.

“When a new iteration of the BOINC worker app or virtual machine image are released, not only do we get to find the bugs, but we are encouraged to help develop the solutions as well.  For someone with my background, it doesn't get much better than that,” he said. 

Can volunteers create a virtual atom smasher?

“We would never be able to simulate as many events as the LHC,” Skands said. “The fastest Monte Carlo I have would produce one collision per millisecond, on average. But the LHC produces 40 million per second.”

Skands paused and leant back in his chair. “How many volunteers would we need to produce as many events as the LHC?” He shrugged as he said it, and he meant it merely as a rhetorical question, a nod to the incomprehensible size and speed of the LHC. But it suddenly dawned on him that he could actually produce a fully-fledged virtual atom smasher if just 40,000 volunteers ran his Monte Carlo simulations at the same time.

“I never even considered that a possibility before now,” said Skands. “Honest to God, that would be a fun thing to do!”