The Belle experiment at the KEKB electron-positron collider, based at the High Energy Accelerator Research Organization (KEK) in Tsukuba, Japan, made significant contributions to the field of particle physics before it shut down 30 June 2010. Among them was the observation of charge-parity violation in B mesons. Today, physicists are preparing for the next steps, which they've dubbed SuperKEKB and Belle II. Hiroyuki Matsunaga of KEK reports.
Physicists used to think that charge-parity is a perfect symmetry of the universe: the laws of physics will remain the same if a particle is interchanged with its antiparticle (charge symmetry) and then the left and right are swapped (parity symmetry, sometimes also called mirror symmetry). But in 1964, observations of particles called kaons showed that they sometimes violate CP symmetry, leaving physicists with a burning question: why does this happen? Could this explain why there is more matter than antimatter in the universe?
It was Makoto Kobayashi and Toshihide Maskawa who, in 1972, found the answer to the first question; this earned them half of the 2008 Nobel Prize in physics (the other half was awarded to Yoichiro Nambu who, in 1960, introduced the then theoretical concept of spontaneous symmetry violation).
The emerging standard model of particle physics made numerous predictions that could be tested, and a new generation of high-energy physics experiments was born. Some were searching for particles that the model predicted. Others sought to observe and measure more examples of CP violation. Belle was among the experiments looking for CP violation in the decay of B mesons – and they found it in 2001. Today, scientists know that this CP violation is not large enough to explain the imbalance of matter and antimatter in our universe. Belle II and other experiments are looking for sources of CP violation beyond the Standard Model.
The KEKB accelerator and the Belle experiment are currently undergoing a major upgrade and will resume operations as the SuperKEKB and the Belle II experiment, respectively, in early 2015. The amount of raw data is expected to be roughly 30 petabytes per year in the early stage of the experiment and will increase twofold in the full operation phase.
The target of the SuperKEKB is to provide the world's highest luminosity, topping KEKB's by a factor of 40. By 2021, the SuperKEKB will have produced 50 times more collisions than the KEKB ever made. This will allow scientists to search for new physics beyond the Standard Model of particle physics through precision studies of rare and forbidden processes.
As a result, the data logging rate to the data storage will reach gigabytes per second, and the data volume will be much larger than that of the Belle experiment and the LHC experiments.
In light of the huge amount of data, the Belle II computing group has decided to use remote computing resources outside KEK through distributed computing technology. Since many collaborating institutes are involved in the Worldwide LHC Computing Grid (WLCG), the grid will act as a major pillar, while cloud resources may also be used to meet the peak demand. KEK holds the primary responsibility for archiving and processing all the raw data, and plays a central role in the Belle II Grid infrastructure, which includes the "Grid Operation Centre" and massive data transfers to and from remote sites. The remote resources will be used for Monte Carlo simulations and physics analyses; KEK is also supposed to provide a portion of resources for these purposes.
KEK's Computing Research Center (CRC) has been supporting the data analysis of the Belle experiment by providing computing resources and operational staff. Until recently, CRC has operated the Belle computer system in addition to the central computer system used by many smaller experiments, but the two systems merged into a single system that serves all projects at KEK. Most of the resources, particularly data storage, will be used by Belle and Belle II, but the share of resource use per group or user can be changed fairly easily.
Data storage is one of the most important subsystems. To manage the data storage as a Hierarchical Storage Management system, a High Performance Storage System and a General Parallel File System - both of which had been used for the previous central computer system - were chosen. A GPFS/HPSS Interface was also chosen. Unlike the previous Belle computer system, grid middleware is installed in this system and uses a sizable amount of resources. The new system was introduced this spring and will be replaced in 2015, just after the Belle II experiment starts taking data. Still, it is important to validate the performance of the new system for high throughput under the Belle II computing environment. The results of the validation can serve as a reference for the design of the next system to be used in the full operation phase of the Belle II experiment.
Although KEK is not a WLCG site, CRC has several years of experience in the deployment and development of the Grid middleware. There was a project of Japan's National Research Grid Initiative (NAREGI), aimed at providing a large-scale distributed computing environment. CRC has been involved in the deployment of NAREGI middleware, and run demonstrations of high energy physics applications on NAREGI.
In the Resources Linkage for E-Science (RENKEI) project that has followed NAREGI, the CRC team has been working on the development of application interfaces, which provide a seamless environment to users by hiding differences between grid middleware. The team has been running the Integrated Rule-Oriented Data System (iRODS) in the production system that is successfully used by the Tokai-to-Kamioka (T2K) neutrino experiment. They also operate a Certificate Authority, KEK GRID CA, that issues certificates to Japanese scientists and Grid sites in Japan.
Last but not least, the middleware gLite has been running in production at KEK, although the resources were more limited than those provided by the new system. The gLite system supports many virtual organizations (VOs), some of which (including Belle VO) are served by their own Virtual Organization Membership Services (VOMS).
So far, the computing resources have mostly been used by International Linear Collider physicists and the Belle computing group. The gLite middleware has been deployed on the new system at KEK, like most of the other Belle II sites.
In the Belle II Grid, where the tier structure is not used, KEK becomes a hub to connect participating sites. KEK started to build the Belle II Grid infrastructure in collaboration with the following institutions: the University of Melbourne in Australia, the Czech Educational and Scientific Network (CESNET) in the Czech Republic, the Grid Computing Centre Karlsruhe (GridKA) in Germany, the Tata Institute of Fundamental Research in India, the Pacific Northwest National Laboratory (PNNL) in the US, the Jozef Stefan Institute in Slovenia, the Korea Institute of Science and Technology Information in Korea, and the Institute of High Energy Physics in China. More sites are expected to join in the future.
In order to establish collaborations, it is important to know each other. Holding face-to-face meetings between sites, preferably by visiting a partner's site, would be desirable to build mutual trust. In January, the CRC team visited PNNL, where the Open Science Grid will be used for Belle II; most other sites chose gLite middleware. A meeting with most Belle II partners at the European Grid Infrastructure community forum in Munich took place in late March, and they discussed how to establish the Belle II Grid.
With many tasks needed to complete to create a working system for Belle II, there is not much time left before data taking, which is scheduled to start in 2015. The CRC team has just begun with building a network monitoring system of all sites involved, and running gLite on the new computer system at KEK. Unlike the WLCG, Belle II member institutions are geographically dispersed and the network connectivity between sites is complex with no dedicated links, which may cause difficulties for operations and communications. Hence, it is essential to build a grid infrastructure within a short period of time and subsequently start various tests and practical operations, which will play a key role in the success of the Belle II Grid.