Another stage of JINR MICC modernization completed On 17 April 2025, another stage of the modernization of the JINR Multifunctional Information and Computing Complex (MICC) was completed at the Meshcheryakov Laboratory of Information Technologies. At this stage, the engineering equipment providing power supply and cooling, including climate control systems and power distribution modules, was updated. Module 2 was modernized, and the initial modernization of Module 1 was performed. These modules house the server equipment and the Tier2 computing cluster, which is the most productive in the Russian grid segment (Russian Data Intensive Grid, RDIG). The work was carried out by the MLIT team under the supervision of the Laboratory’s Chief Engineer Andrey Dolbilov and MICC Chief Engineer Alexey Vorontsov. A module represents an air flow isolation system that connects two rows of free-standing equipment racks via a ceiling panel and a system of sliding doors, thereby providing cooling. Thus, an isolated cold corridor is formed inside the module; the heated air is discharged into the common space of the MICC hall. In total, in the MLIT machine hall of the 2nd floor, in addition to the separately standing “Govorun” supercomputer and IBM TS3500 and IBM TS4500 robotic tape libraries, there are four similar isolated modules. These are a module that ensures the operation of the Tier1 computing cluster, Modules 1 and 2 responsible for Tier2, as well as MICC Module 4 ensuring the operation of Tier2, the HybriLIT heterogeneous platform, the cloud infrastructure, the complex of information and computing resources, and the EOS data storage. Module 4 was created in 2020, it is equipped with advanced engineering equipment, and before the modernization, it was quite different in engineering solutions from Modules 1 and 2. In addition, in the machine hall there is a module that houses critical servers of the standard type of business computing (administrative systems and databases, etc.), and a network equipment module that houses the major network services for the MICC, local and global networks. In the foreseeable future, it is planned to build Module 5, which will integrate the network infrastructure with additional computing resources and data storage systems. Since the end of 2024, a lot of work to enhance the MICC engineering infrastructure was done. The necessary upgrade of the equipment providing the power supply and cooling of the computing complex was carried out. This stage of modernization was performed without stopping the operation of all active equipment and at the same time was completed promptly, with little loss of time and performance. It was required to design and create own power distribution modules (PDMs). This complex and extensive work was successfully completed by MLIT Senior Engineer Andrey Evlanov. The assembly, installation and connection of PDMs for MICC Modules 1 and 2 were carried out at the end of 2024. Cabinets with guaranteed power supply from Galaxy 7000 uninterruptible power supplies were installed and put into operation; eight PDMs were mounted in each module. Preparations for connecting three-phase power to the PDMs started. Later, in March-April of this year, Modules 1 and 2 were the first to be completely transferred to three-phase power supply. At the same time, during the modernization, it was decided to significantly enlarge the size of Module 2. In total, ten new cabinets for equipment were added to the module. To ensure the effective cooling of the updated module configuration, ten inter-row air conditioners were installed. In the future, during the next stage of modernization, similar work is planned for Module 1. This work embraced the physical relocation of five cabinets with servers involved in the operation of such services as Tier2 dCache, Tier2 WN, Main EOS, and cvmfs. For this reason, the Laboratory had to announce a downtime in the WLCG (Worldwide LHC Computing Grid) global grid infrastructure in advance on 27—31 March to avoid the loss of running user tasks and ensure data safety. “Power supply and cooling have increased by approximately 30% and have become much more efficient. The expansion of the number of rack cabinets has provided additional opportunities for installing new equipment, enhancing the computing power of Tier2, and developing the JINR MICC as a whole,” Andrey Dolbilov said. Work to modernize the MICC engineering infrastructure is performed in accordance with the Seven-Year Plan for the Development of JINR for 2024-2030, which, in particular, implies that the data storage capacity and computing power for the WLCG project, aimed at solving tasks related to JINR’s participation in the CERN experiments, should grow annually by 10-20%. This will allow maintaining the required data processing rate. The modernization provides the possibility of installing new equipment and actively updating the system, which will ensure the most reliable operation of the entire MICC complex. I would like to highlight a huge amount of work on the shutdown and removal of obsolete equipment, which was done by MICC specialists. Our team of young engineers and duty personnel work with total dedication and commitment. They ensure the smooth 24x7 operation of the MICC all year round and greatly contribute to its enhancement. Thanks to these people, our computing center plays a prominent role in the international scientific community,” Alexey Vorontsov noted. In January 2025, the multi-layer data storage and processing system of the “Govorun” supercomputer was enlarged by two new servers on top of Ruler technology with a total capacity of 2 petabytes. At present, these servers have been put into operation and are actively employed for computing tasks of the NICA megascience project. After commissioning, the total volume of the ultra-fast part of the hierarchical data storage system amounts to 10.6 petabytes, which will considerably reduce the overall execution time of “Govorun” supercomputer tasks. “I see how our computing center is becoming more powerful every year. In terms of a multitude of implemented solutions, it not only occupies a leading position among similar centers providing fundamental research, but also has a number of unique features such as the integration of technological solutions for distributed computing, high-performance computing, and a distributed data lake in a single computing center,” MLIT Director Sergei Shmatov commented.
