Friday, January 18, 2019
VBLHEP Big Conference Hall
V.V. Korenkov

Trends and prospects of development of distributed computing and Big Data analitics to support megascience projects

Experiments at the Large Hadron Collider (LHC) at CERN play a leading role in scientific research in elementary particle physics, nuclear physics, and Big Data analytics (BD). The global distributed system for processing, storing and analyzing WLCG data - Worldwide LHC Computing GRID - unites resources of about 180 computer centers in 50 countries of the world. The total amount of data storage is more than 1000 Petabytes. Data analysis is carried out using high-performance complexes (Grid), academic, national and commercial cloud computing resources, supercomputers and other resources. The Russian centers SIC KI and JINR are involved in the integration of distributed heterogeneous resources and the development of database technologies to support projects in high-energy physics, astrophysics, bioinformatics, etc. At JINR, work is underway to build the NIKA complex, which requires new approaches to data analysis. Megasiens class installations are created in large organizations in Russia and other countries (PIC, FAIR, ELI, SKA, etc.). The review of integrated infrastructures of megaprojects and trends of their evolution is presented. Attention is paid to new directions in data warehousing (DataLake), integration of databases, supercomputer technologies, and machine learning methods.