Russian RSC Group deployed the world first 100% ‘hot water’ liquid cooled supercomputer at Joint Institute for Nuclear Research
Dubna, Moscow region, April 18, 2018. — RSC Group, a leading Russian and well-known in the world supercomputing industry developer and integrator of innovative ultrahigh-dense, scalable and energy efficient solutions for high-performance computing (HPC) and data centers, deployed the world first 100% ‘hot water’ liquid cooled supercomputer at Joint Institute for Nuclear Research (JINR) located in Dubna, Moscow Region (Russia). This new system was named after Nikolay Nikolayevich Govorun, former Director of JINR’s Computing and Automation Lab in 1988-1989, who worked on development of information technologies in this international scientific organization since 1966.
New supercomputer of the Joint Institute for Nuclear Research has been developed and deployed for the Theoretical Physics Laboratory (TPL) named after N.N. Bogolyubov and the Information Technology Laboratory (ITL) with support of JINR management. The project was completed by RSC Group with Intel specialists support.
The project involves plans of radical acceleration of comprehensive theoretical and experimental research in the fields of particle physics, nuclear physics and condensed matter physics. This includes deployment of NICA accelerator complex based at JINR to recreate quark-gluon plasma – a special state of matter of the Universe in first moments after the Big Bang. NICA mega-project started in 2013 and it is planned to commission this accelerator facility in 2020.
New supercomputer of the Joint Institute for Nuclear Research will enable resource-intensive massive parallel calculations to solve problems of lattice quantum chromodynamics for studying the properties of hadronic matter with high energy density and baryon charge and in presence of strong electromagnetic fields. It will greatly accelerate simulation of collision dynamics of relativist heavy ions and provide new resources to study the properties of highly correlated systems in the physics of new materials. It will also be used to develop and adapt software for NICA mega-project for new computing architectures, create an innovative hardware and software environment for high performance computing and simulating complex scientific problems and to train IT specialists in all relevant areas.
“It's great to note that we launch the new heterogeneous supercomputer named after professor Govorun at JINR’s Information Technology Laboratory in the year of 60th anniversary of commissioning of the first Ural-1 supercomputer at our institute in 1958. Our scientists and research groups now have a powerful and modern tool that will greatly accelerate theoretical and experimental research of nuclear physics and condensed matter physics,” – said Vladimir Vasilyevich Korenkov, Director of the Information Technology Laboratory of the Joint Institute for Nuclear Research.
New JINR's supercomputer is a natural development of HybriLIT heterogeneous platform with significant increase of performance. Theoretical peak performance of the new supercomputer is about 500 Teraflops (in double precision).
The new supercomputer includes high dense and energy efficient RSC Tornado solution based on Intel server products with 100% ‘hot water’ direct liquid cooling. The Joint Institute for Nuclear Research deployed new universal RSC Tornado computing cabinets with record-breaking power density and precision liquid cooling balanced for continuous usage of high temperature cooling agent (up to +63°C at cabinet entry point). JINR’s specifications required optimal cabinet temperature mode with +45°C constant cooling agent temperature at node entries (+57°C peak temperature).
‘Hot water’ mode enabled deployment of free cooling solution (24x7x365) using only dry coolers running at up to +50°C ambient temperature and complete elimination of freon circuit and chillers. As the result, average power efficiency factor (PUE) is less than 1.027. The system uses less than 3% of consumed power for cooling, which is an outstanding result for HPC industry.
Computing nodes of JINR system are based on Intel server products: the most powerful 72-core Intel® Xeon Phi™ 7290 processors, Intel® Xeon® Scalable family processors (Intel® Xeon® Gold 6154), Intel® Server Board S7200AP and Intel® Server Board S2600BP, Intel® SSD DC S3520 with SATA interface in M.2 form-factor and the newest high-speed Intel® SSD DC P45xx drives NVMe interface and 1TB capacity.
High-speed data transfer between JINR’s supercomputer nodes uses innovative Intel® Omni-Path Architecture communication technology providing up to 100 Gbps non-blocking switching rate. It is based on 48-port Intel® Omni-Path Edge Switch 100 Series with 100% liquid cooling to provide high efficiency of the cooling system in ‘hot water’ mode and the lowest TCO of the system. Intel® Omni-Path Architecture meets current requirements of resource-intensive user applications and provides sufficient throughput reserve for the future.
Innovative monitoring and management system based on RSC BasIS software stack guarantees high availability, redundancy and simplicity of usage of computing systems based on RSC solutions. It can be used to manage single nodes and the entire solution, including infrastructure components. All elements of the system (computing nodes, power supplies, hydraulic regulation modules, etc.) have an integrated management module providing broad capabilities for detailed telemetry and flexible management. Cabinet design supports replacement of computing nodes, power supplies and hydraulic regulation modules (with redundancy) in hot-swap mode without interruption of system operation. Most components of the system (such as computing nodes, power supplies, network and infrastructure components, etc.) are software-defined, and this significantly simplifies and speeds up initial deployment, maintenance and future upgrades of the system. 100% liquid cooling of all components ensures their longevity.
“We are glad that RSC’s innovative HPC solutions will help to advance the Russian science and to improve the efficiency of international scientific research cooperation that has a long history in JINR,” - said Alexander Moskovsky, CEO of RSC Group.
RSC has unique experience of developing, creating and operating high-density energy efficient supercomputer facilities with direct liquid cooling. The company is a de-facto global technological leader in this field. RSC’s key customers are higher education institutions (leading Russian universities), scientific institutions, research centers, laboratories and design agencies. Besides the new JINR project, RSC solutions are installed and actively used for simulations and computing of a wide range of research and industrial problems in St. Petersburg State Polytechnic University named after Peter the Great (SPbPU), Joint Supercomputer Center of the Russian Academy of Sciences (JSCC RAS), Siberian Supercomputer Center (SSCC) of the Institute of Computing Mathematics and Mathematical Geophysics of the Siberian Branch of RAS, South Urals State University (SUSU), Oceanology Institute of RAS named after P. P. Shirshov, Moscow Institute of Physics and Technology (MIPT), Russian Weather Forecast Agency (Roshydromet) and other customers from various industries and fields including civil aircraft engine production, energy industry, computer graphics, oil&gas industry, etc.
Joint Institute for Nuclear Research (JINR) is an international inter-governmental scientific research organization established by eleven member countries on March 26, 1956 and registered in UN on February 1, 1957. JINR is located in Dubna, Moscow region (Russia).
18 countries are members of JINR: Azerbaijan, Armenia, Belarus, Bulgaria, Vietnam, Georgia, Kazakhstan, Democratic People's Republic of Korea, Cuba, Moldova, Mongolia, Poland, Russia, Romania, Slovakia, Uzbekistan, Ukraine, Czech Republic. The institute has government-level Cooperation Agreements with Hungary, Germany, Egypt, Italy, Serbia and South African Republic.
Main areas of scientific and experimental research In JINR: particle physics, nuclear physics, condensed matter physics. International Scientific Council is responsible for development of JINR’s science policy.
JINR includes seven laboratories: Theoretical Physical Laboratory named after N. N. Bogolyubov, Nuclear Problems Laboratory named after V.P. Jelepov, Nuclear Reactions Laboratory named after G.N. Flerov, High Energy Physics Laboratory named after V.I. Vexler and A.M. Baldin, Neutron Physics Laboratory named after I.M. Frank, Information Technology Laboratory, Radiation Biology Laboratory. Each laboratory conducts research on a scale of a large academic research institute. JINR has about 4500 staff employees including over 1200 scientists with members and correspondent members of national academies of sciences, over 260 doctors and 570 candidates of sciences, about 2000 engineering and technical employees.
JINR has an impressive collection of experimental physical appliances: the first superconducting accelerator of heavy cores and heavy ions in Europe and Asia (nuclotron), U-400, U-400M cyclotrons, new DC-280 cyclotron, unique neutron pulse reactor IBR-2 and proton accelerator (FM cyclotron) used for radiation treatment. JINR successfully works on construction of a mega science grade experimental facility — NICA (Nuclotron based Ion Collider facility). JINR actively works on creating a modern heavy ion accelerator complex DRIBs (Dubna Radioactive Ion Beams) and a super heavy element factory, which is a key part of this project.
Dubna’s scientists have synthesized new long-lived super heavy elements with sequence numbers 113, 114, 115, 116, 117 and 118. The International Union of Pure and Applied Chemistry acknowledged contribution of JINR’s scientists in modern physics and chemistry naming element 105 of the Mendeleyev’s Periodic System “dubnium”; element 114 – flerovium in honor of JINR's Nuclear Reactions Laboratory and its founder, academic G. N. Flerov; element 115 ‒ moscovium in honor of Moscow region where JINR is located; and element 118 – oganesson in honor of professor Yuri Oganesyan for his principal contribution in TAE research.
JINR has powerful HPC facilities integrated with global computer networks through high-speed communication channels (100 Gbps). The project of JINR’s Multi-Functional Information and Computing Facility is intended to create a technological base for scientific research in unified computing environment combining numerous technological solutions, concepts and techniques. JINR successfully operates the only Tier-1 grid center in JINR countries for СМS experiment on LHC, which became one of the best CMS centers in WLCG ‒ global grid infrastructure for data processing, analysis and storage.
Broad international scientific cooperation is an important aspect of JINR’s activity: JINR maintains connections with over 800 scientific centers and universities in 64 countries of the world. Only in Russia, the largest JINR partner, the institute cooperates with 150 research centers, universities, industrial plants and other companies from 50 Russian cities.
JINR has created excellent training environment for talented young scientists. JINR’s Education and Science Center runs education program of the Institute to prepare top grade young scientists to research in JINR's laboratories and scientific centers of member countries. Dubna town hosts a branch of the Moscow State University, theoretical physics and nuclear physics departments of the Dubna International Institute of Nature, Society and Human.
For more details see JINR’s official website: http://www.jinr.ru/main-en/.
About RSC Group
RSC Group is the leading Russian developer and integrator of innovative high-density HPC and data center solutions based on Intel architecture, the latest liquid cooling technologies and its own extensive know-how. Since 2018, RSC is a member of “National Champions” priority project of the Ministry of Economic Development of the Russian Federation.
RSC has the potential to create the most energy efficient solutions with record-breaking power usage effectiveness (PUE), the highest computing density in the industry with standard x86-based processors, to use fully "green" design, provide the highest solution reliability, noise-free operation of computing modules, 100% compatibility and guaranteed scalability with unmatched low cost of ownership and low power consumption. RSC specialists also have the experience of developing and implementing an integrated software stack of solutions to improve work efficiency and application of supercomputer systems from system software to vertically oriented platforms based on cloud computing technologies.
RSC is Intel® Technology Provider Program member at Platinum level, Intel® Fabric Builders Program member, Intel® HPC and Data Center Specialist and participate at Intel® Solutions for Lustre Reseller Elite Program. Performance and scalability of RSC Tornado solutions are Intel® Cluster Ready certified and certified as Intel® Select Solution for Simulation and Modeling.
For more information please visit www.rscgroup.ru.
RSC, RSC BasIS, PetaStream and RSC logos are registered trademarks of RSC Group in Russia, USA, Japan and many European countries.