Dubna, Moscow region, April 18, 2018. — RSC Group, a leading Russian and well-known in the world supercomputing industry developer and integrator of innovative ultrahigh-dense, scalable and energy efficient solutions for high-performance computing (HPC) and data centers, deployed the world first 100% ‘hot water’ liquid cooled supercomputer at Joint Institute for Nuclear Research (JINR) located in Dubna, Moscow Region (Russia). This new system was named after Nikolay Nikolayevich Govorun, former Director of JINR’s Computing and Automation Lab in 1988-1989, who worked on development of information technologies in this international scientific organization since 1966.
New supercomputer of the Joint Institute for Nuclear Research has been developed and deployed for the Theoretical Physics Laboratory (TPL) named after N.N. Bogolyubov and the Information Technology Laboratory (ITL) with support of JINR management. The project was completed by RSC Group with Intel specialists support.
The project involves plans of radical acceleration of comprehensive theoretical and experimental research in the fields of particle physics, nuclear physics and condensed matter physics. This includes deployment of NICA accelerator complex based at JINR to recreate quark-gluon plasma – a special state of matter of the Universe in first moments after the Big Bang. NICA mega-project started in 2013 and it is planned to commission this accelerator facility in 2020.
New supercomputer of the Joint Institute for Nuclear Research will enable resource-intensive massive parallel calculations to solve problems of lattice quantum chromodynamics for studying the properties of hadronic matter with high energy density and baryon charge and in presence of strong electromagnetic fields. It will greatly accelerate simulation of collision dynamics of relativist heavy ions and provide new resources to study the properties of highly correlated systems in the physics of new materials. It will also be used to develop and adapt software for NICA mega-project for new computing architectures, create an innovative hardware and software environment for high performance computing and simulating complex scientific problems and to train IT specialists in all relevant areas.
“It's great to note that we launch the new heterogeneous supercomputer named after professor Govorun at JINR’s Information Technology Laboratory in the year of 60th anniversary of commissioning of the first Ural-1 supercomputer at our institute in 1958. Our scientists and research groups now have a powerful and modern tool that will greatly accelerate theoretical and experimental research of nuclear physics and condensed matter physics,” – said Vladimir Vasilyevich Korenkov, Director of the Information Technology Laboratory of the Joint Institute for Nuclear Research.
New JINR's supercomputer is a natural development of HybriLIT heterogeneous platform with significant increase of performance. Theoretical peak performance of the new supercomputer is about 500 Teraflops (in double precision).
The new supercomputer includes high dense and energy efficient RSC Tornado solution based on Intel server products with 100% ‘hot water’ direct liquid cooling. The Joint Institute for Nuclear Research deployed new universal RSC Tornado computing cabinets with record-breaking power density and precision liquid cooling balanced for continuous usage of high temperature cooling agent (up to +63°C at cabinet entry point). JINR’s specifications required optimal cabinet temperature mode with +45°C constant cooling agent temperature at node entries (+57°C peak temperature).
‘Hot water’ mode enabled deployment of free cooling solution (24x7x365) using only dry coolers running at up to +50°C ambient temperature and complete elimination of freon circuit and chillers. As the result, average power efficiency factor (PUE) is less than 1.027. The system uses less than 3% of consumed power for cooling, which is an outstanding result for HPC industry.
Computing nodes of JINR system are based on Intel server products: the most powerful 72-core Intel® Xeon Phi™ 7290 processors, Intel® Xeon® Scalable family processors (Intel® Xeon® Gold 6154), Intel® Server Board S7200AP and Intel® Server Board S2600BP, Intel® SSD DC S3520 with SATA interface in M.2 form-factor and the newest high-speed Intel® SSD DC P45xx drives NVMe interface and 1TB capacity.
High-speed data transfer between JINR’s supercomputer nodes uses innovative Intel® Omni-Path Architecture communication technology providing up to 100 Gbps non-blocking switching rate. It is based on 48-port Intel® Omni-Path Edge Switch 100 Series with 100% liquid cooling to provide high efficiency of the cooling system in ‘hot water’ mode and the lowest TCO of the system. Intel® Omni-Path Architecture meets current requirements of resource-intensive user applications and provides sufficient throughput reserve for the future.
Innovative monitoring and management system based on RSC BasIS software stack guarantees high availability, redundancy and simplicity of usage of computing systems based on RSC solutions. It can be used to manage single nodes and the entire solution, including infrastructure components. All elements of the system (computing nodes, power supplies, hydraulic regulation modules, etc.) have an integrated management module providing broad capabilities for detailed telemetry and flexible management. Cabinet design supports replacement of computing nodes, power supplies and hydraulic regulation modules (with redundancy) in hot-swap mode without interruption of system operation. Most components of the system (such as computing nodes, power supplies, network and infrastructure components, etc.) are software-defined, and this significantly simplifies and speeds up initial deployment, maintenance and future upgrades of the system. 100% liquid cooling of all components ensures their longevity.
“We are glad that RSC’s innovative HPC solutions will help to advance the Russian science and to improve the efficiency of international scientific research cooperation that has a long history in JINR,” - said Alexander Moskovsky, CEO of RSC Group.
RSC has unique experience of developing, creating and operating high-density energy efficient supercomputer facilities with direct liquid cooling. The company is a de-facto global technological leader in this field. RSC’s key customers are higher education institutions (leading Russian universities), scientific institutions, research centers, laboratories and design agencies. Besides the new JINR project, RSC solutions are installed and actively used for simulations and computing of a wide range of research and industrial problems in St. Petersburg State Polytechnic University named after Peter the Great (SPbPU), Joint Supercomputer Center of the Russian Academy of Sciences (JSCC RAS), Siberian Supercomputer Center (SSCC) of the Institute of Computing Mathematics and Mathematical Geophysics of the Siberian Branch of RAS, South Urals State University (SUSU), Oceanology Institute of RAS named after P. P. Shirshov, Moscow Institute of Physics and Technology (MIPT), Russian Weather Forecast Agency (Roshydromet) and other customers from various industries and fields including civil aircraft engine production, energy industry, computer graphics, oil&gas industry, etc.
