RSC Group presents St. Petersburg Polytechnic University supercomputer center and new scientific research results achieved with RSC PetaStream™ massively parallel supercomputer at the international supercomputer conference ISC’15
Frankfurt am Main (Germany), International Supercomputing Conference (ISC’15), July 13th, 2015 – RSC Group, the leading Russian developer and system integrator of innovative solutions for high-performance computing (HPC) and data centers, has presented a new St. Petersburg Polytechnic University supercomputer center with Polytechnic RSC Tornado and Polytechnic RSC PetaStream systems as well as new scientific research results achieved with RSC PetaStream™ massively parallel supercomputer at the international conference ISC'15.
SPbPU supercomputer center
Last year, RSC has completed deployment one of the largest and the most innovative data centers in Russia for the St. Petersburg Polytechnic University. RSC specialists have developed and deployed two new computing systems with total peak performance over 1.1 Petaflops.
SPbPU supercomputer center objectives are:
• Improving efficiency of fundamental and applied scientific research at SPbPU with supercomputer technologies;
• Training qualified engineers highly competent in supercomputer technology usage for hi-tech product development;
• Creating a regional competence center based on SPbPU specializing in supercomputer technology usage in knowledge intensive industries (power machinery, aircraft construction, bioengineering, radioelectronics).
"Our Supercomputer Center will be fully open and its computing resources and services will be available not only to SPbPU scientists but also to other research institutions of St. Petersburg and Russia to solve important scientific research tasks in the fields of mechanical engineering, hydrodynamics, aerodynamics, plasma physics, material science, electronics, biophysics and biotechnologies, cybersecurity. Developers and engineers of the University and production companies will be able to use powerful workstations and CAD/CAM systems provided by the Center to design new products for power machinery, biotechnologies, radioelectronics and other industries. For students and teachers the Center becomes the technology base for deep training of specialists in the fields of computer science and computer engineering. We proudly present SPbPU supercomputer center, one of the most innovative HPC centers in Russia, at ISC’15 international conference. First of all we would like to point on abilities of our scientists in various subject areas that require high performance computing. We would also like to know more about the experience of our foreign colleagues in this field," – said Vladislav Sinepol, Director of SPbPU supercomputer center.
New Polytechnic RSC Tornado cluster system with direct liquid cooling has the peak performance of 829 Teraflops. It is the 3rd in Russian Top50 list of the most powerful supercomputers with 658 Teraflops performance at LINPACK test as well as it is ranked by Top500 and Green500 global ratings. It is the first project in Russia and CIS based on new Intel® Xeon® E5-2600 v3 server processors.
Polytechnic RSC Tornado includes 712 dual-processor nodes containing 1424 high-performance Intel® Xeon® E5-2697 v3 server processors (14 cores, 2.6 GHz frequency each), Intel® S2600KP and Intel® S2600WT server boards for this processor generation, and the latest Intel® SSD DC S3500 solid state drives for enterprise data centers. Resources of this supercomputer will be used not only for relevant scientific research tasks but also for cloud, VDI (Virtual Desktop Infrastructure) and graphical services.
The second part of the hybrid HPC complex at SPbPU contains a unique high-dense massively parallel RSC PetaStream system with direct liquid cooling and total performance of 258 Teraflops. Polytechnic RSC PetaStream supercomputer has demonstrated LINPACK performance of 170.5 Teraflops and has 8th position in the current edition of the local Top50 list. This SPbPU supercomputer system is based on 60-core Intel® Xeon Phi™ 5120D and Intel® Xeon® E5-2600 v2 family processors with Intel server boards and Intel SSD DC S3500 solid-state drives for enterprise data centers.
Polytechnic RSC Tornado and Polytechnic RSC PetaStream are combined into a single HPC solution at SPbPU supercomputer center with high-speed Infiniband FDR switching network.
The new SPbPU supercomputer complex also includes a parallel data storage system with Lustre distributed file system that will support an use up to 1 Petabyte of data and a 0.5 Petabyte block-based storage for cloud environments. Both storages use server technologies based on Intel architecture.
New scientific results with RSC PetaStream
Various scientific groups actively use RSC PetaStream massively parallel supercomputers with Intel® Xeon Phi™ 5120D and Intel® Xeon Phi™ 7120D processors at St. Petersburg Polytechnic University and Joint Supercomputer Center of the Russian Academy of Sciences (JSCC RAS). This enabled new results in numerous prospective scientific research and achieving priority objectives in fundamental science some of which were presented at ISC’15 conference.
Scientists of the Computing Mathematics and Mathematical Geophysics Institute of the Siberian Division of the Russian Academy of Sciences and Novosibirsk State University continue their research in the field of supercomputer modeling of astrophysical processes using RSC PetaStream massively parallel supercomputer and AstroPhi application that they have developed to model astrophysical processes of various scales.
Galaxy collision is a regular part of their evolution. On the average any galaxy has up to ten collisions in Hubble time and there is a popular hypothesis stating that collisions of galaxies result in their diversity. Supercomputer modeling is the only way of studying such processes. As the number of supercomputers with hybrid architecture increases, the problem of creating efficient parallel computing methods for such platforms becomes the most important.
"Innovative technologies developed by RSC Group specialists and our ability to launch the AstroPhi on a powerful RSC PetaStream supercomputing system with nodes based on Intel Xeon Phi enabled us not only to create a galaxy collision model with record resolution using over a billion design cells but also to conduct new important research in the areas of star formation, supernova effects, cooling, heating and chemical evolution of colliding galaxies. With increased number of logical cores, high performance and scalability of RSC PetaStream provide significant acceleration of the modeling process and significant reduction of time to results with sufficient spatial resolution around 100 parsecs," – say Igor Chernykh and Igor Kulikov, scientists of the Computing Mathematics and Mathematical Geophysics Institute of the Siberian Division of the Russian Academy of Sciences and Novosibirsk State University.
Scientists of Petersburg Nuclear Physics Institute NRC “Kurchatov Institute” (PNPI NRC KI) in collaboration with Research Institute of Influenza and St. Petersburg Polytechnic University are researching special influenza virus A modification applicable for a new live vaccine creation. The live vaccine is more effective than inactivated, because immune system fights against real virus. The studied virus is reproducible at
low temperatures only (about 26 °C) and inactive at human body temperature (37 °С). Currently vaccine particular feature is supposed to be connected with amino acid substitution in influenza virus nucleoprotein (NP).
Several potential states of NP were simulated with GROMACS MD code on RSC PetaStream massively-parallel supercomputer system. Trajectory analysis is allowed to speak an assumption of reason of virus cold-adaptivity mechanism disorder. For assumption confirmation the consideration of full NP filament structure is needed.
NP filament structure
"High level of problem scalability was achieved using 64x Intel Xeon Phi processors at RSC PetaStream system – down to 50 atoms per thread. With such high scalability level full NP filament simulation becomes available. It’s planned to perform on 256x Intel Xeon Phi processors of Polytechnic RSC PetaStream system installed at SPbPU supercomputer center," – explain Evgeniy Petukhov, chief manager of SPbPU system software department and Alexey Shvetzov, junior scientist of molecular and radiation biophysics division at St. Petersburg Nuclear Physics Institute.
Intel Parallel Compute Center (IPCC) created at the Chemistry Department of Moscow State University with technical support of RSC specialists has received high score at GAMESS(US) benchmarks running on RSC PetaStream supercomputing platform. Their study is devoted to the optimization of GAMESS(US) code for native execution on massively parallel architectures like Intel Xeon Phi. GAMESS(US) is popular open-source quantum chemistry software that is developed and currently maintained by Prof. Mark Gordon group in Iowa State University. This software has officially no threading support which significantly limits its usage on novel parallel architectures.
"We studied parallelization of basic quantum chemistry methods like Hartree-Fock and DFT. Calculation of the Fock matrix is the main bottleneck of the Hartree-Fock and to some extent Kohn-Sham DFT methods. We were able to achieve a good scaling of the Fock matrix two-electron contribution code on hybrid MPI/OpenMP regime using up to 1920 threads. This result is very promising and it illustrates that Xeon Phi is well suited for quantum chemistry. The work of this IPCC was among several attempts to run traditional quantum chemistry codes on Intel Xeon Phi. It is remarkable, that the benchmark was done using native regime. Therefore we expect a good scaling also on next generations of Intel Xeon Phi platform," – says Vladimir Mironov, senior scientist of the Chemistry Department of Moscow State University.
Scientists of the Space Research Institute of the Russian Academy of Sciences and Earth Surveillance System of U.S. National Oceanic and Atmospheric Administration have developed a high performance algorithm for analyzing multi-spectral images of remote Earth probing (REP) in visible and infrared spectrum to research temperature fields and detect combustion sources on Earth surface. This analysis is very important to fight forest fires, reduce ecological damage from gas flaring at oil fields and rational usage of fishery resources.
"In the original version analysis of a single image took over 4000 seconds, but due to code adaptation ad optimization for RSC PetaStream massively parallel system at the Joint Supercomputer Center of the Russian Academy of Sciences, frame processing time was reduced to 12 seconds on 1st node (based on Intel Xeon Phi 7120D) and to less than 2 seconds on sixteen RSC PetaStream nodes. Therefore, calculation time was reduced by over 2000 times. This provides broad prospects of using the developed data analysis methods for data analysis with latest Suomi NPP, Landsat 8 and GOES-R meteorological satellites," – said Mikhail Zhizhin, chief scientist of the Space Research Institutes of the Russian Academy of Sciences.
Joint Supercomputer Center of the Russian Academy of Sciences used RSC PetaStream massively parallel supercomputer system to test several applications related to weather forecast. Weather Research and Forecasting (WRF) model has been tested in Conus 2.5 km version. WRF code is developed by a community of scientists including groups from the USA and Great Britain and is broadly used in various weather forecast researches including Russia. Performance measurement at 32 nodes of RSC PetaStream has demonstrated 60% acceleration of theoretically possible value. RSC specialists have migrated SL-AV model developed in Marchuk's Computing Mathematics Institute and Main Computing Center of Roshydromet (Russian Weather Forecast Agency) to RSC PetaStream platform. First performance measurements have shown good parallelism level of the software code – performance on 16 nodes of RSC PetaStream has increased by more than three times while the number of threads has increased by 4 times.
Specialists of Marchuk's Computing Mathematics Institute have researched performance of new dynamic ocean model with parallelism support based on 3-dimensional decomposition of subject area. First measurements have shown more than 30x acceleration at RSC PetaStream system deployed at JSCC RAS. These results and the above results demonstrate high potential of RSC PetaStream for calculations in the fields of climate science and meteorology.
RSC PetaStream™ massively parallel supercomputer
The RSC PetaStream™ architecture is distinguished with massively parallel concept providing execution of up to 250,000 parallel threads on 1024 x86 computing nodes in a single cabinet taking only 1 square meter of space. RSC PetaStream™ sets the world records of computing density (1.2 Petaflops per cabinet), power density (over 400 kW per cabinet), performance per volume (over 560 Teraflops/m3) and small footprint.
RSC PetaStream™ solution uses best available standard components of the highest integration level and a unique HPC system architecture developed by RSC specialists. Every node of massively parallel supercomputer is independent and based on 61-core Intel® Xeon Phi™ 7120D processor with 16GB high-speed GDDR5 memory working under a Linux family OS. All RSC PetaStream computing nodes are joined with high-speed interconnects based on InfiniBand FDR technology providing revolutionary ultra-dense HPC solution.
RSC PetaStream™ solution based on x86 architecture that is a de-facto industry standard throughout the world provides protection of investments in software development and optimization. The developers will be able to use existing programming model in new software for future computing systems thus getting the opportunity to run existing applications on Exascale level supercomputers. At the same time, flexibility of RSC PetaStream™ architecture enables testing of innovative programming models that might be more efficient for Exascale computing.
In addition to highest application performance provided by RSC PetaStream™, this solution enables double reduction of power consumption compared to traditional cluster systems in modeling tasks with high level of parallelism. Therefore, users can significantly reduce TCO while obtaining leading performance with small solution footprint.
Energy-efficient direct liquid cooling technology developed by RSC has been used by customers in numerous projects since 2009. It provides very high energy efficiency demonstrated by RSC PetaStream™ solution. According to measurement results for several RSC systems operated by customers, average PUE (power usage efficiency) is 1.06. It means that no more than 6% of power is used for system cooling. PUE is a critical factor for Exascale level supercomputers with power consumption level of several dozen megawatts.
RSC PetaStream™ implements an innovative subsystem based on the leading industry standard of 400V DC power developed in cooperation with Emerson Electric. This subsystem enables power distribution efficiency over 90% improving overall energy efficiency of the system, reduction of operating costs and overall reliability of the complex.
RSC PetaStream™ provides extended hardware and software system monitoring and control features using integrated RSC BasIS software stack for user convenience. The system uses Intel® Node Manager Technology for control and power consumption monitoring of every node enabling supercomputer centers to establish flexible policies of power optimization reducing power costs and costs of computing system operation.
RSC PetaStream™ solution is supplied with Intel® Cluster Studio XE tool set for Linux and a set of optimized libraries.
