Search NASA⌕ Search

SEARCH · Search NASA

Results for “NERSC”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Evaluating and optimizing the NERSC workload on Knights Landing

NERSC has partnered with 20 representative application teams to evaluate performance on the Xeon-Phi Knights Landing architecture and develop an application-optimization strategy for the greater NERSC workload on the recently installed Cori system. In this article, we present early case studies and summarized results from a subset of the 20 applications highlighting the impact of important architecture differences between the Xeon-Phi and traditional Xeon processors. We summarize the status of the applications and describe the greater optimization strategy that has formed.

97 MATHEMATICS AND COMPUTING↗

Towards understanding HPC users and systems: A NERSC case study

High performance computing (HPC) scheduling landscape currently faces new challenges due to thechanges in the workload. Previously, HPC centers were dominated by tightly coupled MPI jobs. HPCworkloads increasingly include high-throughput, data-intensive, and stream-processing applications. Asa consequence, workloads are becoming more diverse at both application and job levels, posing newchallenges to classical HPC schedulers. There is a need to understand the current HPC workloads andtheir evolution to facilitate informed future scheduling research and enable efficient scheduling in futureHPC systems.In this paper, we present a methodology to characterize workloads and assess their heterogeneity,at a particular time period and its evolution over time. We apply this methodology to the workloads ofthree systems (Hopper, Edison, and Carver) at the National Energy Research Scientific Computing Center(NERSC). We present the resulting characterization of jobs, queues, heterogeneity, and performance thatincludes detailed information of a year of workload (2014) and evolution through the systems’ lifetime(2010–2014).

Rodrigo, Gonzalo P.↗

NERSC_Lightweight Distributed Metric Service (NERSC_LDMS) v4.4.2

Miscellany This LDMS Loftsman/Helm Chart horizontally scales LDMS daemons in order to achieve a 1Hz sample rate from over 5,000 nodes, collecting 38k metrics per minute on Perlmutter. This LMDS Configuration relies on already running `ldmsd` producers running on nodes, which produce metrics via sampler plugins. The Helm chart distributes the collection of metrics from producer acrross many aggregator and storage `ldmsd` daemons, ensuring no damon is overloaded and data loss is avoided.

Stile, John [Lawrence Berkeley National Laboratory↗

Software Deployment Process at NERSC: Deploying the Extreme-scale Scientific Software Stack (E4S) Using Spack at the National Energy Research Scientific Computing Center (NERSC)

One of the many benefits of using a high-performance computing (HPC) system at a Department of Energy (DOE) Office of Science (SC) HPC facility is the large number of software products, built and optimized for the system. The HPC center staff and HPC vendors provide optimized software such as libraries and even full scientific applications, ready to be used by users as building blocks to accelerate scientific discovery. Behind each provided packaged software module are a large number of decisions - which compiler, optimizations, variants/options - to build the software on the target system. And, even before the software gets deployed, the software must be developed, tested, and maintained, including deprecating old versions and ensuring compatibility across versions. The software lifecycle is complex and is further convoluted by a web of interdependencies on other software.

97 MATHEMATICS AND COMPUTING↗

Large Scale Computing and Storage Requirements for Basic Energy Sciences: Target 2017. Report of the NERSC Requirements Review Conducted October 8–9, 2013, Gathersburg, MD

The National Energy Research Scientific Computing Center (NERSC) is the primary computing center for the DOE Office of Science, serving approximately 5,000 users working on some 700 projects that involve nearly 600 codes in a wide variety of scientific disciplines. In addition to large-scale computing and storage resources NERSC provides support and expertise that help scientists make efficient use of its systems. The latest review revealed several key requirements, in addition to achieving its goal of characterizing BES computing and storage needs. high-level findings are: 1) Scientists will need access to significantly more computational and storage resources to achieve their goalds and reach BES research objectives; 2) Users will need assistance from NERSC to prepare for Cori (NERSC-8) and follow-on manycore systems; 3) Research teams need to run complex jobs of many different types and scales; 4) BES is a leader in innovative use of HPC and requires a diverse set of resources and services from NERSC; and 5) BES facilities need computational analysis and data storage resources beyond what they can provide. This report expands upon these key points and add others. The results are based upon representative samples, called "case studies," of the needs of science teams within BES. The case study topics were selected by the NERSC meeting coordinators and BES program managers to represent the BES production computing workload. Prepared by BES workshop participants, the case studies contain a summary of science goals, methods of solution, current and future computing requirements, and special software and support needs. Also included are strategies for computing in the highly parallel "many-core" environment that is expected to dominate HPC architectures over the next few years.

97 MATHEMATICS AND COMPUTING↗

Best Practices for NERSC Training

The National Energy Research Supercomputing Center (NERSC) at Lawrence Berkeley National Laboratory (LBNL) organizes approximately 20 training events per year for its 8,000 users from 800 projects, who have varying levels of High Performance Computing (HPC) knowledge and familiarity with NERSC's HPC resources. Due to the novel circumstances of the pandemic, NERSC began transforming our traditional smaller-scale, on-site training events to larger-scale, fully virtual sessions in March 2020. We treated this as an opportunity to try new approaches and improve our training best practices. This paper describes the key practices we have developed since the start of this transformation, including considerations for organizing events; collaboration with other HPC centers and the DOE ECP Program to increase reach and impact of events; targeted emails to users to increase attendance; efficient management of user accounts for computational resource access; strategies for preventing Zoombombing; streamlining the publication of professional-quality, closed-captioned videos on the NERSC YouTube channel for accessibility; effective communication channels for Q&A; tailoring training contents to NERSC user needs via close collaboration with vendors and presenters; standardized training procedures and publishing of training materials; and considerations for planning HPC training topics. Additionally, most of these practices will be continued after the pandemic as effective norms for training.

97 MATHEMATICS AND COMPUTING↗

Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores

The newest NERSC supercomputer Cori is a Cray XC40 system consisting of 2,388 Intel Xeon Haswell nodes and 9,688 Intel Xeon-Phi “Knights Landing” (KNL) nodes. Compared to the Xeon-based clusters NERSC users are familiar with, optimal performance on Cori requires consideration of KNL mode settings; process, thread, and memory affinity; fine-grain parallelization; vectorization; and use of the high-bandwidth MCDRAM memory. This paper describes our efforts preparing NERSC users for KNL through the NERSC Exascale Science Application Program, Web documentation, and user training. We discuss how we configured the Cori system for usability and productivity, addressing programming concerns, batch system configurations, and default KNL cluster and memory modes. Here, system usage data, job completion analysis, programming and running jobs issues, and a few successful user stories on KNL are presented.

97 MATHEMATICS AND COMPUTING↗

Quantum Computing Technology Roadmaps and Capability Assessment for Scientific Computing - An analysis of use cases from the NERSC workload

The National Energy Research Scientific Computing Center (NERSC), as the high-performance computing (HPC) facility for the Department of Energy’s Office of Science, recognizes the essential role of quantum computing in its future mission. In this report, we analyze the NERSC workload and identify materials science, quantum chemistry, and high-energy physics as the science domains and application areas that stand to benefit most from quantum computers. These domains jointly make up over 50% of the current NERSC production workload, which is illustrative of the impact quantum computing could have on NERSC’s mission going forward. We perform an extensive literature review and determine the quantum resources required to solve classically intractable problems within these science domains. This review also shows that the quantum resources required have consistently decreased over time due to algorithmic improvements and a deeper understanding of the problems. At the same time, public technology roadmaps from a collection of ten quantum computing companies predict a dramatic increase in capabilities over the next five to ten years. Our analysis reveals a significant overlap emerging in this time frame between the technological capabilities and the algorithmic requirements in these three scientific domains. We anticipate that the execution time of large-scale quantum workflows will become a major performance parameter and propose a simple metric, the Sustained Quantum System Performance (SQSP), to compare system-level performance and throughput for a heterogeneous workload.

97 MATHEMATICS AND COMPUTING↗

NERSC-6 Workload Analysis and Benchmark Selection Process

This report describes efforts carried out during early 2008 to determine some of the science drivers for the"NERSC-6" next-generation high-performance computing system acquisition. Although the starting point was existing Greenbooks from DOE and the NERSC User Group, the main contribution of this work is an analysis of the current NERSC computational workload combined with requirements information elicited from key users and other scientists about expected needs in the 2009-2011 timeframe. The NERSC workload is described in terms of science areas, computer codes supporting research within those areas, and description of key algorithms that comprise the codes. This work was carried out in large part to help select a small set of benchmark programs that accurately capture the science and algorithmic characteristics of the workload. The report concludes with a description of the codes selected and some preliminary performance data for them on several important systems.

97 MATHEMATICS AND COMPUTING↗

Accelerating Science with the NERSC Burst Buffer Early User Program

NVRAM-based Burst Buffers are an important part of the emerging HPC storage landscape. The National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory recently installed one of the first Burst Buffer systems as part of its new Cori supercomputer, collaborating with Cray on the development of the DataWarp software. NERSC has a diverse user base comprised of over 6500 users in 700 different projects spanning a wide variety of scientific computing applications. The use-cases of the Burst Buffer at NERSC are therefore also considerable and diverse. We describe here performance measurements and lessons learned from the Burst Buffer Early User Program at NERSC, which selected a number of research projects to gain early access to the Burst Buffer and exercise its capability to enable new scientific advancements. To the best of our knowledge this is the first time a Burst Buffer has been stressed at scale by diverse, real user workloads and therefore these lessons will be of considerable benefit to shaping the developing use of Burst Buffers at HPC centers.

97 MATHEMATICS AND COMPUTING↗

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Large File System Backup: NERSC Global File System Experience

NERSC's Global File system (NGF), accessible from all compute systems at NERSC, holds files and data from many scientific projects. A full backup of this file system to our High Performance Storage System (HPSS) is performed periodically. Disk storage usage by projects at NERSC has grown seven fold over a two year period, from ~;;20TB in June 2006 to ~;;140 TB in June 2008. The latest full backup took about 13 days and more than 200 T10k tape cartridges (.5 TB capacity). Petabyte file systems are becoming a reality in the next few years and the existing utilities are already strained in handling backup tasks.

97 MATHEMATICS AND COMPUTING↗

NERSC Annual Report 2008-2009

This report presents highlights of the research conducted on NERSC computers in a variety of scientific disciplines during the years 2008-2009. It also reports on changes and upgrades to NERSC's systems and services as well as activities of NERSC staff.

97 MATHEMATICS AND COMPUTING↗

Real-Time XFEL Data Analysis at SLAC and NERSC: a Trial Run of Nascent Exascale Experimental Data Analysis

X-ray scattering experiments using Free Electron Lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID-19 viral proteins). XFEL experiments are a challenge to computing in two ways: i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; ii) data collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min -- sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS-NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Furthermore, completing data analysis with 10 mins is a first for XFEL experiments and an important milestone if we are to keep up with data collection trends.

Blaschke, Johannes P.↗

Invisible Milky Way Satellite Uncovered With Help from NERSC

Astronomers predict that large spiral galaxies, like our Milky Way, have hundreds of satellite galaxies orbiting around them. While a few satellites are visible, like the Magellanic Clouds, many other galaxies are too dim to see. Scientists suspect that these faint satellite galaxies are primarily comprised of mysterious "dark matter"; which makes up 85 percent of all matter in the universe and so far remains undetected. Using supercomputers at the National Energy Research Scientific Computing Center (NERSC), Sukanya Chakrabarti, an assistant professor of physics at Florida Atlantic University, developed a mathematical method to uncover these "dark"; satellites. When she applied this method to our own Milky Way galaxy, Chakrabarti discovered a faint satellite might be lurking on the opposite side of the galaxy from Earth, approximately 300,000 light-years from the galactic center. According to Chakrabarti, the dark satellite galaxies create disturbances in the cold atomic hydrogen gas at the edges of the spiral galaxy's disk, and these perturbations reveal the mass, distance and location of the satellite. With the help of NERSC systems, she successfully validated her method by analyzing the radio observations of the Whirlpool Galaxy, which has a visible satellite one-third of its size, and NGC 1512, which has a satellite one-hundredth its size. Her calculations correctly predicted the mass and location of both of the known satellite galaxies. http://www.lbl.gov/cs/Archive/news031411.html

79 ASTRONOMY AND ASTROPHYSICS↗

Python in the NERSC Exascale Science Applications Program for Data

We describe a new effort at the National Energy Re- search Scientific Computing Center (NERSC) in performance analysis and optimization of scientific Python applications targeting the Intel Xeon Phi (Knights Landing, KNL) many- core architecture. The Python-centered work outlined here is part of a larger effort called the NERSC Exascale Science Applications Program (NESAP) for Data. NESAP for Data focuses on applications that process and analyze high-volume, high-velocity data sets from experimental/observational science (EOS) facilities supported by the US Department of Energy Office of Science. We present three case study applications from NESAP for Data that use Python. These codes vary in terms of “Python purity” from applications developed in pure Python to ones that use Python mainly as a convenience layer for scientists without expertise in lower level programming lan- guages like C, C++ or Fortran. The science case, requirements, constraints, algorithms, and initial performance optimizations for each code are discussed. Our goal with this paper is to contribute to the larger conversation around the role of Python in high-performance computing today and tomorrow, highlighting areas for future work and emerging best practices

Ronaghi, Zahra↗

Checkpoint/Restart Vision and Strategies for NERSC’s Production Workloads

As a primary approach to fault-tolerant computing, Checkpoint/Restart (C/R) improves scientific productivity for users, provides scheduling flexibility for computing centers, and protects against system failures. While both applicationspecific (or application-level) and transparent C/R are used in practice, we are interested in transparent checkpointing, which is vital for system-level checkpointing. Developing and maintaining transparent C/R tools for HPC applications, however, is labor intensive and highly complex due to ever-changing HPC systems and diverse production workloads. Existing C/R tools are often research-oriented, so there is a gap to close before they can be used reliably with production workloads, especially on cutting edge HPC systems. In this position paper, we present our journey to prepare a production-ready MPI-Agnostic Network-Agnostic (MANA) transparent checkpointing tool for NERSC, and share our vision and strategies to bring transparent C/R capabilities to NERSC’s production workloads on current and future systems.

42 ENGINEERING↗

Edison - A New Cray Supercomputer Advances Discovery at NERSC

When a supercomputing center installs a new system, users are invited to make heavy use of the computer as part of the rigorous testing. In this video, find out what top scientists have discovered using Edison, a Cray XC30 supercomputer, and how NERSC's newest supercomputer will accelerate their future research.

97 MATHEMATICS AND COMPUTING↗