SEARCH · Search NASA
Results for “Distributed Computing Resources”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) (Final Report)
This report details the accomplishments from the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows” under the award numbers FWP-66406 and DE-SC0012630, with a focus on the UC San Diego (Award No. DE-SC0012630) part of the accomplishments. We refer to the project as IPPD. The main activities of IPPD were centered on the development and integration of provenance information to capture empirically workflow information and identify the sources of bottlenecks as well as variability, and modeling and simulation to provide insights and predictively explore multiple scenarios for development of advanced techniques for optimization of resources and workflow execution. After a Phase I of the project, a Phase II effort focused on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. The project leveraged and extended our existing tools with new research and demonstrated our work on the Belle II workflow suite as well as on workflows from NSLS-II.
Operational Evolution of FTS3: A DevOps Driven Approach to Elastic Operations
The File Transfer Service (FTS3) is a distributed data movement service developed at CERN and widely used to transfer data across the Worldwide LHC Computing Grid (WLCG). At Fermilab, FTS3 supports data transfers for multiple experiments, including Intensity Frontier experiments such as DUNE, enabling reliable data movement between WebDAV endpoints in Europe and the Americas. At CHEP 2021, we reported on the initial containerized deployment of FTS3 on OKD, the community Kubernetes distribution of Red Hat OpenShift. In this work, we present the subsequent evolution of this deployment, focusing on new operational capabilities introduced to improve scalability, robustness, and long-term maintainability. We describe the adoption of more secure and reproducible container build workflows, the integration of DevOps-driven operational practices, and enhancements in monitoring and automation. A key new result is the introduction of horizontal scaling and elastic resource management, allowing FTS3 components to dynamically adapt to workload variations while maintaining service reliability. We also discuss improvements in fault tolerance and operational procedures derived from production experience. Finally, we summarize lessons learned from operating FTS3 as a Kubernetes-native service and outline how these developments have improved the resilience and efficiency of data movement operations at Fermilab.
Characterization of Stereo Vision Performance for Roving at the Lunar Poles
Surface rover operations at the polar regions of airless bodies, particularly the Moon, are of particular interest to future NASA science missions such as Resource Prospector (RP). Polar optical conditions present challenges to conventional imaging techniques, with repercussions to driving, safeguarding and science. High dynamic range, long cast shadows, opposition and white out conditions are all significant factors in appearance. RP is currently undertaking an effort to characterize stereo vision performance in polar conditions through physical laboratory experimentation with regolith simulants, obstacle distributions and oblique lighting.
SciDAC's Earth System Grid Center for Enabling Technologies Semiannual Progress Report October 1, 2010 through March 31, 2011
This report summarizes work carried out by the Earth System Grid Center for Enabling Technologies (ESG-CET) from October 1, 2010 through March 31, 2011. It discusses ESG-CET highlights for the reporting period, overall progress, period goals, and collaborations, and lists papers and presentations. To learn more about our project and to find previous reports, please visit the ESG-CET Web sites: http://esg-pcmdi.llnl.gov/ and/or https://wiki.ucar.edu/display/esgcet/Home. This report will be forwarded to managers in the Department of Energy (DOE) Scientific Discovery through Advanced Computing (SciDAC) program and the Office of Biological and Environmental Research (OBER), as well as national and international collaborators and stakeholders (e.g., those involved in the Coupled Model Intercomparison Project, phase 5 (CMIP5) for the Intergovernmental Panel on Climate Change (IPCC) 5th Assessment Report (AR5); the Community Earth System Model (CESM); the Climate Science Computational End Station (CCES); SciDAC II: A Scalable and Extensible Earth System Model for Climate Change Science; the North American Regional Climate Change Assessment Program (NARCCAP); the Atmospheric Radiation Measurement (ARM) program; the National Aeronautics and Space Administration (NASA), the National Oceanic and Atmospheric Administration (NOAA)), and also to researchers working on a variety of other climate model and observation evaluation activities. The ESG-CET executive committee consists of Dean N. Williams, Lawrence Livermore National Laboratory (LLNL); Ian Foster, Argonne National Laboratory (ANL); and Don Middleton, National Center for Atmospheric Research (NCAR). The ESG-CET team is a group of researchers and scientists with diverse domain knowledge, whose home institutions include eight laboratories and two universities: ANL, Los Alamos National Laboratory (LANL), Lawrence Berkeley National Laboratory (LBNL), LLNL, NASA/Jet Propulsion Laboratory (JPL), NCAR, Oak Ridge National Laboratory (ORNL), Pacific Marine Environmental Laboratory (PMEL)/NOAA, Rensselaer Polytechnic Institute (RPI), and University of Southern California, Information Sciences Institute (USC/ISI). All ESG-CET work is accomplished under DOE open-source guidelines and in close collaboration with the project's stakeholders, domain researchers, and scientists. Through the ESG project, the ESG-CET team has developed and delivered a production environment for climate data from multiple climate model sources (e.g., CMIP (IPCC), CESM, ocean model data (e.g., Parallel Ocean Program), observation data (e.g., Atmospheric Infrared Sounder, Microwave Limb Sounder), and analysis and visualization tools) that serves a worldwide climate research community. Data holdings are distributed across multiple sites including LANL, LBNL, LLNL, NCAR, and ORNL as well as unfunded partners sites such as the Australian National University (ANU) National Computational Infrastructure (NCI), the British Atmospheric Data Center (BADC), the Geophysical Fluid Dynamics Laboratory/NOAA, the Max Planck Institute for Meteorology (MPI-M), the German Climate Computing Centre (DKRZ), and NASA/JPL. As we transition from development activities to production and operations, the ESG-CET team is tasked with making data available to all users who want to understand it, process it, extract value from it, visualize it, and/or communicate it to others. This ongoing effort is extremely large and complex, but it will be incredibly valuable for building 'science gateways' to critical climate resources (such as CESM, CMIP5, ARM, NARCCAP, Atmospheric Infrared Sounder (AIRS), etc.) for processing the next IPCC assessment report. Continued ESG progress will result in a production-scale system that will empower scientists to attempt new and exciting data exchanges, which could ultimately lead to breakthrough climate science discoveries.
Real-Time Optimal Power Flow
Future power networks are expected to incorporate a large number of distributed energy resources, which introduce randomness and fluctuations as well as fast control capabilities. But traditional optimal power flow methods are only appropriate for applications that operate on a slow timescale. In this paper, we build on recent work to develop a real-time algorithm for AC optimal power flow, based on quasi-Newton methods. The algorithm uses second-order information to provide suboptimal solutions on a fast timescale, and can be shown to track the optimal power flow solution when the estimated second-order information is sufficiently accurate. We also give a specific implementation based on L-BFGS-B method, and show by simulation that the proposed algorithm has good performance and is computationally efficient.
Recent Progress on Rans-Based Transition Model Verification
The current efforts to assess and improve the Reynolds-averaged Navier-Stokes (RANS)-coupled transition models in the NASA FUN3D and OVERFLOW codes are summarized in this study. The first AIAA Transition Modeling Workshop and the NATO AVT-313 Transition Workshop both emphasized the need for code verification for transport equations based transition models as a top priority. We discuss the methods used for the model verification, the resulting grid families, the flow solutions, and other supporting information collected with at least two established NASA flow solvers, namely, FUN3D and OVERFLOW. These results, which will be uploaded onto the NASA Turbulence Modeling Resource, should assist other members of the computational fluid dynamics (CFD) community in verifying their own implementations of various transition models, such as the Langtry-Menter (LM2009) model, the one-equation γ model, and Coder’s amplification factor transport (AFT) model. Grid convergence is assessed using both global and local flow metrics of interest such as lift and drag as well as local skin-friction coefficients. We also explore the anisotropic unstructured metric-based adaptive mesh refinement library known as refine with the NASA FUN3D solver to determine if this capability can achieve the same accuracy as handcrafted structured grids with a significantly smaller node count and to learn the characteristics of the resulting grid distribution, especially in the vicinity of the transition zone.
Recent Progress on RANS-Based Transition Model Verification
The current efforts to assess and improve the Reynolds-averaged Navier-Stokes (RANS)-coupled transition models in the NASA FUN3D and OVERFLOW codes are summarized in this study. The first AIAA Transition Modeling Workshop and the NATO AVT-313 Transition Workshop both emphasized the need for code verification for transport equations based transition models as a top priority. We discuss the methods used for the model verification, the resulting grid families, the flow solutions, and other supporting information collected with at least two established NASA flow solvers, namely, FUN3D and OVERFLOW. These results, which will be uploaded onto the NASA Turbulence Modeling Resource, should assist other members of the computational fluid dynamics (CFD) community in verifying their own implementations of various transition models, such as the Langtry-Menter (LM2009) model, the one-equation γ model, and Coder’s amplification factor transport (AFT) model. Grid convergence is assessed using both global and local flow metrics of interest such as lift and drag as well as local skin-friction coefficients. We also explore the anisotropic unstructured metric-based adaptive mesh refinement library known as refine with the NASA FUN3D solver to determine if this capability can achieve the same accuracy as handcrafted structured grids with a significantly smaller node count and to learn the characteristics of the resulting grid distribution, especially in the vicinity of the transition zone.
Optimal Load Ensemble Control in Chance-Constrained Optimal Power Flow
Distribution system operators (DSOs) world-wide foresee a rapid roll-out of distributed energy resources. From the system perspective, their reliable and cost effective integration requires accounting for their physical properties in operating tools used by the DSO. This work describes an decomposable approach to leverage the dispatch flexibility of thermostatically controlled loads (TCLs) for operating distribution systems with a high penetration level of photovoltaic resources. Each TCL ensemble is modeled using the Markov Decision Process (MDP). The MDP model is then integrated with a chance constrained optimal power flow that accounts for the uncertainty of PV resources. Since the integrated optimization model cannot be solved efficiently by existing dynamic programming methods or off-the-shelf solvers, this paper proposes an iterative Spatio-Temporal Dual Decomposition algorithm (ST-D2). Finally, we demonstrate the merits of the proposed integrated optimization and ST-D2 algorithm on the IEEE 33-bus test system.
SPARC: Demonstrate burst-buffer-based checkpoint/restart on ATS-1.
Recent high-performance computing (HPC) platforms such as the Trinity Advanced Technology System (ATS-1) feature burst buffer resources that can have a dramatic impact on an application’s I/O performance. While these non-volatile memory (NVM) resources provide a new tier in the storage hierarchy, developers must find the right way to incorporate the technology into their applications in order to reap the benefits. Similar to other laboratories, Sandia is actively investigating ways in which these resources can be incorporated into our existing libraries and workflows without burdening our application developers with excessive, platform-specific details. This FY18Q1 milestone summaries our progress in adapting the Sandia Parallel Aerodynamics and Reentry Code (SPARC) in Sandia’s ATDM program to leverage Trinity’s burst buffers for checkpoint/restart operations. We investigated four different approaches with varying tradeoffs in this work: (1) simply updating job script to use stage-in/stage out burst buffer directives, (2) modifying SPARC to use LANL’s hierarchical I/O (HIO) library to store/retrieve checkpoints, (3) updating Sandia’s IOSS library to incorporate the burst buffer in all meshing I/O operations, and (4) modifying SPARC to use our Kelpie distributed memory library to store/retrieve checkpoints. Team members were successful in generating initial implementation for all four approaches, but were unable to obtain performance numbers in time for this report (reasons: initial problem sizes were not large enough to stress I/O, and SPARC refactor will require changes to our code). When we presented our work to the SPARC team, they expressed the most interest in the second and third approaches. The HIO work was favored because it is lightweight, unobtrusive, and should be portable to ATS-2. The IOSS work is seen as a long-term solution, and is favored because all I/O work (including checkpoints) can be deferred to a single library.
Optimizing High-Throughput Inference on Graph Neural Networks at Shared Computing Facilities with the NVIDIA Triton Inference Server
Abstract With machine learning applications now spanning a variety of computational tasks, multi-user shared computing facilities are devoting a rapidly increasing proportion of their resources to such algorithms. Graph neural networks (GNNs), for example, have provided astounding improvements in extracting complex signatures from data and are now widely used in a variety of applications, such as particle jet classification in high energy physics (HEP). However, GNNs also come with an enormous computational penalty that requires the use of GPUs to maintain reasonable throughput. At shared computing facilities, such as those used by physicists at Fermi National Accelerator Laboratory (Fermilab), methodical resource allocation and high throughput at the many-user scale are key to ensuring that resources are being used as efficiently as possible. These facilities, however, primarily provide CPU-only nodes, which proves detrimental to time-to-insight and computational throughput for workflows that include machine learning inference. In this work, we describe how a shared computing facility can use the NVIDIA Triton Inference Server to optimize its resource allocation and computing structure, recovering high throughput while scaling out to multiple users by massively parallelizing their machine learning inference. To demonstrate the effectiveness of this system in a realistic multi-user environment, we use the Fermilab Elastic Analysis Facility augmented with the Triton Inference Server to provide scalable and high-throughput access to a HEP-specific GNN and report on the outcome.
Clouds and Earth Radiant Energy System (CERES), a Review: Past, Present and Future
The Clouds and Earth Radiant Energy System (CERES) project s objectives are to measure the reflected solar radiance (shortwave) and Earth-emitted (longwave) radiances and from these measurements to compute the shortwave and longwave radiation fluxes at the top of the atmosphere (TOA) and the surface and radiation divergence within the atmosphere. The fluxes at TOA are to be retrieved to an accuracy of 2%. Improved bidirectional reflectance distribution functions (BRDFs) have been developed to compute the fluxes at TOA from the measured radiances with errors reduced from ERBE by a factor of two or more. Instruments aboard the Terra and Aqua spacecraft provide sampling at four local times. In order to further reduce temporal sampling errors, data are used from the geostationary meteorological satellites to account for changes of scenes between observations by the CERES radiometers. A validation protocol including in-flight calibrations and comparisons of measurements has reduced the instrument errors to less than 1%. The data are processed through three editions. The first edition provides a timely flow of data to investigators and the third edition provides data products as accurate as possible with resources available. A suite of cloud properties retrieved from the MODerate-resolution Imaging Spectroradiometer (MODIS) by the CERES team is used to identify the cloud properties for each pixel in order to select the BRDF for each pixel so as to compute radiation fluxes from radiances. Also, the cloud information is used to compute radiation at the surface and through the atmosphere and to facilitate study of the relationship between clouds and the radiation budget. The data products from CERES include, in addition to the reflected solar radiation and Earth emitted radiation fluxes at TOA, the upward and downward shortwave and longwave radiation fluxes at the surface and at various levels in the atmosphere. Also at the surface the photosynthetically active radiation and ultraviolet radiation (total, UVA and UVB) are computed. The CERES instruments aboard the Terra and Aqua spacecraft have served well past their design life times. A CERES instrument has been integrated onto the NPP platform and is ready for launch in 2011. Another CERES instrument is being built for launch in 2014, and plans are being made for a series of follow-on missions.
Collaborative Decision Approach for Electricity Pricing-demand Response Stackelberg Game
Demand response programs are considered as a valuable resource in smart grids that provide several advantages of load shifting, peak load reduction, mediating intermittency of renewable energy integration, etc. Flexible price-based incentives have been recognized as a critical strategy in motivating and compensating consumers' load adjustment actions for successful implementation of demand response. Game theoretical approaches, especially Stackelberg games are popularly adopted to model the relationship between electricity price and customers' demand response and solved by the classical centralized backward induction (BI) method. However, the BI method generally requires convexity of the follower's model for necessary optimality conditions, and the computational time of any centralized approach increases sharply with larger problem instances. In this paper, the Stackelberg game of electricity pricing-demand response between a distribution system operator (DSO) and load aggregators (LAs) is decomposed based on a collaborative optimization (CO) framework, where each LA is treated as a discipline with its own domain constraints (e.g. building temperature control), while the DSO at the system level tries to reduce the solution discrepancy and guide the searching towards optimality. Several groups of comparison experiments have demonstrated the effectiveness of the proposed collaborative decision approach in solving the demand response game.
Eastern Renewable Generation Integration Study
The Eastern Interconnection (EI) is one of the largest power systems in the world, and its size and complexity have historically made it difficult to study in high levels of detail in a modeling environment. In order to understand how this system might be impacted by high penetrations (30% of total annual generation) of wind and solar photovoltaic (PV) during steady state operations, the National Renewable Energy Laboratory (NREL) and the U.S. Department of Energy (DOE) conducted the Eastern Renewable Generation Integration Study (ERGIS). This study investigates certain aspects of the reliability and economic efficiency problem faced by power system operators and planners. Specifically, the study models the ability to meet electricity demand at a 5-minute time interval by scheduling resources for known ramping events, while maintaining adequate reserves to meet random variation in supply and demand, and contingency events. To measure the ability to meet these requirements, a unit commitment and economic dispatch (UC&ED) model is employed to simulate power system operations. The economic costs of managing this system are presented using production costs, a traditional UC&ED metric that does not include any consideration of long-term fixed costs. ERGIS simulated one year of power system operations to understand regional and sub-hourly impacts of wind and PV by developing a comprehensive UC&ED model of the EI. In the analysis, it is shown that, under the study assumptions, generation from approximately 400 GW of combined wind and PV capacity can be balanced on the transmission system at a 5-minute level. In order to address the significant computational burdens associated with a model of this detail we apply novel computing techniques to dramatically reduce simulation solve time while simultaneously increasing the resolution and fidelity of the analysis. Our results also indicate that high penetrations of wind and PV (collectively variable generation (VG)), significantly impact the operation of traditional generating resources and cause these resources to be used less frequently and operate across a broader output range because wind and PV have lower operating costs and variable output levels.
Physical and Hydraulic Properties of RCRA Borehole Samples from the Hanford Site : Final Report, Fiscal Years 2023-2024
Sediment from 44 core samples collected from Resource Conservation and Recovery Act (RCRA) boreholes drilled in the 200 East and 200 West areas of the Hanford Site were characterized for physical and hydraulic properties (Table S.1). Characterization data included gravimetric water contents and matric potentials, grain-size distributions, saturated hydraulic conductivity, water retention characteristics, and unsaturated hydraulic conductivity. These properties provide site-specific data and parameters that can be used in subsurface flow and contaminant transport models to assess the transport and fate of contaminants in the vadose zone and underlying aquifer systems. The analyzed core samples come from specific areas and depth intervals at the Hanford Site that were targeted for sampling to address data gaps identified by site contractors (Khaleel 2020). X-ray computed tomography (XCT) was used to evaluate the general textural characteristics of the samples and to determine which samples to use for further physical and hydraulic property characterization. Subsequent sample selection was determined by consensus after review of the XCT images by Pacific Northwest National Laboratory, Central Plateau Cleanup Company, and INTERA staff.
DER Digital Supply Chain Gap Analysis
Solar photovoltaic (PV) cybersecurity is a growing field of research. As deployments of solar PV have increased, cyber risk has also increased. Utility solar PV installations, however, are not required to comply with the North American Electric Reliability Corporation (NERC) Critical Infrastructure Protection (CIP) plan unless they meet a minimum generation threshold of 75 MW. Individual residential-scale solar PV deployments will not meet that generation threshold and are therefore excluded from the NERC CIP requirements. With most solar installations less than 75 MW, solar PV has been deployed with minimal oversight and highly variable cybersecurity maturity. The resources that comprise the digital supply chain can include software, code, data, and other digital components. But as clean energy technologies advance, cybersecurity threats and vulnerabilities continue to evolve and grow in sophistication. Supply chain cybersecurity represents a critical area for ensuring safe operations as the U.S. moves toward a clean energy future.
A brief parallel I/O tutorial.
This document provides common best practices for the efficient utilization of parallel file systems for analysts and application developers. A multi-program, parallel supercomputer is able to provide effective compute power by aggregating a host of lower-power processors using a network. The idea, in general, is that one either constructs the application to distribute parts to the different nodes and processors available and then collects the result (a parallel application), or one launches a large number of small jobs, each doing similar work on different subsets (a campaign). The I/O system on these machines is usually implemented as a tightly-coupled, parallel application itself. It is providing the concept of a 'file' to the host applications. The 'file' is an addressable store of bytes and that address space is global in nature. In essence, it is providing a global address space. Beyond the simple reality that the I/O system is normally composed of a small, less capable, collection of hardware, that concept of a global address space will cause problems if not very carefully utilized. How much of a problem and the ways in which those problems manifest will be different, but that it is problem prone has been well established. Worse, the file system is a shared resource on the machine - a system service. What an application does when it uses the file system impacts all users. It is not the case that some portion of the available resource is reserved. Instead, the I/O system responds to requests by scheduling and queuing based on instantaneous demand. Using the system well contributes to the overall throughput on the machine. From a solely self-centered perspective, using it well reduces the time that the application or campaign is subject to impact by others. The developer's goal should be to accomplish I/O in a way that minimizes interaction with the I/O system, maximizes the amount of data moved per call, and provides the I/O system the most information about the I/O transfer per request.
REopt Federal User Support Session Two
This presentation is for FEMP's second user support session for federal REopt users. These sessions cover topics such as: overviews of REopt capabilities, hands-on REopt trainings and demonstrations, discussion and feedback about the tool and user support, federal REopt case studies and success stories, and links to REopt resources. This particular session covers discussion about federal onsite energy projects and challenges, using REopt to evaluate air-source and geothermal heat pumps, a federal success story about using REopt for heat pump analysis, a summary of available resources, and open question and answer session.