Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Computing Resources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Verification of an improved equation-free projective integration method for neoclassical plasma-profile evolution in tokamak geometry

A brute-force, long-time gyrokinetic simulation of plasma profile evolution in magnetic fusion devices is not desirable due to large computational resource requirements and a possible accumulation of numerical error. The equation-free projective integration method of Keverekidis et al. [Commun. Math. Sci. 1(4), 715–762 (2003)] is one of the outstanding candidates in projecting micro-scale simulations to a longer timescale. However, its application to tokamak plasma has not been fruitful due to the appearance of spurious transient oscillations in the lifting process, which are present when the kinetic simulations are initialized with a simplified model distribution function and which make the kinetic simulations to deviate from the desired paths. In this work, a kinetically informed lifting algorithm is added to the equation-free projective integration method, which is then verified in the electrostatic gyrokinetic particle-in-cell code XGCa [R. Hager and C. S. Chang, Phys. Plasmas 23, 042503 (2016)] for a neoclassical ion heat transport problem with adiabatic electrons. This new lifting operator is demonstrated to control spurious transients, enabling an over four-times reduction in the overall computing time in the time-evolution of the ion temperature profile in an axisymmetric toroidal plasma. Further reduction in the computing time is found to be limited due to the stability properties of the linear least squares projective integrator.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Characterization and classification of vegetation canopy structure and distribution within the Great Smoky Mountains National Park using LiDAR

Vegetation canopy structure is a critically important habit characteristic for many threatened and endangered birds and other animal species, and it is key information needed by forest and wildlife managers for monitoring and managing forest resources, conservation planning and fostering biodiversity. Advances in Light Detection and Ranging (LiDAR) technologies have enabled remote sensing-based studies of vegetation canopies by capturing three-dimensional structures, yielding information not available in two-dimensional images of the landscape pro- vided by traditional multi-spectral remote sensing platforms. However, the large volume data sets produced by airborne LiDAR instruments pose a significant computational challenge, requiring algorithms to identify and analyze patterns of interest buried within LiDAR point clouds in a computationally efficient manner, utilizing state-of-art computing infrastructure. We developed and applied a computationally efficient approach to analyze a large volume of LiDAR data and to characterize and map the vegetation canopy structures for 139,859 hectares (540 sq. miles) in the Great Smoky Mountains National Park. This study helps improve our understanding of the distribution of vegetation and animal habitats in this extremely diverse ecosystem.

Clustering↗

How Much Higher Can HTCondor Fly?

The HTCondor high throughput computing system is heavily used in the high energy physics (HEP) community as the batch system for several Worldwide LHC Computing Grid (WLCG) resources. Moreover, it is the backbone of GlidelnWMS, the pilot system used by the computing organization of the Compact Muon Solenoid (CMS) experiment. To prepare for LHC Run 2, we probed the scalability limits of new versions and configurations of HTCondor with a goal of reaching 200,000 simultaneous running jobs in a single internationally distributed dynamic pool.In this paper, we first describe how we created an opportunistic distributed testbed capable of exercising runs with 200,000 simultaneous jobs without impacting production. This testbed methodology is appropriate not only for scale testing HTCondor, but potentially for many other services. In addition to the test conditions and the testbed topology, we include the suggested configuration options used to obtain the scaling results, and describe some of the changes to HTCondor inspired by our testing that enabled sustained operations at scales well beyond previous limits.

Fajardo, E. M.↗

Reflections on the Shifting Experiences of Scientific Infrastructure

Infrastructure of all types is fundamental to modern work and life. Computing for scientific work, especially, extends from distributed local research sites, often at the edges of other major systems, outward into globally connected high-performance facilities and infrastructures. This commentary reviews longstanding research on the social characteristics of infrastructure. We reflect on social concerns that affect the ongoing development, use, and maintenance of a wide range of scientific computing and data resources. Reflecting on the social nature of infrastructure is timely for Computing in Science & Engineering readers, given continued emphasis on developing even more expansive platforms for data and artificial intelligence work in science (e.g., the United States’ Genesis Mission). We assert that, regardless of technological advances, the complex nature of scientific research and data will require continued understanding of longstanding and nascent social practices across varied communities. This is fundamentally necessary to build and sustain usable infrastructure or platforms that can productively advance scientific research.

Paine, Drew [Lawrence Berkeley National Laboratory↗

Residential Demand Side Aggregation of Privacy-Conscious Consumers

The increasing adoption of smart meters has led to growing concerns regarding privacy risks stemming from the high resolution measurements. This has given rise to privacy protection techniques that physically alter the consumer's energy load profile, masking private information by using localised devices, e.g. batteries or flexible loads. Meanwhile, there has also been increasing interest in aggregating the distributed energy resources (DERs) of residential consumers to provide services to the grid. In this paper, we propose an online distributed algorithm to aggregate the DERs of privacy-conscious consumers to provide services to the grid, whilst preserving their privacy. Results show that the optimisation solution from the distributed method converges to one close to the optimum computed using an ideal centralised solution method, balancing between grid service provision, consumer preferences and privacy protection. More importantly, the distributed method preserves consumer privacy, and does not require high-bandwidth two-way communications infrastructure.

ancillary services↗

Performance Characterization and Provenance of Distributed Task-based Workflows on HPC Platforms

Understanding performance and provenance of task-based workflows poses significant challenges, particularly in distributed configurations where resources are shared by multiple applications. Task-based workflow management systems further complicate performance predictability because of their dynamicity that subtly alters task execution order from run to run. In this paper we propose a layered characterization framework for performance and task provenance for Dask.distributed workflows running on high-performance computing (HPC) platforms. It collects data from jobs, the workflow management system, and the operating system to aid in understanding the performance of these workflows. Our approach encompasses three main contributions: first, an extension of Dask.distributed to capture high-fidelity task provenance using Mochi data services; second, the adaptation of the established HPC I/O characterization tool Darshan to gather high-fidelity I/O data, thereby enhancing the granularity of our analysis; and third, a framework to combine and process the collected data and provide helpful insights into performance characterization and reproducibility, alongside our lessons learned.

Dask↗

Use of glide-ins in CMS for production and analysis

With the evolution of various grid federations, the Condor glide-ins represent a key feature in providing a homogeneous pool of resources using late-binding technology. The CMS collaboration uses the glide-in based Workload Management System, glideinWMS, for production (ProdAgent) and distributed analysis (CRAB) of the data. The Condor glide-in daemons traverse to the worker nodes, submitted via Condor-G. Once activated, they preserve the Master-Worker relationships, with the worker first validating the execution environment on the worker node before pulling the jobs sequentially until the expiry of their lifetimes. The combination of late-binding and validation significantly reduces the overall failure rate visible to CMS physicists. We discuss the extensive use of the glideinWMS since the computing challenge, CCRC-08, in order to prepare for the forthcoming LHC data-taking period. The key features essential to the success of large-scale production and analysis on CMS resources across major grid federations, including EGEE, OSG and NorduGrid are outlined. Use of glide-ins via the CRAB server mechanism and ProdAgent, as well as first hand experience of using the next generation CREAM computing element within the CMS framework is discussed.

99 GENERAL AND MISCELLANEOUS↗

Leveraging the Cloud for Robust and Efficient Lunar Image Processing

The Lunar Mapping and Modeling Project (LMMP) is tasked to aggregate lunar data, from the Apollo era to the latest instruments on the LRO spacecraft, into a central repository accessible by scientists and the general public. A critical function of this task is to provide users with the best solution for browsing the vast amounts of imagery available. The image files LMMP manages range from a few gigabytes to hundreds of gigabytes in size with new data arriving every day. Despite this ever-increasing amount of data, LMMP must make the data readily available in a timely manner for users to view and analyze. This is accomplished by tiling large images into smaller images using Hadoop, a distributed computing software platform implementation of the MapReduce framework, running on a small cluster of machines locally. Additionally, the software is implemented to use Amazon's Elastic Compute Cloud (EC2) facility. We also developed a hybrid solution to serve images to users by leveraging cloud storage using Amazon's Simple Storage Service (S3) for public data while keeping private information on our own data servers. By using Cloud Computing, we improve upon our local solution by reducing the need to manage our own hardware and computing infrastructure, thereby reducing costs. Further, by using a hybrid of local and cloud storage, we are able to provide data to our users more efficiently and securely. 12 This paper examines the use of a distributed approach with Hadoop to tile images, an approach that provides significant improvements in image processing time, from hours to minutes. This paper describes the constraints imposed on the solution and the resulting techniques developed for the hybrid solution of a customized Hadoop infrastructure over local and cloud resources in managing this ever-growing data set. It examines the performance trade-offs of using the more plentiful resources of the cloud, such as those provided by S3, against the bandwidth limitations such use encounters with remote resources. As part of this discussion this paper will outline some of the technologies employed, the reasons for their selection, the resulting performance metrics and the direction the project is headed based upon the demonstrated capabilities thus far.

Cloud Computing↗

The AQUA-MER databases and aqueous speciation server: A web resource for multiscale modeling of mercury speciation

To assess the chemical reactivity, toxicity, and mobility of pollutants in the environment, knowledge of their species distributions is critical. Because their direct measurement is often infeasible, speciation modeling is widely adopted. Mercury (Hg) is a representative pollutant for which study of its speciation benefits from modeling. However, Hg speciation modeling is often hindered by a lack of reliable thermodynamic constants. Although computational chemistry (e.g., density functional theory [DFT]) can generate these constants, methods for directly coupling DFT and speciation modeling are not available. Here, we combine computational chemistry and continuum-scale modeling with curated online databases to ameliorate the problem of unreliable inputs to Hg speciation modeling. Our AQUA-MER databases and web server (https://aquamer.ornl.gov) provides direct speciation results by combining web-based interfaces to a speciation calculator, databases of thermodynamic constants, and a computational chemistry toolkit to estimate missing constants. Lastly, although Hg is presented as a concrete use case, AQUA-MER can also be readily applied to other elements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Dynamic Restoration Strategy for Distribution System Resilience Enhancement

In electric power distribution systems, distributed energy resources (DERs) can act as controllable power sources and support utility operators to minimize power outages after extreme weather events (e.g., hurricane, earthquake, wildfire) and thus help enhance the grid's resilience. Meanwhile, the influences of extreme events and the capabilities of DERs are dynamic and difficult to predict. Hence, the desired distribution system restoration strategy should be able to evolve according to real-time fault/dis-turbance information and the availability of DERs. In this paper, we propose a new dynamic restoration strategy for distribution systems to enhance system resilience against potential hazards. An efficient reconfiguration algorithm is developed to eliminate the use of integer variables to relieve the computational burden. Model predictive control is implemented to adjust the system topology and DER operation set points based on the updated fault information and DER forecasts. The effectiveness of the proposed restoration model in enhancing distribution system resilience is validated through an IEEE 123-bus test system. Simulation results also validate that the proposed restoration model can mitigate the occurrence of unexpected events and the fluctuations of DERs.

distributed energy resources (DERs)↗

FOX: A Fault Oblivious Extreme-Scale Execution Environment

The FOX project, funded under the ASCR X-stack I program, developed systems software and runtime libraries for a new approach to the data and work distribution for massively parallel, fault oblivious application execution. Our work was motivated by the premise that exascale computing systems will provide a thousand-fold increase in parallelism and a proportional increase in failure rate relative to today’s machines. To deliver the capability of exascale hardware, the systems software must provide the infrastructure to support existing applications while simultaneously enabling efficient execution of new programming models that naturally express dynamic, adaptive, irregular computation; coupled simulations; and massive data analysis in a highly unreliable hardware environment with billions of threads of execution. Our OS research has prototyped new methods to provide efficient resource sharing, synchronization, and protection in a many-core compute node. We have experimented with alternative task/dataflow programming models and shown scalability in some cases to hundreds of thousands of cores. Much of our software is in active development through open source projects. Concepts from FOX are being pursued in next generation exascale operating systems. Our OS work focused on adaptive, application tailored OS services optimized for multi → many core processors. We developed a new operating system NIX that supports role-based allocation of cores to processes which was released to open source. We contributed to the IBM FusedOS project, which promoted the concept of latency-optimized and throughput-optimized cores. We built a task queue library based on distributed, fault tolerant key-value store and identified scaling issues. A second fault tolerant task parallel library was developed, based on the Linda tuple space model, that used low level interconnect primitives for optimized communication. We designed fault tolerance mechanisms for task parallel computations employing work stealing for load balancing that scaled to the largest existing supercomputers. Finally, we implemented the Elastic Building Blocks runtime, a library to manage object-oriented distributed software components. To support the research, we won two INCITE awards for time on Intrepid (BG/P) and Mira (BG/Q). Much of our work has had impact in the OS and runtime community through the ASCR Exascale OS/R workshop and report, leading to the research agenda of the Exascale OS/R program. Our project was, however, also affected by attrition of multiple PIs. While the PIs continued to participate and offer guidance as time permitted, losing these key individuals was unfortunate both for the project and for the DOE HPC community.

97 MATHEMATICS AND COMPUTING↗

Software Architecture to Support the Evolution of the ISRU RESOLVE Engineering Breadboard Unit 2 (EBU2)

The In-Situ Resource Utilization (ISRU) Regolith & Environmental Science and Oxygen & Lunar Volatiles Extraction (RESOLVE) software provides operation of the physical plant from a remote location with a high-level interface that can access and control the data from external software applications of other subsystems. This software allows autonomous control over the entire system with manual computer control of individual system/process components. It gives non-programmer operators the capability to easily modify the high-level autonomous sequencing while the software is in operation, as well as the ability to modify the low-level, file-based sequences prior to the system operation. Local automated control in a distributed system is also enabled where component control is maintained during the loss of network connectivity with the remote workstation. This innovation also minimizes network traffic. The software architecture commands and controls the latest generation of RESOLVE processes used to obtain, process, and quantify lunar regolith. The system is grouped into six sub-processes: Drill, Crush, Reactor, Lunar Water Resource Demonstration (LWRD), Regolith Volatiles Characterization (RVC) (see example), and Regolith Oxygen Extraction (ROE). Some processes are independent, some are dependent on other processes, and some are independent but run concurrently with other processes. The first goal is to analyze the volatiles emanating from lunar regolith, such as water, carbon monoxide, carbon dioxide, ammonia, hydrogen, and others. This is done by heating the soil and analyzing and capturing the volatilized product. The second goal is to produce water by reducing the soil at high temperatures with hydrogen. This is done by raising the reactor temperature in the range of 800 to 900 C, causing the reaction to progress by adding hydrogen, and then capturing the water product in a desiccant bed. The software needs to run the entire unit and all sub-processes; however, throughout testing, many variables and parameters need to be changed as more is learned about the system operation. The Master Events Controller (MEC) is run on a standard laptop PC using Windows XP. This PC runs in parallel to another laptop that monitors the GC, and a third PC that monitors the drilling/ crushing operation. These three PCs interface to the process through a CompactRIO, OPC Servers, and modems.

Moss, Thomas↗

Day-ahead continuous double auction-based peer-to-peer energy trading platform incorporating trading losses and network utilisation fee

Integration of distributed energy resources, such as photovoltaic solar (PV), introduces new opportunities to establish local energy market frameworks to improve renewable energy utilisation in residential sectors. Such peer-to-peer (P2P) energy trading refers to a local market structure where customers (and prosumers) interact to share excess PV generation to enhance the individual and community social welfare. In this work, a day-ahead continuous double auction (CDA)-based P2P market structure considering network losses and network utilisation fees was designed. Day-ahead PV energy is modelled using fractional integral polynomials and the output is forecasted using an autoregressive integrated moving average model for each market interval. Based on the customer load and excess PV energy, the CDA market is cleared using a bid/ask matching mechanism. The performance of the P2P market was evaluated by computing different welfare metrics while analysing the effect of network constraints. The results show that the designed CDA-based P2P market structure increases the social welfare of all participants by an average of 17.75% compared to the baseline for the presented cases. Moreover, the impact of the forecasting error between the day-ahead and real-time market was also quantified.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Framework for Large-scale Implementation of Wholesale-Retail Transactive Control Mechanism

Transactive energy is a control technique that uses market mechanisms to achieve desired control objectives. Several simulation studies and field demonstrations were carried out in recent years, but all focused on small scale systems and purpose-built simplified models which are not always capable of pointing out all the advantages and shortcomings of transac- tive energy methods. This work describes a co-simulation framework built on a hierarchical control architecture that allows for conducting studies of the impacts of a very large-scale deployment of transactive energy. The hierarchical transactive control architecture adopted in this work helps with alleviating computation and communication burden to facilitate a more effective large scale real-time market operation among device level resources and the system level operators. The co-simulation framework is evaluated an integrated power sys- tem model of unprecedented scale composed of the Western Electricity Coordination Council (WECC) transmission system with tens of thousands of distribution systems deployed with flexible device-level distributed energy resources (DERs) using off-the-shelf simulators.

Transactive energy, market-based controls, DER int↗

High-energy synchrotron X-ray multimodal computed tomography: enabling multiscale materials characterization at NSLS-II

We report the commissioning of a multimodal computed tomography experimental setup at the 28-ID-2 (XPD) beamline of the National Synchrotron Light Source II. This high-energy (>60 keV) resource features a tunable X-ray beam size ranging from several millimetres to a few micrometres and enables comprehensive characterization of high-Z materials—an essential capability for nuclear and advanced materials research. It provides four complementary computed tomography modalities: X-ray absorption, X-ray fluorescence, X-ray diffraction, and pair distribution function tomography. A case study using a custom-made heterogeneous sample demonstrates these abilities to simultaneously capture atomic, elemental, and morphological information. This unique combination of imaging, structural, and chemical sensitive methods provides a holistic approach to study complex materials with amorphous and crystalline systems across multiple length scales.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Federated Learning for Efficient Condition Monitoring and Anomaly Detection in Industrial Cyber-Physical Systems

Detecting and localizing anomalies in cyber-physical systems (CPS) has become increasingly challenging as systems grow in complexity, particularly due to varying sensor reliability and node failures in distributed environments. While federated learning (FL) offers a foundation for distributed model training, existing approaches lack mechanisms to handle these CPS-specific challenges. This paper presents an enhanced FL framework that introduces three key innovations: adaptive model aggregation based on sensor reliability, dynamic node selection for resource optimization, and Weibull-based checkpointing for fault tolerance. Our framework enables reliable condition monitoring while addressing the computational and reliability challenges of industrial CPS deployments. Experiments on NASA Bearing and Hydraulic System Datasets demonstrate superior performance over state-of-the-art FL methods, achieving 99.5% AUC-ROC in anomaly detection and maintaining accuracy under node failures. Statistical validation using Mann-Whitney (U) test confirms significant improvements (p < 0.05) in both detection accuracy and computational efficiency across diverse operational scenarios.1

Marfo, William [University of Texas at El Paso,Dep↗

Penetration Through Slots in Overmoded Cavities

A resonant cavity undergoes three distinct behaviors with increasing frequency: 1) fundamental modes, localized in frequency with well defined modal distribution; 2) undermoded region, where modes are still separated, but are sufficiently perturbed by small imperfections that their spectral positions (and distributions) are statistical in nature; and 3) overmoded region, where modes overlap, field distributions follow stochastic distributions, and the slot acts as if in free space. Understanding the penetration through slots in the overmoded region is of great interest, and is the focus of this article. Since full-wave solvers may not be able to provide a timely answer for very high frequencies due to a lack of memory and/or computation resources, we develop bounding methods to estimate worst-case average and maximum fields within the cavity. Finally, after discussing the bounding formulation, we compare its results to full-wave simulations at the first, second, and third resonance supported by the slot in the case of a cylindrical cavity. Note that the bounding formulation indicates that results are nearly independent of cavity shape: only the cavity volume, frequency, and cavity quality factor affect the overmoded region, making this formulation a powerful tool to assess electromagnetic interference and electromagnetic compatibility effects within cavities.

overmoded cavity↗

MIRACL Co-Simulation Platform Lab assets and tools integration

Pacific Northwest National Laboratory's (PNNL) co-simulation platform (CSP) for the Microgrids, Infrastructure Resilience, and Advanced Controls Launchpad (MIRACL) project, also known as MIRACL-CSP, is a functional layer designed and developed to oversee the operational exchanges of data at the application level to and from different resources residing on the MIRACL Data Hub shared platform. MIRACL-CSP allows virtual interactions between various data hub resources during co-simulation runtime.

24 POWER TRANSMISSION AND DISTRIBUTION↗