Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time

The Vera C. Rubin Observatory is preparing to execute the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently the final phase of construction is under way in the Chilean Andes, with the Observatory’s ten-year science mission scheduled to begin in 2025. Rubin’s 8.4-meter telescope will nightly scan the southern hemisphere collecting imagery in the wavelength range 320–1050 nm covering the entire observable sky every 4 nights using a 3.2 gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties. In this article we present an overview of the system currently being constructed to perform data distribution as well as the annual campaigns which reprocess the entire image dataset collected since the beginning of the survey. These processing campaigns will utilize computing and storage resources provided by three Rubin data facilities (one in the US and two in Europe). Each year a Data Release will be produced and disseminated to science collaborations for use in studies comprising four main science pillars: probing dark matter and dark energy, taking inventory of solar system objects, exploring the transient optical sky and mapping the Milky Way. Also presented is the method by which we leverage some of the common tools and best practices used for management of large-scale distributed data processing projects in the high energy physics and astronomy communities. We also demonstrate how these tools and practices are utilized within the Rubin project in order to overcome the specific challenges faced by the Observatory.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Data-Driven Mean-Corrected Recursive Estimation-Based Optimal DER Dispatch for Distribution System Voltage Control

Recent advances in smart inverters offer opportunities to mitigate adverse grid impacts caused by high penetrations of distributed photovoltaics (PV) in distribution grids, such as voltage violations. Here, this paper proposes a novel measurement-driven optimal power flow (OPF)-based distributed energy resource management system (DERMS) voltage regulation via recursive sensitivity estimation informed coordinated control of distributed PV inverters. The proposed approach leverages available grid and controllable DER measurements, eliminating reliance on system model information while being adaptive and robust to volatile operating conditions. A mean-corrected recursive ridge regression (MCRRR) algorithm is proposed for sensitivity estimation, continuously refining the sensitivity model through a closed-form solution. It effectively manages varying grid operating conditions, such as changes in power injections and topology reconfiguration, to facilitate a time-varying update of the Load Sensitivity Factors (LSF). The proposed approach is formulated as a linear programming (LP) problem and is thus scalable to larger-scale distribution systems. Its effectiveness and efficiency are demonstrated on a realistic distribution feeder with high PV penetrations in Southern California, USA.

14 SOLAR ENERGY↗
DOE OSTI2025

ArborX 2.0

ArborX library tackles a problem of efficiently finding geometric objects that are close in space. Variations of this problem, such as finding the nearest neighbors of a point, or finding all objects within a certain distance, are inherent components of applications in many fields. The data may be large so that solving the problem efficiently may require significant computational resources, such as multiple processors or accelerators such as general purpose GPUs. ArborX' main advantage in its ability to solve large problems efficiently utilizing a combination of distributed and on-node parallelism. ArborX can be run efficiently on a wide variety of hardware, including GPUs from different vendors, which distinguishes it from other available libraries which typically choose only few of these. The other advantage is that it supports both types of user problems: spatial problems (useful for intersections and finding objects within certain distance), and nearest neighbor problems. ArborX also supports flexible interface in its interaction with a user. Particularly, it allows a user to call user's own function on a positive match, a functionality not rarely available in other libraries. ArborX implements construction and traversal algorithms using efficient tree structures, such as bounding volume hierarchy (BVH). At its core, ArborX uses linear BVH for its low construction cost and sufficient quality. ArborX implements both spatial and nearest-neighbor traversal algorithms. ArborX also provides several clustering algorithms (minimum spanning tree, DBSCAN, HDBSCAN*), interpolation using minimum least squares and ray tracing. ArborX is written using C++, and is parallelized using the message passing interface (MPI) for the distributed communication, and the Kokkos library for on-node parallelism. This approach allows ArborX to be run on a wide variety of hardware, from common laptops and desktops to supercomputers while using the same codebase.

Prokopenko, Andrey [Oak Ridge National Laboratory ↗
DOE OSTI2026

Maximizing efficiency of dataset compression for machine learning potentials with information theory

Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Use of Digital Real-Time Simulation and Optimization to Identify Maximum Real Power Injection on Banshee Distribution Network

This study investigates the hosting capacity of the Banshee Distribution Network by optimizing the real power injection at carefully selected Distributed Energy Resource (DER) locations. The analysis is conducted within the framework of power system operational constraints, including bus voltage ranges, thermal line ratings, and transformer loading limits. A Python-based Genetic Algorithm (GA), implemented using the PyGAD library, is employed to iteratively identify the optimal power injection configuration that maximizes network utilization while preserving system reliability. The methodology integrates a real-time simulation environment using the Real-Time Digital Simulator (RTDS), allowing high-fidelity evaluation of power flow and voltage behavior under each proposed injection scenario. By coupling the optimization algorithm with real-time simulation feedback, this approach ensures that both static and dynamic constraints are enforced during the evaluation process. The GA leverages evolutionary operators such as selection, crossover, and mutation to navigate the nonlinear search space efficiently. The results of the study delineate the feasible hosting capacity at three targeted buses, reflecting maximum real power levels that can be injected without causing voltage violations, transformer overloading, or line congestion. These findings provide a decision- support tool for distribution planners and utilities aiming to integrate higher penetrations of DERs in existing infrastructure. Additionally, the work lays the foundation for extending such optimization techniques to multi-objective formulations, including economic dispatch and reactive power coordination, in future studies.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Use of Digital Real-Time Simulation and Optimization to Identify Maximum Real Power Injection on the Banshee Distribution Network: Preprint

This study investigates the hosting capacity of the Banshee Distribution Network by optimizing the real power injection at carefully selected Distributed Energy Resource (DER) locations. The analysis is conducted within the framework of power system operational constraints, including bus voltage ranges, thermal line ratings, and transformer loading limits. A Python-based Genetic Algorithm (GA), implemented using the PyGAD library, is employed to iteratively identify the optimal power injection configuration that maximizes network utilization while preserving system reliability. The methodology integrates a real-time simulation environment using the Real-Time Digital Simulator (RTDS), allowing high-fidelity evaluation of power flow and voltage behavior under each proposed injection scenario. By coupling the optimization algorithm with real-time simulation feedback, this approach ensures that both static and dynamic constraints are enforced during the evaluation process. The GA leverages evolutionary operators such as selection, crossover, and mutation to navigate the nonlinear search space efficiently. The results of the study delineate the feasible hosting capacity at three targeted buses, reflecting maximum real power levels that can be injected without causing voltage violations, transformer overloading, or line congestion. These findings provide a decision- support tool for distribution planners and utilities aiming to integrate higher penetrations of DERs in existing infrastructure. Additionally, the work lays the foundation for extending such optimization techniques to multi-objective formulations, including economic dispatch and reactive power coordination, in future studies.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

A Data-Driven Voltage Control Strategy for Distribution Grids With Distributed Energy Resources

Traditionally, distribution system control approaches have been model-based. The deployment of advanced metering infrastructure has provided electric utilities with the capability of data-driven control with real-time measurements. The shift from model-based to data-driven control represents a significant advancement in the management of distribution systems, offering a more adaptive approach to system control because of the ability to dynamically adapt to changing conditions without the need for system modeling. Here, in this paper, a behavioral data-driven control method is developed to provide voltage regulation to an actual distribution system by controlling the legacy devices and distributed energy resource (DER) assets. The studied distribution system has a load tap changer and three capacitor banks as the legacy devices and photovoltaic systems as the DERs. The performance of the proposed control algorithm is validated using a laboratory test bed setup considering multiple scenarios. The results show that the proposed control achieved 99% voltage regulation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Physics-Driven Construction of Compact Primitive Gaussian Density Fitting Basis Sets

We present a model-assisted density fitting (MADF) basis set generator, an algorithm for generating primitive atomic Gaussian density fitting (DF) basis sets (DFBSs) from a contracted Gaussian orbital basis set (OBS). The MADF algorithm produces DFBSs suitable for accurate robust DF approximation of 2-particle interactions in mean-field and correlated electronic structures. The algorithm is designed to (a) saturate the OBS product space by a large regularized set of primitive solid-harmonic Gaussian shells with nonuniform distribution of exponents, followed by (b) pruning of the shells according to their contributions to the 2- body energy of a correlated atomic ensemble. Building the DFBS generator model almost exclusively on mathematical and physical principles allows one to limit the number of parameters that control the density fitting error to three, with a single set of parameters sufficient for computations with all basis cardinal numbers, with and without correlation of core electrons, with and without scalar and spin-dependent relativistic effects, spanning almost all of the Periodic Table. Performance assessment included basis sets up to quadruple-ζ quality from several major basis set families, using molecules composed of main-group, d-block, and f-block elements. The resulting DF errors in Hartree−Fock and second-order MP2 energies (with relativistic all-electron treatments, when appropriate) were on the order of 20 and 10 μE h per electron, respectively.

Approximation↗
DOE OSTI2024

ARM shortwave spectrometers to study the clear-cloud transition zone and mixing processes

The proposed research is a collaborative effort between NASA/Goddard Space Flight Center and the Hebrew University of Jerusalem. While the NASA team focused on analyzing ground-based hyper-spectral radiance observations to understand cloud edge properties and their connection to mixing processes, the Hebrew University team tackled the problem through cloud modeling activities. By approximating the shortwave spectra in the cloud-clear transition zone as a linear combination of purely clear and purely cloudy spectra we can characterize the variations of cloud optical thickness and cloud droplet effective radius in the transition zone. When applying this method to the measurements of a ground-based shortwave spectroradiometer at the ARM’s SGP site, representing continental conditions, and MAGIC field campaign between Log Angeles, California and Honolulu, Hawaii, representing maritime scenarios, we found that cloud optical depth consistently decreases in both cases, but droplet size decreases much more substantially for the continental regime, suggesting different mixing processes for the continental and maritime conditions. The investigation and measurements of radiation clouds were coupled with a unique cloud modeling. A novel spectral bin microphysics was developed and implemented to the System of Atmospheric Modeling (SAM). In order to resolve cloud transition zones with high spatial gradients of microphysical variables a unique high resolution (10 m) was used in simulations. The model calculates droplet size distributions in each grid point. The model output was transferred to the NASA/GSFC team for utilization in radiative calculations and testing of both radiative algorithm and model representation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Reactor System Facility Modification to Detect Compromised Human Machine Interfaces

This study focuses on a multi-layered Industrial Control System (ICS)/Operational Technology (OT) security architecture to aid in the discovery and mitigation of compromised Human Machine Interface (HMI)/Instrumentation & Control (I&C) based systems for modifying a prototypical reactor condition test facility called the Flowing Autoclave System (FAS) at Idaho National Laboratory (INL). This is achieved through a three-layered combination of network security solutions, hash-based algorithms, and blockchain technologies. Hash algorithms are mathematical functions used to generate a predetermined set of fixed-length values. They are widely used in computer security to verify the integrity of system information and data, both on a local network and the wider internet. Even small amounts of unauthorized system modification will cause the hash algorithm to output a set of characters that deviate significantly from its original value. Assisting secure hash functions, blockchain technology is a secure and distributed technology used to provide an immutable set of records replicated on all devices within a decentralized network. Blockchain offers a cost-effective solution to detect system compromise by providing a traceable breadcrumb trail of all network activity and data modification happening on a system. If both are used in conjunction with network monitoring tools, the integration of this three-pronged approach can become an asset in detecting suspected system compromises before any real damage can occur.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

DMTN-260: Failure Modes and Error Handling for Prompt Processing

The Prompt Processing system will be responsible for processing roughly a thousand visits per night, and distributing the results in near real time, for at least ten years of Rubin Observatory operations. As such, it must be highly robust to algorithmic, network, and infrastructure failures, ranging from momentary glitches to extended downtimes. DMTN-219 introduced the initial design for the Prompt Processing framework; this document expands on the design to address expected failure modes and recovery strategies for each.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Polarized and unpolarized gluon PDFs: Generative machine learning applications for lattice QCD matrix elements at short distance and large momentum

Lattice quantum chromodynamics (QCD) calculations share a defining challenge by requiring a small finite range of spatial separation z between quark/gluon bilinears for controllable power corrections in the perturbative QCD factorization, and a large hadron boost p z for a successful determination of collinear parton distribution functions (PDFs). However, these two requirements make the determination of PDFs from lattice data very challenging. We present the application of generative machine learning algorithms to estimate the polarized and unpolarized gluon correlation functions utilizing short-distance data and extending the correlation up to z p z ≲ 14 , surpassing the current capabilities of lattice QCD calculations. We train physics-informed machine learning algorithms to learn from the short-distance correlation at z ≲ 0.36 fm and take the limit, p z → ∞ , thereby minimizing possible contamination from the higher-twist effects for a successful reconstruction of the polarized gluon PDF. We also expose the bias and problems with underestimating uncertainties associated with the use of model-dependent and overly constrained functional forms, such as x α ( 1 − x ) β and its variants to extract PDFs from the lattice data. We propose the use of generative machine learning algorithms to mitigate these issues and present our determination of the polarized and unpolarized gluon PDFs in the nucleon. Published by the American Physical Society 2025

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Short-Term Probabilistic Solar Forecasting via Reinforcement Learning over ECMWF

In this paper, we present an innovative reinforcement learning approach for short-term solar forecasting, leveraging data from the European Centre for Medium-Range Weather Forecasts (ECMWF). The methodology begins with the application of the System Advisor Model (SAM) to transform various ECMWF numerical weather prediction members into predictive photovoltaic power generation. To enhance the precision of deterministic forecasting, we introduce a dynamic model selection algorithm based on Q-learning. This algorithm dynamically identifies and utilizes the most accurate ensemble member for forecasting purposes. Furthermore, we employ a support vector regression surrogate model with a Gaussian distribution to generate probabilistic forecasts, providing a holistic view of solar energy generation uncertainty. To expedite the training process and make it more practical for real-world applications, we integrate a rolling update workflow. This innovative workflow reduces the training period from months to a mere 19 days, making our method highly efficient. Numerical results of the case study show that in comparison to benchmark models, the proposed method improves the deterministic and probabilistic solar forecasting accuracy by up to 40.84% and 48.42%, respectively.

ensemble forecasting↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom Probe Tomography (APT) is a powerful technique for visualizing the atomic-scale distribution of solutes in materials, but quantitative cluster analysis of APT datasets remains a challenge due to the need for subjective parameter selection in clustering algorithms. While distance-based and density-based methods such as HDBSCAN are widely used, their performance is highly sensitive to user-defined parameters, which undermines reproducibility and accuracy. This study proposes an image-based, deep learning-aided workflow for automating parameter selection and cluster detection in APT data analysis. By projecting 3D APT point clouds onto 2D planes, we leverage pretrained convolutional neural networks (ConvNeXt-Tiny and ResNet-50) through transfer learning to predict the number of clusters present in synthetic datasets. The output is used to guide K-means clustering and estimate HDBSCAN parameters, specifically minimum cluster size and minimum sample points. This approach reduces reliance on manual parameter tuning, improving consistency and scalability. The methodology demonstrates the feasibility of using image-based deep learning for interpreting complex spatial patterns in APT data, enabling faster and more objective analysis. The complete workflow and code are made publicly available to support reproducibility and future research.

Density-based clustering↗
DOE OSTI2025

A matheuristic for design and dispatch of a utility-connected distributed energy system

Modeling distributed power generation systems often requires complicated mathematical expressions that present challenges for commercial optimization solvers. Here, this paper presents a matheuristic to solve a mixed-integer optimization model that informs decisions regarding the design and dispatch of a utility-connected microgrid. We deploy a genetic algorithm to search the system design space and a linear program to solve the economic dispatch problem. The model is a component of a web tool that requires solutions within a few minutes. Our method yields objective function values within 5% of an exogenously produced optimal in fewer than 30 seconds for 90% of our test cases compared to only 10% of our test cases by a traditional optimization solver in the same amount of time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

CIRCLEZ : Reliable photometric redshifts for active galactic nuclei computed solely using photometry from Legacy Survey Imaging for DESI

Photometric redshifts for galaxies hosting an accreting supermassive black hole in their center, known as active galactic nuclei (AGNs), are notoriously challenging. At present, they are most optimally computed via spectral energy distribution (SED) fittings, assuming that deep photometry for many wavelengths is available. However, for AGNs detected from all-sky surveys, the photometry is limited and provided by a range of instruments and studies. This makes the task of homogenizing the data challenging, presenting a dramatic drawback for the millions of AGNs that wide surveys such as SRG/eROSITA are poised to detect. This work aims to compute reliable photometric redshifts for X-ray-detected AGNs using only one dataset that covers a large area: the tenth data release of the Imaging Legacy Survey (LS10) for DESI. LS10 provides deep grizW1-W4 forced photometry within various apertures over the footprint of the eROSITA-DE survey, which avoids issues related to the cross-calibration of surveys. We present the results from CIRCLEZ, a machine-learning algorithm based on a fully connected neural network. CIRCLEZ is built on a training sample of 14 000 X-ray-detected AGNs and utilizes multi-aperture photometry, mapping the light distribution of the sources. The accuracy (σNMAD) and the fraction of outliers (η) reached in a test sample of 2913 AGNs are equal to 0.067 and 11.6%, respectively. The results are comparable to (or even better than) what was previously obtained for the same field, but with much less effort in this instance. We further tested the stability of the results by computing the photometric redshifts for the sources detected in CSC2 and Chandra-COSMOS Legacy, reaching a comparable accuracy as in eFEDS when limiting the magnitude of the counterparts to the depth of LS10. The method can be applied to fainter samples of AGNs using deeper optical data from future surveys (for example, LSST, Euclid), granting LS10-like information on the light distribution beyond the morphological type. Along with this paper, we have released an updated version of the photometric redshifts (including errors and probability distribution functions) for eROSITA/eFEDS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

A Class of Sparse Johnson–Lindenstrauss Transforms and Analysis of their Extreme Singular Values

The Johnson–Lindenstrauss (JL) lemma is a powerful tool for dimensionality reduction in modern algorithm design. The lemma states that any set of high-dimensional points in a Euclidean space can be projected into lower dimensions while approximately preserving pairwise Euclidean distances. Random matrices satisfying this lemma are called JL transforms (JLTs). Inspired by existing $s$-hashing JLTs with exactly $s$ nonzero elements on each column, the present work introduces an ensemble of sparse matrices encompassing so-called $s$-hashing-like matrices whose expected number of nonzero elements on each column is $s$. The independence of the sub-Gaussian entries of these matrices and the knowledge of their exact distribution play an important role in their analyses. Using properties of independent sub-Gaussian random variables, these matrices are demonstrated to be JLTs, and their smallest nontrivial singular values and largest singular values are estimated nonasymptotically using a technique from geometric functional analysis. As the dimensions of the matrix grow to infinity, these singular values are proved to converge almost surely to fixed quantities (by using the universal Bai–Yin law) and in distribution to the Gaussian orthogonal ensemble Tracy–Widom law after proper rescalings. Understanding the behaviors of extreme singular values is important in general because they are often used to define a measure of stability of matrix algorithms. For example, JLTs were recently used in derivative-free optimization algorithmic frameworks to select random subspaces in which are constructed random models or poll directions to achieve scalability, and hence estimating their smallest singular value in particular helps determine the dimension of these subspaces.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.

14 SOLAR ENERGY↗