Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Conditional distribution estimation of building characteristics with diffusion models for urban energy modeling

Understanding current energy consumption behavior in communities is critical for informing future energy use decisions and enabling efficient energy management. Urban energy models, which are used to simulate these energy use patterns, require large datasets with detailed building characteristics for accurate outcomes. However, such detailed characteristics at the individual building level are often unknown and costly to acquire, or unavailable. Through this work, we propose using a generative modeling approach to generate realistic building attributes to fill in the data gaps and finally provide complete characteristics as inputs to energy models. Our model learns complex, building-level patterns from training on a large-scale residential building stock model containing 2.2 million buildings. We employ a tabular diffusion-based framework that is designed to handle heterogeneous (discrete and continuous) features in tabular building data, such as occupancy, floor area, heating, cooling, and other equipment details. We develop a capability for conditional diffusion, enabling the imputation of missing building characteristics conditioned on known attributes. We conduct a comprehensive validation of our conditional diffusion model, firstly by comparing the generated conditional distributions against the underlying data distribution, and secondly, by performing a case study for a Baltimore residential region, showing the practical utility of our approach. Our work is one of the first to demonstrate the potential of generative modeling to accelerate building energy modeling workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

An Analysis Workflow-Aware Storage System for Multi-Core Active Flash Arrays

Here, the need for novel data analysis is urgent in the face of a data deluge from modern applications. Traditional approaches to data analysis incur significant data movement costs, moving data back and forth between the storage system and the processor. Emerging Active Flash devices enable processing on the flash, where the data already resides. An array of such Active Flash devices allows us to revisit how analysis workflows interact with storage systems. By seamlessly blending together the flash storage and data analysis, we create an analysis workflow-aware storage system, AnalyzeThis. Our guiding principle is that analysis-awareness be deeply ingrained in each and every layer of the storage, elevating data analyses as first-class citizens, and transforming AnalyzeThis into a potent analytics-aware appliance. To evaluate the AnalyzeThis system, we have adopted both emulation and simulation approaches. In particular, we have evaluated AnalyzeThis by implementing the AnalyzeThis storage system on top of the Active Flash Array's emulation platform. We have also implemented an event-driven AnalyzeThis simulator, called AnalyzeThisSim, which allows us to address the limitations of the emulation platform, e.g., performance impact of using multi-core SSDs. The results from our emulation and simulation platforms indicate that AnalyzeThis is a viable approach for expediting workflow execution and minimizing data movement.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Electrifying High-Efficiency Future Communities: Impact on Energy, Emissions, and Grid

To combat climate change and meet decarbonization goals, the building sector is improving energy efficiency and electrifying end uses to reduce carbon emissions from fossil fuels. All-electric buildings are becoming a trend among new constructions, introducing opportunities for decarbonization but also technical challenges and research gaps. For instance, further investigation is needed to understand how the adoption of energy efficiency measures (EEMs) and distributed energy resources (DERs) in all-electric communities would affect energy consumption, carbon emissions, and grid planning. This paper presents a case study of a mixed-use, all-electric community located in Denver, Colorado. We use URBANopt TM , a physics-based urban energy modeling platform to model the community and then evaluate the impact of EEMs and DERs (i.e., photovoltaics [PV], electric vehicles [EVs], and batteries) on the community's energy usage, carbon emissions, and peak demand. The results show that adding EEMs and PV led to both energy consumption and carbon emissions reductions across all building types. However, we saw fairly limited impact of EEMs and PV on buildings' peak demand in our case. Additionally, due to overnight EV charging activities and higher grid carbon intensity at night, the carbon emissions in multifamily buildings have a noticeable increase compared to scenarios without vehicles. Finally, the addition of batteries helped reduce peak demand by 11%-29%. The modeling workflow and evaluation methods can be applied to similar communities to evaluate their performance and the effect of integrating EEMs and DERs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

High-throughput micro-scale bandgap mapping for perovskite-inspired materials with complex composition space

Abstract To realize the full promise of high-throughput experimental workflows, the rate of sample synthesis must be matched by that of characterization. Of growing interest are contactless optical techniques that can rapidly measure material homogeneity and properties. Here, we present a hyperspectral imaging method to measure local optical bandgap distributions within samples, utilizing spatially-resolved reflectance spectra coupled with automated data analysis. We collect approximately one million optical bandgap data across the compositional space of Cs 3 (Bi x Sb 1-x ) 2 (Br y I 1-y ) 9 perovskite-inspired materials. Our results show non-monotonic bandgap variations (i.e., bandgap bowing) along six composition gradient sequences, in addition to identifying samples with multiple bandgaps in statistics. High-throughput transient absorption spectroscopy reveals that within these compositions, the depletion of the ground state carriers to excited states occurred at discrete energy levels with independent carrier dynamics, consistent with the bandgap observation and indicative of phase separation. This work demonstrates the potential for rapid optical measurements to assess material quality and homogeneity in a high-throughput experimental setting, supporting screening and recipe optimization of optoelectronic material candidates with desired carrier dynamics and optical properties.

Science & Technology - Other Topics↗

From Reproducible Edge–Cloud Experimentation to Real-World Practice: The E2Clab Experience

Reproducibility is already difficult in distributed systems; on the computing continuum, it becomes substantially harder. Applications that span sensing devices, edge and fog resources, and cloud platforms must be evaluated across heterogeneous hardware, variable network conditions, cross-layer orchestration decisions, and long-running workflow lifecycles. We use E2Clab as a case study to examine these challenges and their implications for experimental methodology. We explain why reproducible experimentation is harder on the continuum, then revisit E2Clab as an initial response based on explicit modeling of infrastructure, workflow lifecycle, and artifacts. Lastly, we discuss how its evolution toward more realistic application settings can be understood through the lens of Translational Computer Science. We argue that reproducible continuum experimentation requires methods that are rigorous enough for research while remaining adaptable to real-world practice.

42 ENGINEERING↗

Urban Deployment Model (UDM) v1.0

The software, called the Urban Deployment Model (UDM), is an integrated set of tools for the simulation and analysis of static and mobile radiation detection systems in complex urban environments. The UDM includes a graphical user interface through which the workflow associated with setting up and executing a simulation may be established. The modular toolsets combine aspects of radiation transport, navigation, graph theory, optimization, and visualization to allow the complete end-to-end simulation of large scale, highly-distributed sensor networks.

Abgrall, Nicolas↗

LandCast Mosaic: Reconstructing Global Population Distributions, 1975-2025

LandCast Mosaic (LCM) provides a global, high-resolution gridded population dataset spanning 1975–2025, representing annual, scenario-consistent estimates of daytime, nighttime, and ambient population distributions. LCM builds on the 2025 LandScan Mosaic (LSM) population data by backcasting to earlier years using historical changes in built-surface area derived from the Global Human Settlement Layer (GHSL) and authoritative population counts from international datasets. The workflow scales 2025 building-informed gridded population estimates according to observed changes in built surface, applies linear interpolation for intermediate years, and normalizes estimates to match administrative- and country-level totals. The resulting dataset offers consistent, globally gridded population estimates over fifty years, suitable for temporal analyses of population dynamics, disaster risk modeling, and urban planning applications.

97 MATHEMATICS AND COMPUTING↗

Pypromice: A Python Package for Processing Automated Weather Station Data

The pypromice Python package is for processing and handling observation datasets from automated weather stations (AWS). It is primarily aimed at users of AWS data from the Geological Survey of Denmark and Greenland (GEUS), which collects and distributes in situ weather station observations to the cryospheric science research community. Functionality in pypromice is primarily handled using two key open-source Python packages, xarray (Hoyer & Hamman, 2017) and pandas (The pandas development team, 2020). A defined processing workflow is included in pypromice for transforming original AWS observations (Level 0, L0) to a usable, CF-convention-compliant dataset (Level 3, L3) (Figure 1). Intermediary processing levels (L1,L2) refer to key stages in the workflow, namely the conversion of variables to physical measurements and variable filtering (L1), cross-variable corrections and user-defined data flagging and fixing (L2), and derived variables (L3). Information regarding the station configuration is needed to perform the processing, such as instrument calibration coefficients and station type (one-boom tripod or two-boom mast station design, for example), which are held in a toml configuration file. Two example configuration files are provided with pypromice , which are also used in the package’s unit tests. More detailed documentation of the AWS design, instrumentation, and processing steps are described in Fausto et al. (2021).

pypromice↗

Integrated laser ablation-dropletProbe-mass spectrometry for absolute drug quantitation, metabolite detection, and distribution in tissue

Rationale: Spatially resolved and accurate quantitation of drug-related compounds in tissue is a much-needed capability in drug discovery research. In this work, application of an integrated laser ablation-dropletProbe-mass spectrometry surface sampling system (LADP-MS) is reported, which achieved absolute quantitation of propranolol measured from <500 × 500 μm thin tissue samples. Methods: Mouse liver and kidney thin tissue sections were coated with parylene C and analyzed for propranolol by a laser ablation/liquid extraction workflow. Non-coated adjacent sections were microdissected for validation and processed using standard bulk tissue extraction protocols. High-performance liquid chromatography with positive ion mode electrospray ionization tandem mass spectrometry was applied to detect the drug and its metabolites. Results: Absolute propranolol concentration in ~500 × 500 μm tissue regions measured by the two methods agreed within ±8% and had a relative standard deviation within ±17%. Quantitation down to ~400 × 400 μm tissue regions was shown, and this resolution was also used for automated mapping of propranolol and phase II hydroxypropranolol glucuronide metabolites in kidney tissue. Conclusions: This study exemplifies the capabilities of integrated laser ablation-dropletProbe-mass spectrometry (LADP-MS) for high resolution absolute drug quantitation analysis of thin tissue sections. This capability will be valuable for applications needing to quantitatively understand the spatial distribution of small molecules in tissue.

47 OTHER INSTRUMENTATION↗

Direct Nanoscale Characterization of Deep Levels in AgCuInGaSe 2 Using Electron Energy–Loss Spectroscopy in the Scanning Transmission Electron Microscope

A new experimental framework for the characterization of defects in semiconductors is demonstrated. Through the direct, energy-resolved correlation of three analytical techniques spanning six orders of magnitude in spatial resolution, a critical mid-bandgap electronic trap level (E V + 0.56 eV) within Ag 0.2 Cu 0.8 In 1–x Ga x Se 2 is traced to its nanoscale physical location and chemical source. This is achieved through a stepwise, site-specific correlated characterization workflow consisting of device-scale (≈1 mm 2 ) deep level transient spectroscopy (DLTS) to survey the traps present, scanning probe–based DLTS (scanning-DLTS) for mesoscale-resolved (hundreds of nanometers) mapping of the target trap state’s spatial distribution, and scanning transmission electron microscope based electron energy-loss spectroscopy (STEM-EELS) and X-ray energy-dispersive spectroscopy for nanoscale energy-, structure, and chemical-resolved investigation of the defect source. Furthermore, this first demonstration of the direct observation of sub-bandgap defect levels via STEM-EELS, combined with the DLTS methods, provides strong evidence that the longsuspected Cu In/Ga substitutional defects are indeed the most likely source of the E V + 0.56 eV trap state and serves as a key example of this approach for the fundamental identification of defects within semiconductors, in general.

14 SOLAR ENERGY↗

Boundary-Aware Adversarial Learning Domain Adaption and Active Learning for Cross-Sensor Building Extraction

The use of convolutional neural networks (CNNs) for building extraction from remote sensing images has been widely studied and many public datasets have been made available for accelerating development of these CNN models. Yet adapting pretrained models at scale in real-world scenarios remains a challenging task. The main barrier is that certain new labels are still needed to compensate for domain shifting between the labeled data and new images that potentially cover new geographic locations or that are from a different sensor. In this article, we propose to add informatively labeled samples from a new image pool under the paradigm of active learning. To select the most useful samples based on model uncertainty, we first tackle the problem of uncalibrated uncertainty estimation due to distribution shifting by adapting feature extractors with boundary-based adversarial learning. Calibrated uncertainty is used as the query criterion in the active learning process, where the most uncertain samples are selected for annotation and included for model retraining. The proposed workflow was tested with three data pairs in which each workflow represents a scenario often encountered in real-world applications, including adapting pretrained models to new images collected with different sensors or to new geographic areas where appearances and types of buildings are very different. Compared to several baselines, including random sampling, temperature scaling (a well-known uncertainty calibration technique), different query strategies, and active domain adaptation methods, the proposed workflow shows that strategically querying a smaller set of samples for labeling achieves comparable or better building extraction performance. The proposed method reduces the number of labeled samples required to achieve sufficient model accuracy, thus significantly reducing hundreds of person-hours for labeled data creation. In addition, we include a few considerations when deploying this workflow in a GPU cluster that can be easily adapted to achieve operational building extraction model retraining.

97 MATHEMATICS AND COMPUTING↗

Automating Mid- and Long-Range Scheduling for NASA's Deep Space Network

NASA has recently deployed a new mid-range scheduling system for the antennas of the Deep Space Network (DSN), called Service Scheduling Software, or S(sup 3). This system is architected as a modern web application containing a central scheduling database integrated with a collaborative environment, exploiting the same technologies as social web applications but applied to a space operations context. This is highly relevant to the DSN domain since the network schedule of operations is developed in a peer-to-peer negotiation process among all users who utilize the DSN (representing 37 projects including international partners and ground-based science and calibration users). The initial implementation of S(sup 3) is complete and the system has been operational since July 2011. S(sup 3) has been used for negotiating schedules since April 2011, including the baseline schedules for three launching missions in late 2011. S(sup 3) supports a distributed scheduling model, in which changes can potentially be made by multiple users based on multiple schedule "workspaces" or versions of the schedule. This has led to several challenges in the design of the scheduling database, and of a change proposal workflow that allows users to concur with or to reject proposed schedule changes, and then counter-propose with alternative or additional suggested changes. This paper describes some key aspects of the S(sup 3) system and lessons learned from its operational deployment to date, focusing on the challenges of multi-user collaborative scheduling in a practical and mission-critical setting. We will also describe the ongoing project to extend S(sup 3) to encompass long-range planning, downtime analysis, and forecasting, as the next step in developing a single integrated DSN scheduling tool suite to cover all time ranges.

scheduling↗

A multi-dimensional parametric study of variability in multi-phase flow dynamics during geologic CO 2 sequestration accelerated with machine learning

Successful geologic CO 2 storage projects depend on numerical simulations to predict reservoir performance during site selection, injection verification, and post-injection monitoring phases of the project. These numerical simulations solve non-linear sets of coupled partial differential equations, while accounting for multi-phase fluid dynamics on the basis of constitutive equations that are embedded into the solution scheme. As a consequence, individual simulations often require tens to hundreds of hours to complete on high-performance computing clusters. Moreover, laboratory experiments reveal that parametric functions for capillary pressure and relative permeability exhibit substantial variability, even within the same rock type. This combination of computational expense and wide-ranging parametric variability means that there remains substantial uncertainty in the behavior of multi-phase CO 2 -water systems, particularly in the context of feedbacks between relative permeability and capillary pressure. To bridge this knowledge gap, here we develop a novel workflow that utilizes physics-based numerical simulation to train an artificial neural network (ANN) emulator for interrogating the multivariate parameter space that governs both capillary pressure and relative permeability. With this approach, the ANN is trained to emulate both fluid pressure distribution and CO 2 saturation, which are then interrogated quantitatively to generate parametric response surface mappings with high-fidelity resolution. Results from this study initially show that capillary entry pressure is the dominant control on both CO 2 plume geometry and fluid pressure propagation when considering the combined effects of capillary pressure and relative permeability, particularly when phase interference is low and residual CO 2 saturation is high. Moreover, the ANN emulator provides tremendous computational speed-up by computing 2691 individual simulations in several minutes; whereas, the same simulation ensemble would have required ~3 years of simulation time using only physics-based simulation methods (25,000 times speed up).

58 GEOSCIENCES↗

An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Wang, Tianle [Brookhaven National Laboratory (BNL)↗

Operational Evolution of FTS3: A DevOps Driven Approach to Elastic Operations

The File Transfer Service (FTS3) is a distributed data movement service developed at CERN and widely used to transfer data across the Worldwide LHC Computing Grid (WLCG). At Fermilab, FTS3 supports data transfers for multiple experiments, including Intensity Frontier experiments such as DUNE, enabling reliable data movement between WebDAV endpoints in Europe and the Americas.​ At CHEP 2021, we reported on the initial containerized deployment of FTS3 on OKD, the community Kubernetes distribution of Red Hat OpenShift. In this work, we present the subsequent evolution of this deployment, focusing on new operational capabilities introduced to improve scalability, robustness, and long-term maintainability.​ We describe the adoption of more secure and reproducible container build workflows, the integration of DevOps-driven operational practices, and enhancements in monitoring and automation. A key new result is the introduction of horizontal scaling and elastic resource management, allowing FTS3 components to dynamically adapt to workload variations while maintaining service reliability. We also discuss improvements in fault tolerance and operational procedures derived from production experience.​ Finally, we summarize lessons learned from operating FTS3 as a Kubernetes-native service and outline how these developments have improved the resilience and efficiency of data movement operations at Fermilab.

Munoz Flores, Victor Leopoldo [Fermilab]↗

Towards PCC for Concurrent and Distributed Systems (Work in Progress)

We outline some conceptual challenges in extending the PCC paradigm to a concurrent and distributed setting, and sketch a generalized notion of module correctness based on viewing communication contracts as economic games. The model supports compositional reasoning about modular systems and is meant to apply not only to certification of executable code, but also of organizational workflows.

Henriksen, Anders S.↗

Application of unsupervised deep learning to image segmentation and in-situ contact angle measurements in a CO 2 -water-rock system

Rock surface wettability is a critical property that regulates multiphase flows in porous media, which can be quantified using the surface contact angle (CA). X-ray micro-computed tomography (μCT) provides an effective approach to in-situ measurements of surface CAs. However, the CA measurement accuracy depends significantly on the quality of CT image segmentation, which is the clustering of CT pixels into separate phases. Inspired by this, we developed a deep learning (DL)-based CA measurement workflow. Motivated by the recent tremendous progress in unsupervised learning techniques and aiming to avoid expensive manual data annotations, an unsupervised DL pipeline for CT image segmentation was proposed and implemented, which includes unsupervised model training and post-processing. The unsupervised model training was driven by a novel loss function constrained with feature similarity and spatial continuity and implemented by iterative forward and backward paths; the former clustered the pixel-wise feature vectors extracted by convolution neural networks, whereas the latter updated the parameters using gradient descent. An over-segmentation strategy was adopted for model training. The post-processing steps based on agglomerative hierarchical clustering (AHC) were implemented to further merge the over-segmented model output to the desired cluster number, which is intended to improve the efficiency of image segmentation. The developed unsupervised DL pipeline was compared with other commonly-used image segmentation methods using pixel-wise and physics-based evaluation metrics on a synthetic raw-image dataset, which had a known ground truth. The unsupervised DL pipeline showed the best performance. Next, the segmented images were input to an automatic CA measurement tool, and the results were validated by comparisons with manual measurements. The CA values from the manual and automatic measurements showed similar distributions and statistical properties. The automatic measurement demonstrated a wider spectrum because of the much larger number of measurement data points. The primary novelty of the unsupervised DL pipeline developed in this study lies in the novel loss function and the over-segmentation strategy associated with AHC post-processing. Finally, the workflow has been proven an efficient tool for pore-scale wettability characterization, which has a wide range of applications in fundamental studies of multiphase flows in natural porous media, which have critical implications to geological carbon sequestration, hydrocarbon energy recovery, and contaminant transport in groundwater.

42 ENGINEERING↗

Simulation-based characterization of the variability of earthquake risk to buildings in the near-field

Recent advancements in high performance computing platforms and computational workflow for regional-scale simulations are enabling unprecedented modeling of fault-to-structure earthquake processes. Regional simulations resolving ground motions at frequencies relevant to engineered systems are becoming computationally viable and provide a new capability to improve understanding of the geographical distribution and intensity of risk to buildings and critical infrastructure. As computational capabilities advance, it is essential to move beyond illustrative single rupture realizations for scenario earthquake events towards the development of a full suite of rupture realizations that appropriately characterize the range of risk to building systems. The work described in this article investigates the application of a suite of fault rupture realizations with the objective of assessing near-fault, site-specific seismic demand variability for building structures. A representative high-performance regional-scale computational model is utilized to execute ground motion and building response simulations based on 18 kinematic rupture realizations of an M7 strike-slip scenario earthquake. The fault rupture models for the scenario earthquake are created by systematically perturbing the hypocenter location and stochastically generating rupture parameters (slip, rise time, rake angle) to represent a breadth of ground motion intensities resulting from the spatial and temporal variabilities of an earthquake rupture process. The resulting seismic demand variability for three-story (short period) and forty-story (long period) steel moment-resisting frame buildings is characterized in terms of the median and distribution of peak inter-story drift ratio for a range of near-fault sites. The full suite of 18 fault rupture realizations and approximately 280,000 nonlinear dynamic building simulations indicate that the three-story building undergoes higher median seismic demand and significantly greater variability of demand at a given site than the forty-story building, which has important implications for the level of certainty in predicting building performance during an earthquake. The simulations performed provide deeper insight into the relationship between fault rupture parameterization and building response, which is essential information for developing a representative suite of rupture realizations for specific earthquake scenarios.

58 GEOSCIENCES↗