Search NASA⌕ Search

SEARCH · Search NASA

Results for “Advanced Workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A Processing and Analytics System for Microscopy Data Workflows: The Pycroscopy Ecosystem of Packages

Major advancements in fields as diverse as biology and quantum computing have relied on a multitude of microscopy techniques. Despite the considerable proliferation of these instruments, significant bottlenecks remain in terms of processing, analysis, storage, and retrieval of the acquired datasets. Aside from lack of file standards, individual domain-specific analysis packages are often disjoint from the underlying datasets, and thus keeping track of analysis and processing steps remains tedious for the end-user, hampering reproducibility. Here, in this study, the pycroscopy ecosystem of packages is introduced, an open-source python-based ecosystem underpinned by a common data model. The data model, termed the N-dimensional spectral imaging data format, is realized in pycroscopy's sidpy package. This package is built on top of dask arrays, thus leveraging dask array attributes, but expanding them to accelerate microscopy relevant analysis and visualization. Several examples of the use of the pycroscopy ecosystem to create workflows for data ingestion and analysis of scanning transmission electron microscopy (STEM) and scanning probe microscopy data are shown. Adoption of such standardized routines will be critical to usher in the next generation of autonomous instruments where processing, computation, and meta-data storage will be critical to overall experimental operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A Case Study of Multimodal, Multi-institutional Data Management for the Combinatorial Materials Science Community

Although the convergence of high-performance computing, automation, and machine learning has significantly altered the materials design timeline, transformative advances in functional materials and acceleration of their design will require addressing the deficiencies that currently exist in materials informatics, particularly a lack of standardized experimental data management. The challenges associated with experimental data management are especially true for combinatorial materials science, where advancements in automation of experimental workflows have produced datasets that are often too large and too complex for human reasoning. The data management challenge is further compounded by the multimodal and multi-institutional nature of these datasets, as they tend to be distributed across multiple institutions and can vary substantially in format, size, and content. Furthermore, modern materials engineering requires the tuning of not only composition but also of phase and microstructure to elucidate processing–structure–property–performance relationships. To adequately map a materials design space from such datasets, an ideal materials data infrastructure would contain data and metadata describing (i) synthesis and processing conditions, (ii) characterization results, and (iii) property and performance measurements. In this work, we present a case study for the low-barrier development of such a dashboard that enables standardized organization, analysis, and visualization of a large data lake consisting of combinatorial datasets of synthesis and processing conditions, X-ray diffraction patterns, and materials property measurements generated at several different institutions. While this dashboard was developed specifically for data-driven thermoelectric materials discovery, we envision the adaptation of this prototype to other materials applications, and, more ambitiously, future integration into an all-encompassing materials data management infrastructure.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Human Gut Microbiota from Autism Spectrum Disorder Promote Behavioral Symptoms in Mice

Large-scale phosphoproteomics with coverage of over 10,000 sites of phosphorylation have now been routinely achieved with advanced mass spectrometry (MS)-based workflows. However, accurate targeted MS-based quantification of phosphorylation dynamics, an important direction for gaining quantitative understanding of signaling pathways or networks, has been much less investigated. Herein, we report an assessment of the targeted workflow in the context of signal transduction pathways, using the epidermal growth factor receptor (EGFR)–mitogen-activated protein kinase (MAPK) pathway as our model. 43 phosphopeptides from the EGFR–MAPK pathway were selected for the study. The recovery and sensitivity of a workflow consisted of two commonly used enrichment methods, immobilized metal affinity chromatography (IMAC) and titanium oxide (TiO2), combined with selected reaction monitoring (SRM)-MS, were evaluated. The recovery of phosphopeptides by IMAC and TiO2 enrichment was quantified to be 38 ± 5% and 58 ± 20%, respectively, based on internal standards. Moreover, both enrichment methods provided comparable sensitivity from 1-100 g starting peptides. Robust quantification was consistently achieved for most targeted phosphopeptides when starting with 25-100 g peptides. However, the numbers of quantified targets significantly dropped when peptide samples were in the 1-25g range. Finally, IMAC-SRM was applied to quantify signaling dynamics of EGFR-MAPK pathway in Hs578T cells following 3 ng/mL EGF treatment. The kinetics of phosphorylation clearly revealed early and late phases of phosphorylation, even for very low abundance proteins. These results demonstrate the feasibility of robust targeted quantification of phosphorylation dynamics for specific pathways, even starting with relatively small amounts of protein.

Sharon, Gil↗
DOE OSTI2022

Peridynamics and surrogate modeling of pressure-driven well stimulation

In this work we use the peridynamics theory of solid mechanics to simulate fracture in an annular rock domain subject to an in-situ stress and create surrogate models that predict the area of the resulting cracks. Peridynamics is a non-local formulation of continuum mechanics that naturally accommodates material discontinuities. Furthermore, unlike other fracture modeling techniques there is no need to provide information about the crack path. We utilize the peridynamics code Peridigm and take a two-stage approach to fracture modeling. First an implicit solve is performed to compute the in-situ stress state. We then execute an explicit solve where a pressure loading designed to emulate fluid-driven hydraulic fracture is applied at the borehole and transmitted to the pre-stressed rock. We present results from polynomial and single and multi-level Gaussian process surrogate models constructed from a sampling study of the peridynamics model. The surrogates predict crack area given a measure of the in-situ stress anisotropy and rise time and amplitude of the pressure loading. These surrogates take a minuscule fraction of peridynamics model's running time to evaluate and are a step towards enabling advanced optimization and uncertainty quantification workflows that require many model evaluations.

42 ENGINEERING↗
DOE OSTI2026

Model data for infrastructure-aware simulation of compound flooding at Alligator Bayou Watershed, southeast Texas

This dataset supports infrastructure-aware hydrologic modeling and flood scenario analysis for the Alligator Bayou Watershed, a highly managed urban watershed in Southeast Texas. It includes Jupyter notebooks for figure reproduction, model configuration files, simulation outputs, and derived products used to quantify the influence of engineered stormwater infrastructure on flood behavior across multiple spatial scales. The dataset was generated using the Watershed Workflow Python package and the Advanced Terrestrial Simulator (ATS), enabling integrated surface–subsurface hydrologic simulations on a channel-aligned mesh with explicit representations of pump stations, gate structures, detention basins, and impervious surfaces. Outputs include time series of gate and pump flows, stage observations, and water balance components, as well as spatially explicit fields of peak ponded depth and flood duration across multiple infrastructure scenarios spanning a single-location detention basin expansion, distributed drainage limitations, and compound coastal flooding. These data facilitate full reproducibility of the manuscript figures and support further research on urban flood dynamics and the role of stormwater infrastructure in shaping watershed-scale flood response.

EARTH SCIENCE > OCEANS > COASTAL PROCESSES↗
NASA NTRS2005

Software to Manage the Unmanageable

In 1995, NASA s Jet Propulsion Laboratory (JPL) contracted Redmond, Washington-based Lucidoc Corporation, to design a technology infrastructure to automate the intersection between policy management and operations management with advanced software that automates document workflow, document status, and uniformity of document layout. JPL had very specific parameters for the software. It expected to store and catalog over 8,000 technical and procedural documents integrated with hundreds of processes. The project ended in 2000, but NASA still uses the resulting highly secure document management system, and Lucidoc has managed to help other organizations, large and small, with integrating document flow and operations management to ensure a compliance-ready culture.

Source record↗
DOE OSTI2022

Smart sensors using artificial intelligence for on-detector electronics and ASICs

Cutting edge detectors push sensing technology by further improving spatial and temporal resolution, increasing detector area and volume, and generally reducing backgrounds and noise. This has led to a explosion of more and more data being generated in next-generation experiments. Therefore, the need for near-sensor, at the data source, processing with more powerful algorithms is becoming increasingly important to more efficiently capture the right experimental data, reduce downstream system complexity, and enable faster and lower-power feedback loops. In this paper, we discuss the motivations and potential applications for on-detector AI. Furthermore, the unique requirements of particle physics can uniquely drive the development of novel AI hardware and design tools. We describe existing modern work for particle physics in this area. Finally, we outline a number of areas of opportunity where we can advance machine learning techniques, codesign workflows, and future microelectronics technologies which will accelerate design, performance, and implementations for next generation experiments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Resilient, Rural, and Revolutionary: Salisbury Square's Direct-Current Affordable Microgrid Community: Preprint

The technology to interconnect buildings with a dedicated direct-current (DC) power distribution network is in place today; what is missing is a turnkey approach to designing a DC microgrid - and the business models allowing such systems to be deployed, owned, and operated at scale. To close this gap, the Salisbury Square Development Team, comprising clean-energy experts, has engineered a resilient community DC microgrid for an affordable housing community in Randolph, Vermont. Ten single-family, occupant-owned residences and 12 multifamily rental units will share locally generated and stored solar energy via a DC power distribution bus capable of operating during extended grid outages. With a DC power distribution network in place, each home will be equipped with high-efficiency DC lighting and appliances, operating alongside alternating current (AC) appliances, even during an islanded mode of operation. To obtain a comprehensive understanding of what is possible and achievable, the Team collaborated with the local utility, regulatory agencies, a national laboratory, energy-as-service providers, and vendors. The collaborators evaluated microgrid typologies, business models, and energy modeling, and analyzed electrification and resilience. Further, the Team applied the URBANoptTM (Urban Renewable Building and Neighborhood optimization, NREL 2022) software development kit (SDK) to Salisbury Square's single-family and multifamily buildings to validate workflows and identify needs for advanced capability. This paper addresses the barriers to entry, scalability, and impact on residents and system ownership. It also examines the analysis that informed the design and engineering of the DC microgrid and the opportunities to streamline the process.

DER↗
DOE OSTI2024

Scaling the SciDAC QuantOm Workflow

As part of the Scientific Discovery through Advanced Computing (SciDAC) program, the Quantum Chromodynamics Nuclear Tomography (QuantOM) project aims to analyze data from Deep Inelastic Scattering (DIS) experiments conducted at Jefferson Lab and the upcoming Electron Ion Collider. The DIS data analysis is performed on an event-level by combining the input from theoretical and experimental nuclear physics into a single, composable workflow. The optimization itself (I.e. fitting the experimental data with theoretical predictions) is carried out by a machine / deep learning algorithm. The size of the acquired DIS data as well as the complexity of the workflow itself require that the analysis is performed across multiple GPUs on high performance computing systems, such as Polaris at Argonne National Laboratory. This presentation discusses the novelties and challenges that came along with parallelizing this workflow. Recent results are compared to common distributed training techniques.

Lersch, Daniel↗
DOE OSTI2020

Cross-Cutting Software Solutions in Support of Experimental Analysis Challenges at National Scattering Facilities

Rapid improvements in instrumentation at neutron and light source combined with new capabilities in computing, data analytics, and machine learning offer unprecedented scientific capabilities to address fundamental and applied science questions in a wide area of subjects from quantum materials to biological systems. In order to make these capabilities accessible to the broader scientific community, significant developments are needed related to workflow spanning from experimental control to advanced materials simulations as well as the underpinning scientific software. In this chapter, we explore these needs and recent developments at the Spallation Neutron Source (SNS) at Oak Ridge National Laboratory. A central part of the workflow at the SNS is the data reduction framework Mantid which is developed as a partnership between European and US neutron-scattering facilities.

Proffen, Thomas↗
DOE OSTI2024

The SciDAC QuantOm Framework: A composable Workflow

As part of the Scientific Discovery through Advanced Computing (SciDAC) program, the Quantum Chromodynamics Nuclear Tomography (QuantOM) project aims to analyze data from Deep Inelastic Scattering (DIS) experiments conducted at Jefferson Lab and the upcoming Electron Ion Collider. The DIS data analysis is performed on an event-level by leveraging nuclear theory models and accounting for experimental conditions. In order to efficiently run multiple analyses under varying conditions, a composable workflow was designed where each section (theory, experiment, objective minimization, etc.) has its own dedicated module. The optimization, i.e. the fit of theory to experimental data is carried out by deep learning techniques, such as Generative Adversarial Networks (GANs) or Reinforcement Learning (RL). This presentation gives an overview of the current status of the workflow, highlights present and future challenges, and highlights possible extensions to other projects with similar requirements.

Lersch, Daniel↗
DOE OSTI2023

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows

This report details recent progress for the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows”. We refer to the project as IPPD/2, reflecting the 2017 renewal under expanded scope and partners In IPPD/2, we increased our research scope to include data motion. We are focusing on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. This new work on data motion will augment and complement IPPD/2’s research that focused on the computational aspects of tasks. We leverage and extend our existing tools and demonstrate our work on the Belle II workflow suite as well as on workflows from NSLS-II. The highlights of our work are as follows: Provenance for Workflows: Provenance is used to provide information enabling quality control, re-run computational workflows, and reproduce results. IPPD/2 has been building a scalable provenance management system that enables the capture of provenance from the high-level workflow through all relevant system levels in one integrated environment. Leveraging this work, our recent efforts have included using provenance as an enabling technique. Workload characterization: Leveraging provenance and analysis, we characterize data movement within network, storage, and memory over a variety of workloads. This characterization enables an understanding by performance analysts and application developers of the range of behaviors that could be expected. Performance Prediction for Workflows: The goal of modeling distributed workflows is to understand performance bottlenecks and enable more intelligent task scheduling to optimize selected metrics of interest (e.g., task throughput or output data rate). IPPD/2 has utilized both analytical and AI/ML modeling methodologies for performance modeling. Advanced Scheduling and Fault Modeling for Workflows: Scheduling of large-scale scientific workflows on geographically distributed resources is a challenging problem. To improve workflow throughput, we combined novel scheduling algorithms with task predictions from performance modeling and fault modeling. Dynamically Alleviating Bottlenecks in Workflows: Exploiting our provenance, analysis, and modeling efforts, we have explored and developed several techniques for dynamically detecting and alleviating bottlenecks in data movement. In particular, we have spent considerable effort demonstrating our techniques on production-like workflow configurations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Toward digital design at the exascale: An overview of project ICECap

High performance computing has entered the Exascale Age. Capable of performing over 1018 floating point operations per second, exascale computers, such as El Capitan, the National Nuclear Security Administration's first, have the potential to revolutionize the detailed in-depth study of highly complex science and engineering systems. However, in addition to these kind of whole machine “hero” simulations, exascale systems could also enable new paradigms in digital design by making petascale hero runs routine. Currently, untenable problems in complex system design, optimization, model exploration, and scientific discovery could all become possible. Motivated by the challenge of uncovering the next generation of robust high-yield inertial confinement fusion (ICF) designs, project ICECap (Inertial Confinement on El Capitan) attempts to integrate multiple advances in machine learning (ML), scientific workflows, high performance computing, GPU-acceleration, and numerical optimization to prototype such a future. Built on a general framework, ICECap is exploring how these technologies could broadly accelerate scientific discovery on El Capitan. In addition to our requirements, system-level design, and challenges, we describe some of the key technologies in ICECap, including ML replacements for multiphysics packages, tools for human-machine teaming, and algorithms for multifidelity design optimization under uncertainty. As a test of our prototype pre-El Capitan system, we advance the state-of-the art for ICF hohlraum design by demonstrating the optimization of a 17-parameter National Ignition Facility experiment and show that our ML-assisted workflow makes design choices that are consistent with physics intuition, but in an automated, efficient, and mathematically rigorous fashion.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Proceedings for the Workshop on Applied Nuclear Data Activities 2025

The 2025 Workshop for Applied Nuclear Data Activities (WANDA) covered four topic areas in nuclear data: Nuclear Data and Deterrence, Nuclear Data Prioritization for Fusion, High-Assay Low-Enriched Uranium and Novel Moderators for Advanced Reactors, and Data Preservation and Data Workflows. The intention of this workshop is to connect different communities that are invested in nuclear data and have their own unique sets of needs for the purposes of sharing information, fostering collaboration in areas of shared interest, and leveraging synergistic capabilities. The attendance of federal program managers at these workshops is essential in creating awareness of the needs of their respective communities and in providing information to better guide funding investments. In each of these topical sessions, a general description of the nuclear data needs and/or capabilities was presented, along with discussions of existing capabilities that could be leveraged, potential synergistic needs or resources, and challenges that must be overcome for the application space to progress. There are many synergistic nuclear data needs among these application spaces. The discussion largely focused on increasing the accuracy of the nuclear data and better quantifying the data uncertainties that have the greatest impact on applications. The full-day session on data processing and data workflows was by nature intended to be synergistic and applicable to all technical sessions at WANDA. A notable common theme that was highlighted across all sessions was the need for accelerated delivery of nuclear data products across complex and time-consuming workflows.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

New Virtual Test Bed Capabilities: Virtual DOME Model and New Updates to Repository

The Department of Energy (DOE) Office of Nuclear Energy National Reactor Innovation Center accelerates the deployment of novel reactor concepts by establishing both physical and virtual spaces for building and testing various components, systems, and complete pilot plants. The Virtual Test Bed represents the virtual arm of the National Reactor Innovation Center and is a joint effort with the DOE Nuclear Energy Advanced Modeling and Simulation Program. The Virtual Test Bed mission is to accelerate the deployment of advanced reactors by facilitating the adoption of cutting-edge DOE advanced modeling and simulation tools to design, evaluate, and license reactors. This is primarily achieved by storing example challenge problems in an externally available repository and by developing models to fill the M&S gaps needed for potential demonstrators. Activities conducted this fiscal year focused on developing of a Demonstration of Microreactor Experiments shield model to help accelerate the confirmatory analysis required for the reactor demonstration. This model and workflow will allow developers to leverage advanced modeling and simulation tools to ensure their reactor demonstration concept will meet dose requirements and that the surrounding shield will stay within concrete temperature limits during steady-state and transient operation conditions. An initial model has been developed to evaluate the temperature distribution in the concrete shield during steady-state operation, including neutron and gamma heating effects. Various modeling strategies have been examined to understand their applicability and limitations with different reactor designs to make the workflow as reactor-agnostic as possible and computationally effective to maximize its usability. In addition to describing the Demonstration of Microreactor Experiments shield model and associated results, this report summarizes other accomplishments regarding repository maintenance and improvement and new external models hosted on the repository.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization

Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. These optimizations can improve the compression ratio of SOTA approaches by up to 3.3× under the same data quality loss. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.

Wang, Daoce↗
DOE OSTI2021

Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows (IPPD) (Final Report)

This report details the accomplishments from the ASCR funded project “Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows” under the award numbers FWP-66406 and DE-SC0012630, with a focus on the UC San Diego (Award No. DE-SC0012630) part of the accomplishments. We refer to the project as IPPD. The main activities of IPPD were centered on the development and integration of provenance information to capture empirically workflow information and identify the sources of bottlenecks as well as variability, and modeling and simulation to provide insights and predictively explore multiple scenarios for development of advanced techniques for optimization of resources and workflow execution. After a Phase I of the project, a Phase II effort focused on three major aspects: a) observe how data is generated, distributed, and used; b) analyze how data is (repeatedly) consumed with a focus both on repeated patterns and anomalies; and c) explore how to optimize data motion. The project leveraged and extended our existing tools with new research and demonstrated our work on the Belle II workflow suite as well as on workflows from NSLS-II.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Alquimia v1.0: a generic interface to biogeochemical codes – a tool for interoperable development, prototyping and benchmarking for multiphysics simulators

Alquimia v1.0 is a generic interface to geochemical solvers that facilitates development of multiphysics simulators by enabling code coupling, prototyping and benchmarking. The interface enforces the function arguments and their types for setting up, solving, serving up output data and carrying out other common auxiliary tasks while providing a set of structures for data transfer between the multiphysics code driving the simulation and the geochemical solver. Alquimia relies on a single-cell approach that permits operator splitting coupling and parallel computation. We describe the implementation in Alquimia of two widely used open-source codes that perform geochemical calculations: PFLOTRAN and CrunchFlow. We then exemplify its use for the implementation and simulation of reactive transport in porous media by two open-source flow and transport simulators: Amanzi and ParFlow. We also demonstrate its use for the simulation of coupled processes in novel multiphysics applications including the effect of multiphase flow on reaction rates at the pore scale with OpenFOAM, the role of complex biogeochemical processes in land surface models such as the E3SM Land Model (ELM) and the impact of surface–subsurface hydrological interactions on hydrogeochemical export from watersheds with the Advanced Terrestrial Simulator (ATS). These applications make it apparent that the availability of a well-defined yet flexible interface has the potential to improve the software development workflow, freeing up resources to focus on advances in process models and mechanistic understanding of coupled problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗