Search NASA⌕ Search

SEARCH · Search NASA

Results for “computational workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A lightweight method for evaluating in situ workflow efficiency

Performance evaluation is crucial to understanding the behavior of scientific workflows. In this study, we target an emerging type of workflow, called in situ workflows. These workflows tightly couple components such as simulation and analysis to improve overall workflow performance. To understand the tradeoffs of various configurable parameters for coupling these heterogeneous tasks, namely simulation stride, and component placement, separately monitoring each component is insufficient to gain insights into the entire workflow behavior. Through an analysis of the state-of-the-art research, we propose a lightweight metric, derived from a defined in situ step, for assessing resource usage efficiency of an in situ workflow execution. By applying this metric to a synthetic workflow, which is parameterized to emulate behaviors of a molecular dynamics simulation, we explore two possible scenarios (Idle Simulation and Idle Analyzer) for the characterization of in situ workflow execution. In addition to preliminary results from a recently published study [11], we further exploit the proposed metric to evaluate a practical in situ workflow with a real molecular dynamics application, i.e., GROMACS. Here, experimental results show that the in transit placement (analytics on dedicated nodes) sustains a higher frequency for performing in situ analysis compared to the helper-core configuration (analytics co-allocated with simulation).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Seismology in the Cloud: A New Streaming Workflow

Data-intensive research in seismology is experiencing a recent boom, driven in part by large volumes of available data and advances in the growing field of data science. However, there are significant barriers to processing large data volumes, such as long retrieval times from data repositories, complex data management, and limited computational resources. New tools and platforms have reduced the barriers to entry for scientific cluster computing, including the maturation of the commercial cloud as an accessible instrument for research. Here, we build a customized research cluster in the cloud to test a new workflow for large-scale seismic analysis, in which data are processed as a stream (retrieved on-the-fly and acted upon without storing), with data from the Incorporated Research Institutions for Seismology Data Management Center. We use this workflow to deploy a spectral peak detection algorithm over 5.6 TB of compressed continuous seismic data from 2074 stations of the USArray Transportable Array EarthScope network. Using a 50-node cluster in the cloud, we completed the noise survey in 80 hr, with an average data throughput of 1.7 GB per minute. By varying cluster sizes, we find the scaling of our analysis to be sublinear, due to a combination of algorithmic limitations and data center response times. The cloud-based streaming workflow represents an order-of-magnitude increase in acquisition and processing speed compared to a traditional download-store-process workflow, and offers the additional benefits of employing a flexible, accessible, and widely used computing architecture. It is limited, however, due to its reliance on Internet transfer speeds and data center service capacity, and may not work well for repeated analyses or those for which even higher data throughputs are needed. These research applications will require a new class of cloud-native approaches in which both data and analysis are in the cloud.

58 GEOSCIENCES↗
DOE OSTI2024

A galactic approach to neutron scattering science

Neutron scattering science is leading to significant advances in our understanding of materials and will be key to solving many of the challenges that society is facing today. Improvements in scientific instruments are actually making it more difficult to analyze and interpret the results of experiments due to the vast increases in the volume and complexity of data being produced and the associated computational requirements for processing that data. New approaches to enable scientists to leverage computational resources are required, and Oak Ridge National Laboratory (ORNL) has been at the forefront of developing these technologies. We recently completed the design and initial implementation of a neutrons data interpretation platform that allows seamless access to the computational resources provided by ORNL. For the first time, we have demonstrated that this platform can be used for advanced data analysis of correlated quantum materials by utilizing the world's most powerful computer system, Frontier. In particular, we have shown the end-to-end execution of the DCA++ code to determine the dynamic magnetic spin susceptibility χ(q, ω) for a single-band Hubbard model with Coulomb repulsion U/t = 8 in units of the nearest-neighbor hopping amplitude t and an electron density of n = 0.65. The following work describes the architecture, design, and implementation of the platform and how we constructed a correlated quantum materials analysis workflow to demonstrate the viability of this system to produce scientific results.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Machine learning enabled discovery of superhard and ultrahard carbon polymorphs

The demand for multifunctional materials has motivated the move from near-equilibrium materials to metastable i.e. out-of-equilibrium phases that can meet several desired target properties. The search for such metastable phases with exotic properties is non-trivial and often serendipitous. Inverse design approaches based on evolutionary search have been powerful tools, but such traditional searches have focused on identifying primarily stable and metastable materials with the lowest enthalpy. The inverse design of materials, with a focus on a desired property such as, for example, hardness is a challenging task because of the expensive computational cost involved in sampling multiple structures. The recent advances in machine learning have brought new powerful AI techniques to the forefront which can potentially revolutionize the inverse design and discovery of materials, especially metastable phases capable of meeting multifunctionality. Here, in this work, we develop and apply an automated reinforcement learning workflow for inverse design that integrates first principles physics and atomistic simulations with machine learning (ML), and high-performance computing to allow rapid exploration of the superhard and ultrahard metastable phases of Carbon. We demonstrate an automatic machine learning based inverse design workflow to map new undiscovered metastable states ranging from near equilibrium to those far-from-equilibrium that satisfy multiple property objectives, specifically bulk moduli, shear moduli and hardness. We create a comprehensive library of carbon stable and metastable phases with varying hardness and subsequently shortlist 10 top performing candidate carbon structures, including two newly reported phases, based on their hardness and characterize their temperature dependent mechanical properties. A neural network model is built using featurization of allotropes of carbon to predict the quasi-harmonic Gibbs free energies. The Gibbs free energies of the top performing phases are analyzed to get an estimate of the experimental synthesizability of these superhard and ultrahard carbon phases. In general, we show using machine learning based inverse design approaches how hitherto inaccessible metastable states can be identified and potentially synthesized to meet the demand for multifunctional materials.

Balasubramanian, Karthik [Univ. of Illinois, Chica↗
DOE OSTI2025

FAIR Ecosystems for Science at Scale

High Performance Computing (HPC) centers provide resources to users who require greater scale to “get science done”. They deploy infrastructure with singular hardware architectures, cutting-edge software environments, and stricter security measures as compared with users’ own resources. As a result, users often create and configure digital artifacts in ways that are specialized for the unique infrastructure at a given HPC center. Each user of that center will face similar challenges as they develop specialized solutions to take full advantages of the center’s resources, potentially resulting in significant duplication of effort. Much duplicated effort could be avoided, however, if users of these centers found it easier to discover others’ solutions and artifacts as well as share their own. The FAIR principles address this problem by presenting guidelines focused around metadata practices to be implemented by vaguely defined “communities”; in practice, these tend to gather by domain (e.g. bioinformatics, geosciences, agriculture). Domain-based communities can unfortunately end up functioning as silos that tend both to inhibit sharing of solutions and best practices as well as to encourage fragile and unsustainable improvised solutions in the absence of best-practice guidance. We propose that these communities pursuing “science at scale” be nurtured both individually and collectively by HPC centers so that users can take advantage of shared challenges across disciplines and potentially across HPC centers. We describe an architecture based on the EOSC-Life FAIR Workflows Collaboratory, specialized for use with and inside HPC centers such as the Oak Ridge Leadership Computing Facility (OLCF), and we speculate on user incentives to encourage adoption. We note that a focus on FAIR workflow components rather than FAIR workflows is more likely to benefit the users of HPC centers.

Wilkinson, Sean [ORNL] (ORCID:0000000214437479)↗
DOE OSTI2019

Supporting Simulation Families in Nuclear Reactor Workflows SBIR Phase I. Final Scientific/Technical Report

The US Department of Energy’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) Toolkit has accelerated both the solutions to existing problems and deployment of new designs for current and advanced reactors. Within the toolkit, there exist various families of simulation tools that target specific aspects of a multi-physics simulation for a nuclear reactor. Tools within a specific family vary in both complexity and fidelity as well as the types of resources they require. To complement the NEAMS effort, the Computational Model Builder (CMB) has been designed to provide a graphical user interface to design and execute nuclear reactor workflows. Previously, we had designed a workflow in CMB to design and prepare for execution a neutronics simulation using DIF3D, a diffusion and transport solver developed at Argonne National Laboratory. For this Phase I effort, we extended CMB’s reactor modeling tools to design and execute a workflow using Proteus-SN, a higher-fidelity transport solver developed at Argonne with more preparatory requirements from the workflow. Furthermore, this workflow was designed to maximize the common features of the two workflows, describing the analogous tasks between the workflows as belonging to a “simulation family”. This perspective facilitates the abstraction of the neutronics simulations to a single bifurcated workflow, allowing the engineer to design a reactor and then choose the appropriate simulation according to his or her needs for fidelity and computational cost.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

PyARC Status Report: New Integrations and Upgrades to the Fast Reactor Analysis Workflow Management Tool

PyARC was initially developed as an open source tool to support fast reactor analyses using the Argonne Reactor Computation (ARC) code suite as a part of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) Workbench initiative in FY17. The goal of this initiative is to provide a common user interface for model generation, real-time validation, execution, output processing, and visualization for all integrated codes. This is accomplished through the reliance on tools available in the Workbench framework and runtime environment. While initially developed to support the ARC codes, PyARC was extended in FY22 to wrap other NEAMS and non-ARC codes, including Griffin and OpenMC, in the supported other neutronics workflows, and support users in the adoption of NEAMS-supported high fidelity analysis codes. Most recently, NUBOW-3D, a recently adopted ARC code, was integrated to support reactor bowing calculations as well. Integration of these codes into the NEAMS Workbench directly benefits the advanced reactor modeling community by: • Providing a set of controlled, maintained, documented and validated scripts to generate inputs, which promotes best practices, reduces the learning curve, and facilitates project collaboration. • Improving the user experience: the Workbench interface provides assistance for building an input through auto-completion, real-time validation, document navigation, and geometry and results visualization. • Automating complex calculations and workflows for reactor analysis. • Helping users transition to using high-fidelity NEAMS codes along-side the ARC codes. In FY22, a progress report was published that described the state of each of the tools integrated into PyARC. Since then, there have been many enhancements and upgrades to the existing integrations as well as entirely new code integrations as well. This report details all new integrations and major developments in PyARC since the version 2.0.0 release highlighted in the FY22 report.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

The Artificial Scientist: in-Transit Machine Learning of Plasma Simulations

Large-scale simulations or scientific experiments produce petabytes of data per run. This poses massive challenges for I/O and storage when scientific analysis workflows are run manually offline. Unsupervised deep learning-based techniques to extract patterns and non-linear relations from these large amounts of data provide a way to build scientific understanding from raw data, reducing the need for manual pre-selection of analysis steps, but require exascale compute and memory to process the full dataset available. In this paper, we demonstrate a heterogeneous streaming workflow in which plasma simulation data is streamed directly to a Machine Learning (ML) application training a model on the simulation data in-transit, completely circumventing the capacity-constrained filesystem bottleneck. This workflow employs openPMD to provide a high level interface to describe scientific data and also uses ADIOS2, to transfer volumes of data that exceed the capabilities of the filesystem. We employ experience replay to avoid catastrophic forgetting in learning from this non-steady state process in a continual manner and adapt it to improve model convergence while learning in-transit. As a proof-of-concept, we approach the ill-posed inverse problem of predicting particle dynamics from radiation in a particle-incell (PIConGPU) simulation of the Kelvin-Helmholtz instability (KHI). We detail hardware-software co-design challenges as we scale PIConGPU to full Frontier, the Top-1 system as of June 2024 Top500 list.

Kelling, Jeffrey [Helmholtz-Zentrum Dresden Rossen↗
DOE OSTI2019

Earth System Grid Federation (ESGF) Compute Challenge

This Open Geospatial Consortium (OGC) Engineering Report (ER) describes the advancement of an Execution Management System (EMS) to support Web Processing Service (WPS) climate processes deployed on the Earth System Grid Federation (ESGF). The report introduces climate data, processes and applications into Common Workflow Language (CWL) workflows with the intent of advancing: application packaging, deployment and execution in clouds; interoperability of services in federated cyberinfrastructures; and geospatial workflows towards standardization. Work presented in this report is a direct continuation of the Earth Observation & Clouds (EOC) thread of Testbed-14. This report is expected to be of relevance to Testbed-15, both to the Earth Observation Process and Application Discovery (EOPAD) task and the Machine Learning task. This engineering report will describe: relevant work conducted in OGC Testbed-14; ESGF and its compute challenge; adaptations of existing climate processes into workflows; interoperability experiments with ESGF endpoints conforming to a common API.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

In-memory integration of existing software components for parallel adaptive unstructured mesh workflows

Unstructured mesh methods, like finite elements and finite volumes, support the effective analysis of complex physical behaviors modeled by partial differential equations over general threedimensional domains. The most reliable and efficient methods apply adaptive procedures with a-posteriori error estimators that indicate where and how the mesh is to be modified. Although adaptive meshes can have two to three orders of magnitude fewer elements than a more uniform mesh for the same level of accuracy, there are many complex simulations where the meshes required are so large that they can only be solved on massively parallel systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

HEPCloud, a New Paradigm for HEP Facilities: CMS Amazon Web Services Investigation

Historically, high energy physics computing has been performed on large purpose-built computing systems. These began as single-site compute facilities, but have evolved into the distributed computing grids used today. Recently, there has been an exponential increase in the capacity and capability of commercial clouds. Cloud resources are highly virtualized and intended to be able to be flexibly deployed for a variety of computing tasks. There is a growing interest among the cloud providers to demonstrate the capability to perform large-scale scientific computing. In this paper, we discuss results from the CMS experiment using the Fermilab HEPCloud facility, which utilized both local Fermilab resources and virtual machines in the Amazon Web Services Elastic Compute Cloud. We discuss the planning, technical challenges, and lessons learned involved in performing physics workflows on a large-scale set of virtualized resources. Additionally, we will discuss the economics and operational efficiencies when executing workflows both in the cloud and on dedicated resources.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2018

The InSAR Scientific Computing Environment 3.0: A Flexible Framework for NISAR Operational and User-Led Science Processing

The InSAR Scientific Computing Environment (ISCE) was first developed under the NASA Advanced Information Systems Technology as a flexible, extensible object-oriented framework for Interferometric Synthetic Aperture Radar (InSAR) processing. The ISCE framework uses Python 3 at the workflow level, controlling modules of compiled code for functional processing, and managing inputs, outputs, and other flow control services. The currently released version, called ISCE 2.1, is distributed to the research community through the Western North America InSAR Consortium under a research license. The ISCE team is working on the next generation of the code in order to prepare for the NASAISRO SAR (NISAR) mission operational processing. Innovations in this code include augmentation or conversion of the custom Python framework elements in ISCE with the Pyre framework, new workflows for interferometric and polarimetric stack processing, a more intuitive and graphically based user interface, and flow control for hybrid computing environments including CPU/GPU clusters, logging and error tracking facilities, and new more efficient computational modules that exploit graphical processor units (GPUs) when available. The ISCE 3.0 framework is designed to work in an operational environment as well as on a single user’s laptop or compute cluster, with services to discover capabilities and scale computations accordingly.

Buckley, Sean M.↗
DOE OSTI2016

Enabling the democratization of the genomics revolution with a fully integrated web-based bioinformatics platform

Continued advancements in sequencing technologies have fueled the development of new sequencing applications and promise to flood current databases with raw data. A number of factors prevent the seamless and easy use of these data, including the breadth of project goals, the wide array of tools that individually perform fractions of any given analysis, the large number of associated software/hardware dependencies, and the detailed expertise required to perform these analyses. To address these issues, we have developed an intuitive web-based environment with a wide assortment of integrated and cutting-edge bioinformatics tools in pre-configured workflows. These workflows, coupled with the ease of use of the environment, provide even novice next-generation sequencing users with the ability to perform many complex analyses with only a few mouse clicks and, within the context of the same environment, to visualize and further interrogate their results. As a result, this bioinformatics platform is an initial attempt at Empowering the Development of Genomics Expertise (EDGE) in a wide range of applications for microbial research.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2016

A model-based scatter artifacts correction for cone beam CT

Purpose: Due to the increased axial coverage of multislice computed tomography (CT) and the introduction of flat detectors, the size of x-ray illumination fields has grown dramatically, causing an increase in scatter radiation. For CT imaging, scatter is a significant issue that introduces shading artifact, streaks, as well as reduced contrast and Hounsfield Units (HU) accuracy. The purpose of this work is to provide a fast and accurate scatter artifacts correction algorithm for cone beam CT (CBCT) imaging. Methods: The method starts with an estimation of coarse scatter profiles for a set of CBCT data in either image domain or projection domain. A denoising algorithm designed specifically for Poisson signals is then applied to derive the final scatter distribution. Qualitative and quantitative evaluations using thorax and abdomen phantoms with Monte Carlo (MC) simulations, experimental Catphan phantom data, and in vivo human data acquired for a clinical image guided radiation therapy were performed. Scatter correction in both projection domain and image domain was conducted and the influences of segmentation method, mismatched attenuation coefficients, and spectrum model as well as parameter selection were also investigated. Results: Results show that the proposed algorithm can significantly reduce scatter artifacts and recover the correct HU in either projection domain or image domain. For the MC thorax phantom study, four-components segmentation yields the best results, while the results of three-components segmentation are still acceptable. The parameters (iteration number K and weight β) affect the accuracy of the scatter correction and the results get improved as K and β increase. It was found that variations in attenuation coefficient accuracies only slightly impact the performance of the proposed processing. For the Catphan phantom data, the mean value over all pixels in the residual image is reduced from −21.8 to −0.2 HU and 0.7 HU for projection domain and image domain, respectively. The contrast of the in vivo human images is greatly improved after correction. Conclusions: The software-based technique has a number of advantages, such as high computational efficiency and accuracy, and the capability of performing scatter correction without modifying the clinical workflow (i.e., no extra scan/measurement data are needed) or modifying the imaging hardware. When implemented practically, this should improve the accuracy of CBCT image quantitation and significantly impact CBCT-based interventional procedures and adaptive radiation therapy.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Developing a complete AI-accelerated workflow for superconductor discovery

The quest to identify new superconducting materials with enhanced properties is hindered by the prohibitive cost of computing electron-phonon spectral functions, severely limiting the materials space that can be explored. Here, we introduce a Bootstrapped Ensemble of Equivariant Graph Neural Networks (BEE-NET), a machine-learning model trained to predict the Eliashberg spectral function and superconducting critical temperature with a mean-absolute-error of 0.87 K relative to DFT-based Allen-Dynes calculations. Intriguingly, BEE-NET achieves a true-negative-rate of 99.4%, enabling highly efficient screening for the rare property of superconductivity. Integrated into a multi-stage, AI-accelerated discovery pipeline that incorporates elemental-substitution strategies and machine-learned interatomic potentials, our workflow reduced over 1.3 million candidate structures to 741 dynamically and thermodynamically stable compounds with DFT-confirmed T c > 5 K. We report the successful synthesis and experimental confirmation of superconductivity in two of these previously unreported compounds. This study establishes a data-driven framework that integrates machine learning, quantum calculations, and experiments to systematically accelerate superconductor discovery.

Gibson, Jason B. [Quantum Formatics, Cambridge, MA↗
DOE OSTI2016

Component-based workflows for parallel thermomechanical analysis of arrayed geometries

Component-based simulation workflows can increase the agility of the design process by streamlining adaptation of new simulation methods. Here, we present one such workflow for parallel unstructured mesh-based simulations and demonstrate its usefulness in the thermomechanical analysis of an array of solder joints used in microelectronics fabrication. We automate the simulation process from problem specification to the solution of the underlying PDEs, including problem setup, domain definition, and mesh generation. We establish the utility of the proposed approach by demonstrating that qualitatively different stress concentrations are seen in solder joints near the center of such an array and a solder joint seen at the edge of the same array.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

ppdx : Automated modeling of protein–protein interaction descriptors for use with machine learning

This paper describes ppdx, a python workflow tool that combines protein sequence alignment, homology modeling, and structural refinement, to compute a broad array of descriptors for characterizing protein–protein interactions. The descriptors can be used to predict various properties of interest, such as protein–protein binding affinities, or inhibitory concentrations (IC 50 ), using approaches that range from simple regression to more complex machine learning models. The software is highly modular. It supports different protocols for generating structures, and 95 descriptors can be currently computed. More protocols and descriptors can be easily added. The implementation is highly parallel and can fully exploit the available cores in a single workstation, or multiple nodes on a supercomputer, allowing many systems to be analyzed simultaneously. As an illustrative application, ppdx is used to parametrize a model that predicts the IC 50 of a set of antigens and a class of antibodies directed to the influenza hemagglutinin stalk.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

High-Energy X-ray Tomographic Analysis of Precursor Metal Powders (Ti-6Al-4V) Used for Additive Manufacturing

We utilized high-energy x-ray tomography to characterize Ti-6Al-4V metal powders (both as-received and recycled) used in powder-bed additive manufacturing process. The image processing workflow was developed to process and analyze large amount of data objectively by computer program. The distribution of size and shape of the metal particles as well as defect (mainly porosity) inside the particles was analyzed with the statistical representation and resolution in micrometer. The result revealed that circular-shaped porosity with various sizes could be embedded in the powder particles. These porosities could potentially be transferred to the 3D printed part and critically affect the mechanical performance of the component. The present study shows the effectiveness of characterizing metal powders using x-ray imaging techniques where sufficient number of particles can be sampled within tens of minutes with a minimum sample preparation and high accuracy. Clear structural differences in the as-received and recycled powders were delineated that helps in determining the feasibility of using the recycled powders.

36 MATERIALS SCIENCE↗