Search NASA⌕ Search

SEARCH · Search NASA

Results for “computational workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Deep Learning for Enhancing Fault Tolerant Capabilities of Scientific Workflows

Deep Learning for Enhancing Fault TolerantCapabilities of Scientific WorkflowsAlok Singh, Ilkay AltintasSan Diego Supercomputer Center, UCSDLa Jolla, CA, USA{a1singh, ialtintas}@ucsd.eduMalachi Schram, Nathan TallentPacific Northwestern National LaboratoryRichland, WA, USA{malachi.schram, nathan.tallent}@pnnl.govAbstract-In the history of Computer Science, the act of ‘delegation’ has been the greatest multiplier of society’s problem-solving ability. A scientist working on detecting anomalies in a phenomenon, does not need to re-invent matrix multiplication techniques to solve her problem. Scientific workflows provide ultimate ‘delegation’ mechanism -where a domain scientist can completely forget the specifics of ‘how’ her program will execute on a large cluster in an efficient and cost-effective manner and can instead focus on the mathematical formulation and theoretical robustness of her solution. We present here an approach that directly aims to make the execution of Scientific Workflows more reliable, robust and efficient. We aim that the work presented in this paper will propel the larger effort, from the scientific workflow community, of making scientific workflow execution as simple, efficient and robust as a JOIN operation in a modern database. Specifically, we apply Deep Learning techniques to develop a mechanism that forecasts the final state (success or failure) of a dynamic job in a large-scale particle physics experiment, with minimal data gathering, and as early as possible in job’s life cycle. The key advantage of having a predictive mechanism to identify and anticipate failure-prone jobs is the potential for designing intelligent Fault Tolerance mechanisms to handle anomalous events. We achieve a 14% improvement in computational resources utilization, and an overall classification accuracy of 85% on real tasks executed in a High Energy Physics Computing workflow.To the best of our knowledge, this is the most exhaustive and first of its kind study of neural network architectures in context of a real-dataset profiled from a large-scalescientific workflow.

deep learning↗
DOE OSTI2013

Virtualized Network Control (VNC)

The focus of this project was on the development of a "Network Service Plane" as an abstraction model for the control and provisioning of multi-layer networks. The primary motivation for this work were the requirements of next generation networked applications which will need to access advanced networking as a first class resource at the same level as compute and storage resources. A new class of "Intelligent Network Services" were defined in order to facilitate the integration of advanced network services into application specific workflows. This new class of network services are intended to enable real-time interaction between the application co-scheduling algorithms and the network for the purposes of workflow planning, real-time resource availability identification, scheduling, and provisioning actions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Automated metadata--final project report

This report summarizes the work of the Automated Metadata, Provenance Cataloging, and Navigable Interfaces: Ensuring the Usefulness of Extreme-Scale Data Project (MPO Project) funded by the United States Department of Energy (DOE), Offices of Advanced Scientific Computing Research and Fusion Energy Sciences. Initially funded for three years starting in 2012, it was extended for 6 months with additional funding. The project was a collaboration between scientists at General Atomics, Lawrence Berkley National Laboratory (LBNL), and Massachusetts Institute of Technology (MIT). The group leveraged existing computer science technology where possible, and extended or created new capabilities where required. The MPO project was able to successfully create a suite of software tools that can be used by a scientific community to automatically document their scientific workflows. These tools were integrated into workflows for fusion energy and climate research illustrating the general applicability of the project’s toolkit. Feedback was very positive on the project’s toolkit and the value of such automatic workflow documentation to the scientific endeavor.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2016

Feasibility of MRI-only treatment planning for proton therapy in brain and prostate cancers: Dose calculation accuracy in substitute CT images

Purpose: Magnetic resonance imaging (MRI) is increasingly used for radiotherapy target delineation, image guidance, and treatment response monitoring. Recent studies have shown that an entire external x-ray radiotherapy treatment planning (RTP) workflow for brain tumor or prostate cancer patients based only on MRI reference images is feasible. This study aims to show that a MRI-only based RTP workflow is also feasible for proton beam therapy plans generated in MRI-based substitute computed tomography (sCT) images of the head and the pelvis. Methods: The sCTs were constructed for ten prostate cancer and ten brain tumor patients primarily by transforming the intensity values of in-phase MR images to Hounsfield units (HUs) with a dual model HU conversion technique to enable heterogeneous tissue representation. HU conversion models for the pelvis were adopted from previous studies, further extended in this study also for head MRI by generating anatomical site-specific conversion models (a new training data set of ten other brain patients). This study also evaluated two other types of simplified sCT: dual bulk density (for bone and water) and homogeneous (water only). For every clinical case, intensity modulated proton therapy (IMPT) plans robustly optimized in standard planning CTs were calculated in sCT for evaluation, and vice versa. Overall dose agreement was evaluated using dose–volume histogram parameters and 3D gamma criteria. Results: In heterogeneous sCTs, the mean absolute errors in HUs were 34 (soft tissues: 13, bones: 92) and 42 (soft tissues: 9, bones: 97) in the head and in the pelvis, respectively. The maximum absolute dose differences relative to CT in the brain tumor clinical target volume (CTV) were 1.4% for heterogeneous sCT, 1.8% for dual bulk sCT, and 8.9% for homogenous sCT. The corresponding maximum differences in the prostate CTV were 0.6%, 1.2%, and 3.6%, respectively. The percentages of dose points in the head and pelvis passing 1% and 1 mm gamma index criteria were over 91%, 85%, and 38% with heterogeneous, dual bulk, and homogeneous sCTs, respectively. There were no significant changes to gamma index pass rates for IMPT plans first optimized in CT and then calculated in heterogeneous sCT versus IMPT plans first optimized in heterogeneous sCT and then calculated on standard CT. Conclusions: This study demonstrates that proton therapy dose calculations on heterogeneous sCTs are in good agreement with plans generated with standard planning CT. An MRI-only based RTP workflow is feasible in IMPT for brain tumors and prostate cancers.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

MPI jobs within MPI jobs: A practical way of enabling task-level fault-tolerance in HPC workflows

While the use of workflows for HPC is growing, MPI interoperability remains a challenge for workflow management systems. The MPI standard and/or its implementations provide a number of ways to build multiple-programs-multiple-data (MPMD) applications. These methods present limitations related to fault tolerance, and are not easy to use. In this paper, we advocate for a novel MPI_Comm_launch function acting as the parallel counterpart of a system(3) call. MPI_Comm_launch allows a child MPI application to be launched inside the resources originally held by processes of a parent MPI application. Two important aspects of MPI_Comm_launch is that it pauses the calling process, and runs the child processes on the parent’s CPU cores, but in an isolated manner with respect to memory. This function makes it easier to build MPMD applications with well-decoupled subtasks. We show how this feature can provide better flexibility and better fault tolerance in ensemble simulations and HPC workflows. Here, we report results showing 2x throughput improvement for application workflows with faults, and scaling results for challenging workloads up to 256 nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deployment of inference as a service at the US CMS Tier-2 data centers

Coprocessors, especially GPUs, will be a vital ingredient of data production workflows at the HL-LHC. At CMS, the GPU-as-a-service approach for production workflows is implemented by the SONIC project (Services for Optimized Network Inference on Coprocessors). SONIC provides a mechanism for outsourcing computationally demanding algorithms, such as neural network inference, to remote servers, where requests from multiple clients are intelligently distributed across multiple GPUs by a load-balancing service. This talk highlights the recent progress in deploying SONIC at selected U.S. CMS Tier-2 data centers. Using realistic CMS Run3 data processing workflows, such as those containing transformer-based algorithms, we demonstrate how SONIC is integrated into the production-like environment to enable accelerated inference offloading. We will present developments from both the client and server sides, including production job and data center configurations for NVIDIA and AMD GPUs. We will also present performance scaling benchmarks and discuss the challenges of operating SONIC in CMS production, such as server discovery, GPU saturation, fallback server logic, etc.

Holzman, Burt↗
DOE OSTI2025

Performance analysis and data reduction for exascale scientific workflows

Chimbuko is the first in situ, scalable, workflow-level performance analysis tool for trace-level analysis and visualization of application performance. This tool was developed by the Co-design Center for Online Data Analysis and Reduction and funded by the U.S. Department of Energy’s Exascale Computing Project. We provide a detailed description of Chimbuko’s architecture and illustrate our online and offline visualization with multiple use cases. We also present results for the deployment and scalability of the tool as applied to a high-energy physics workflow running at large scale on the Frontier supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

express: Extensible, high-level workflows for swifter ab initio materials modeling

In this work, we introduce an open-source Julia project, express, an extensible, lightweight, high-throughput, high-level workflow framework that aims to automate ab initio calculations for the materials science community. express is shipped with well-tested workflow templates, including structure optimization, equation of state (EOS) fitting, phonon spectrum (lattice dynamics) calculation, and thermodynamic property calculation in the framework of the quasi-harmonic approximation (QHA). It is designed to be highly modularized so that its components can be reused across various occasions, and customized workflows can be built on top of that. Users can also track the status of workflows in real-time, and rerun failed jobs thanks to the data lineage feature express provides. Finally, two working examples, i.e., all workflows applied to lime and akimotoite, are also presented in the code and this paper.

36 MATERIALS SCIENCE↗
DOE OSTI2023

ESnet Requirements Review Program Through the IRI Lens: A Meta-Analysis of Workflow Patterns Across DOE Office of Science Programs (Final Report)

The Department of Energy (DOE) ensures America’s security and prosperity by addressing its energy, environmental, and nuclear challenges through transformative science and technology solutions. The DOE’s Office of Science (SC) delivers groundbreaking scientific discoveries and major scientific tools that transform our understanding of nature and advance the energy, economic, and national security of the United States. The SC’s programs advance DOE mission science across a wide range of disciplines and have developed the research infrastructure needed to remain at the forefront of scientific discovery. The DOE SC’s world-class research infrastructure — exemplified by the 28 SC scientific user facilities — provides the research community with premier observational, experimental, computational, and network capabilities. Each user facility is designed to provide unique capabilities to advance core DOE mission science for its sponsor SC program and to stimulate a rich discovery and innovation ecosystem. Research communities gather and flourish around each user facility, bringing together diverse perspectives. A hallmark of many facilities is the large population of students, postdoctoral researchers, and early-career scientists who contribute as full-fledged users. These facility staff and users collaborate over years to devise new approaches to utilizing the user facility’s core capabilities. The history of the SC user facilities has many examples of wildly inventive researchers challenging operational orthodoxy to pioneer new vistas of discovery; for example, the use of the synchrotron X-ray light sources for study of proteins and other large biological molecules. This continual reinvention of the practice of science — as users and staff forge novel approaches expressed in research workflows — unlocks new discoveries and propels scientific progress. Within this research ecosystem, the high-performance computing (HPC) and networking user facilities stewarded by SC’s Advanced Scientific Computing Research (ASCR) program play a dynamic cross-cutting role, enabling complex workflows demanding high performance data, networking, and computing solutions. The DOE SC’s three HPC user facilities and the Energy Sciences Network (ESnet) high-performance research network serve all of the SC’s programs as well as the global research community. Argonne Leadership Computing Facility (ALCF), the National Energy Research Scientific Computing Center (NERSC), and Oak Ridge Leadership Computing Facility (OLCF) conceive, build, and provide access to a range of supercomputing, advanced computing, and large-scale data-infrastructure platforms, while ESnet interconnects DOE SC research infrastructure and enables seamless exchange of scientific data. All four facilities operate testbeds to expand the frontiers of computing and networking research. Together, the ASCR facilities enterprise seeks to understand and meet the needs and requirements across SC and DOE domain science programs and priority efforts, highlighted by the formal requirements reviews (RRs) methodology. In recent years, the research communities around the SC user facilities have begun experimenting with and demanding solutions integrated with HPC and data infrastructure. This rise of integrated-science approaches is documented in many community and high-level government reports. At the dawn of the era of exascale science and the acceleration of artificial intelligence (AI) innovation, there is a broad need for integrated computational, data, and networking solutions. In response to these drivers, DOE has developed a vision for an Integrated Research Infrastructure (IRI): To empower researchers to meld DOE’s world-class research tools, infrastructure, and user facilities seamlessly and securely in novel ways to radically accelerate discovery and innovation.

42 ENGINEERING↗
DOE OSTI2026

Multibody for Everybody (M4E) - A Linearization Approach to Enable Frequency Domain Analysis, Time Integration and Control Co-Design

1.1 Background/Objectives: Marine energy represents a promising yet underexploited source of power. To increase the harvested power, significant efforts have been made to improve wave energy converter (WEC) modeling capabilities and optimize power take-off (PTO) performance; however, these efforts have often treated WEC dynamics, PTO design, and controller development sequentially. In contrast, control co-design (CCD) is emerging as a promising strategy to address these issues directly, creating a growing need for fast analysis tools suitable for repeated simulation and parametric studies [1]. To support this need, this work presents the Multibody for Everybody (M4E) [2] linearization module, which employs a symbolic toolbox to provide deeper insight of WEC design parameters. The objective is to demonstrate that a minimal-coordinate linearization of articulated WEC dynamics can provide accurate wave response predictions and substantial computational savings relative to nonlinear time-domain simulation, while preserving compatibility with broader wave-energy analysis workflows, enabling CCD. 1.2 Approach/Activities: The proposed approach linearizes the equations of motion, generated by M4E, in minimal coordinates about a selected operating point and combines the resulting system with frequencydomain hydrodynamic terms to incorporate the reduced mass, damping, stiffness, and forcing operators. The linearized model is used for both impedance-based response amplitude operator (RAO) prediction and rapid regular-wave time integration. The methodology is demonstrated on a single-flap device and a FOSWEC configuration, with linearized M4E responses compared against the corresponding nonlinear M4E simulations and WEC-Sim results. Regular-wave time histories, RAO trends, and runtime differences are assessed. The framework is also compatible with broader wave-energy workflows, including coupling to WecOptTool, although that capability is not the focus of this work [3]. 1.3 Results/Lessons: The linearized M4E model reproduces key regularwave response characteristics such as integration and Response Amplitude over multiple frequencies. This module matches nonlinear M4E and WEC-Sim results while substantially reducing integration cost. Thus, the proposed framework can serve as a rapid analysis layer for articulated WEC design, parameter studies, and controls-oriented workflows. The analysis is most appropriate in the near-equilibrium regime, about the linearization point.

16 TIDAL AND WAVE POWER↗
DOE OSTI2026

Wolf

The Workflow Orchestration Language Framework (WOLF) is an agentic framework grounded in natural language with an architecture inspired by reinforcement learning (RL)—designed to orchestrate, scale, and accelerate complex workflows. The concept of WOLF was born out of the very successful ASC Tri-lab Multi-Agent Design Assistant (MADA) project, but extends beyond its domain-specific design agents to provide a more general and extensible architecture. WOLF capitalizes on the lessons learned from MADA and is fully aligned with Sutton’s The Bitter Lesson—that the most enduring progress in AI comes from general-purpose methods that scale with computation, rather than narrow techniques built on domain-specific human knowledge. In this spirit, WOLF enables agents to autonomously learn workflows, capture strategies as reusable playbooks, and build a growing corpus of interpretable, auditable “wisdom artifacts.” These artifacts, expressed in natural language, bridge human and machine understanding while preserving adaptability and scalability as computational power continues to expand.

Boureima, Ismaeal↗
DOE OSTI2024

High-throughput determination of Hubbard $U$ and Hund $J$ values for transition metal oxides via the linear response formalism

DFT+U provides a convenient, cost-effective correction for the self-interaction error (SIE) that arises when describing correlated electronic states using conventional approximate density functional theory (DFT). The success of a DFT+U(+J) calculation hinges on the accurate determination of its Hubbard U and Hund J parameters, and the linear response (LR) methodology has proven to be computationally effective and accurate for calculating these parameters. This study provides a high-throughput computational analysis of the U and J values for transition metal d-electron states in a representative set of over 1000 magnetic transition metal oxides (TMOs), providing a frame of reference for researchers who use DFT+U to study transition metal oxides. In order to perform this high-throughput study, an ATOMATE workflow is developed for calculating U and J values automatically on massively parallel supercomputing architectures. Here, to demonstrate an application of this workflow, the spin-canting magnetic structure and unit cell parameters of the multiferroic olivine LiNiPO4 are calculated using the computed Hubbard U and Hund J values for Ni-d and O-p states, and are compared with experiment. Both the Ni-d U and J corrections have a strong effect on the Ni-moment canting angle. Additionally, including a O-pU value results in a significantly improved agreement between the computed lattice parameters and experiment

36 MATERIALS SCIENCE↗
DOE OSTI2026

DTLMod: A simulation framework for in situ workflow optimization

In situ processing workflows have become essential for coping with the explosion in data volume and velocity in large-scale scientific computing, providing domain scientists with early insights at runtime. Multiple frameworks implement this paradigm through a data transport layer (DTL), offering different data access modes and deployment schemes, but researchers currently lack the appropriate tools to assess design and deployment options before committing to costly real experiments. We introduce DTLMod, an open-source simulated DTL that enables performance evaluation of in situ workflow configurations at scale. Built on SimGrid, it links into any SimGrid-based simulator and is available in C++ and Python. We evaluate DTLMod along four axes: scalability (tens of thousands of simulated processes across interconnected clusters in seconds, with linear memory scaling), versatility (three implementation variants trading fidelity for speed), accuracy (simulated times faithfully reflecting real behavior), and practical utility (two use cases demonstrating evidence-based workflow design decisions).

Suter, Fred [ORNL] (ORCID:0000000319021955)↗
DOE OSTI2020

Multilevel and multifidelity uncertainty quantification for cardiovascular hemodynamics

Standard approaches for uncertainty quantification in cardiovascular modeling pose challenges due to the large number of uncertain inputs and the significant computational cost of realistic three-dimensional simulations. Here, we propose an efficient uncertainty quantification framework utilizing a multilevel multifidelity Monte Carlo (MLMF) estimator to improve the accuracy of hemodynamic quantities of interest while maintaining reasonable computational cost. This is achieved by leveraging three cardiovascular model fidelities, each with varying spatial resolution to rigorously quantify the variability in hemodynamic outputs. We employ two low-fidelity models (zero- and one-dimensional) to construct several different estimators. Our goal is to investigate and compare the efficiency of estimators built from combinations of these two low-fidelity model alternatives and our high-fidelity three-dimensional models. We demonstrate this framework on healthy and diseased models of aortic and coronary anatomy, including uncertainties in material property and boundary condition parameters. Our goal is to demonstrate that for this application it is possible to accelerate the convergence of the estimators by utilizing a MLMF paradigm. Therefore, we compare our approach to single fidelity Monte Carlo estimators and to a multilevel Monte Carlo approach based only on three-dimensional simulations, but leveraging multiple spatial resolutions. We demonstrate significant, on the order of 10 to 100 times, reduction in total computational cost with the MLMF estimators. We also examine the differing properties of the MLMF estimators in healthy versus diseased models, as well as global versus local quantities of interest. As expected, global quantities such as outlet pressure and flow show larger reductions than local quantities, such as those relating to wall shear stress, as the latter rely more heavily on the highest fidelity model evaluations. Similarly, healthy models show larger reductions than diseased models. In all cases, our workflow coupling Dakota’s MLMF estimators with the SimVascular cardiovascular modeling framework makes uncertainty quantification feasible for constrained computational budgets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

AI/ML Expo Boosting Job Performance with AI: Innovative Approaches and Success Stories

Our technology leverages artificial intelligence (AI) to enhance the user experience in High Performance Computing (HPC) environments. By analyzing user behavior and providing personalized recommendations, our AI system helps HPC users optimize their workflows and improve productivity. Additionally, we offer an advanced image similarity search feature, which utilizes AI algorithms to identify and retrieve visually similar images, saving users valuable time and effort in their research and analysis.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Bayesian calibration of coarse-grained forces: Efficiently addressing transferability

Generating and calibrating forces that are transferable across a range of state-points remains a challenging task in coarse-grained (CG) molecular dynamics. In this work, we present a coarse-graining workflow, inspired by ideas from uncertainty quantification and numerical analysis, to address this problem. The key idea behind our approach is to introduce a Bayesian correction algorithm that uses functional derivatives of CG simulations to rapidly and inexpensively recalibrate initial estimates f{sub 0} of forces anchored by standard methods such as force-matching. Taking density-temperature relationships as a running example, we demonstrate that this algorithm, in concert with various interpolation schemes, can be used to efficiently compute physically reasonable force curves on a fine grid of state-points. Importantly, we show that our workflow is robust to several choices available to the modeler, including the interpolation schemes and tools used to construct f{sub 0}. In a related vein, we also demonstrate that our approach can speed up coarse-graining by reducing the number of atomistic simulations needed as inputs to standard methods for generating CG forces.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2017

Process-Structure Linkages Using a Data Science Approach: Application to Simulated Additive Manufacturing Data

A novel data science workflow is developed and demonstrated to extract process-structure linkages (i.e., reduced-order model) for microstructure evolution problems when the final microstructure depends on (simulation or experimental) processing parameters. Our workflow consists of four main steps: data pre-processing, microstructure quantification, dimensionality reduction, and extraction/validation of process-structure linkages. These methods that can be employed within each step vary based on the type and amount of available data. In this paper, this data-driven workflow is applied to a set of synthetic additive manufacturing microstructures obtained using the Potts-kinetic Monte Carlo (kMC) approach. Additive manufacturing techniques inherently produce complex microstructures that can vary significantly with processing conditions. Using the developed workflow, a low-dimensional data-driven model was established to correlate process parameters with the predicted final microstructure. In addition, the modular workflows developed and presented in this work facilitate easy dissemination and curation by the broader community.

36 MATERIALS SCIENCE↗