Search NASA⌕ Search

SEARCH · Search NASA

Results for “computational workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Earth Science Data Processing With Nextflow

Earth science data processing tasks present many challenges. These tasks often process large input datasets and require scores of CPU-hours to generate results. All but the simplest tasks will be decomposed into a series of computational or data manipulation steps, also known as a scientific workflow. In order to reduce the burden of orchestrating and running the dependent processing steps, a workflow execution engine is required. This poster describes the lessons learned by the CLARREO Pathfinder (CPF) team while developing multiple scientific workflows and utilizing the open-source Nextflow engine to execute them in a cloud computing environment. The Nextflow engine is designed with the following stated goals: first, the engine does not dictate how individual steps in the task are implemented (i.e. it is language and interface agnostic); second, the engine supports easy configuration and modularity at the workflow level so that others can easily execute our workflows to reproduce results; lastly, the engine eases development by transparently scaling execution from local to remote environments. Nextflow was developed for the bioinformatics domain but is a good fit for other scientific workflows where the overall task is well-described by a dataflow diagram. The CPF team has developed Nextflow pipelines (i.e. scientific workflows) to simulate CLARREO radiance, generate large look-up tables for inter-calibration algorithms, and generate L4 intercalibration data products. These pipelines consume from single-digits to hundreds of thousands of CPU-hours. In the development and evolution of these pipelines we have discovered many design patterns, pitfalls, and solutions to common problems. Our goal is to demonstrate important aspects of how to design, implement, run, and ultimately share Nextflow pipelines in the domain of Earth science.

Aron D Bartle↗
DOE OSTI2017

Bellerophon Environment for Analysis of Materials (BEAM)

Improvements in scientific instrumentation allow imaging at mesoscopic to atomic length scales, many spectroscopic modes, and now--with the rise of multimodal acquisition systems and the associated processing capability--the era of multidimensional, informationally dense data sets has arrived. Technical issues in these combinatorial scientific fields are exacerbated by computational challenges best summarized as a necessity for drastic improvement in the capability to transfer, store, and analyze large volumes of data. The Bellerophon Environment for Analysis of Materials (BEAM) platform provides material scientists the capability to directly leverage the integrated computational and analytical power of High Performance Computing (HPC) to perform scalable data analysis and simulation and manage uploaded data files via an intuitive, cross-platform client user interface. This framework delivers authenticated, "push-button" execution of complex user workflows that deploy data analysis algorithms and computational simulations utilizing compute-and-data cloud infrastructures and HPC environments like Titan at the Oak Ridge Leadershp Computing Facility (OLCF).

Lingerfelt, Eric J↗
DOE OSTI2023

Automated generation of scientific workflow generators with WfChef

Scientific workflow applications have gained significant importance, and their automated and efficient execution on large-scale computing platforms has been the subject of extensive research and development. For these efforts to be successful, a solid experimental methodology is needed to evaluate workflow algorithms and systems. A foundation for this methodology is the availability of realistic workflow instances. Although public repositories provide workflow instances for a few scientific applications, these are limited in scope, and workflow instances are not available for all application scales of interest. To address this limitation, previous work has developed generators of synthetic workflow instances of arbitrary scales. Despite being popular, the implementation of these generators is a manual and labor-intensive process that requires expert application knowledge. As a result, these generators only target a handful of applications, even though there are hundreds of workflow applications in production. Here, we introduce WfChef , a fully automated framework for constructing a synthetic workflow generator for any scientific application. Based on an input set of workflow instances for a particular application, WfChef automatically produces a synthetic workflow generator. To measure the realism of the generated workflows, we define and evaluate several metrics. Using these metrics, we compare the realism of the workflows generated by WfChef generators to that of the workflows generated by the previously available, hand-crafted generators. We find that WfChef generators not only require zero development effort (because they are automatically produced), but also generate workflows that are more realistic than those generated by hand-crafted generators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design

The Joint Automated Repository for Various Integrated Simulations (JARVIS) is an integrated infrastructure to accelerate materials discovery and design using density functional theory (DFT), classical force-fields (FF), and machine learning (ML) techniques. JARVIS is motivated by the Materials Genome Initiative (MGI) principles of developing open-access databases and tools to reduce the cost and development time of materials discovery, optimization, and deployment. The major features of JARVIS are: JARVIS-DFT, JARVIS-FF, JARVIS-ML, and JARVIS-tools. To date, JARVIS consists of ≈40,000 materials and ≈1 million calculated properties in JARVIS-DFT, ≈500 materials and ≈110 force-fields in JARVIS-FF, and ≈25 ML models for material-property predictions in JARVIS-ML, all of which are continuously expanding. JARVIS-tools provides scripts and workflows for running and analyzing various simulations. We compare our computational data to experiments or high-fidelity computational methods wherever applicable to evaluate error/uncertainty in predictions. In addition to the existing workflows, the infrastructure can support a wide variety of other technologically important applications as part of the data-driven materials design paradigm. The JARVIS datasets and tools are publicly available at the website: https://jarvis.nist.gov .

36 MATERIALS SCIENCE↗
DOE OSTI2024

Accelerating Multivariate Functional Approximation Computation with Domain Decomposition Techniques⋆

Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain -decomposed method for computing the MFA with B -spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees -of -freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high -order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C 0 or at best C 1 continuity for arbitrary B -spline degree expansions, or those that require post -processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real -world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest -neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.

additive Schwarz solvers↗
DOE OSTI2025

Atomate2: modular workflows for materials science

High-throughput density functional theory (DFT) calculations have become a vital element of computational materials science, enabling materials screening, property database generation, and training of “universal” machine learning models. While several software frameworks have emerged to support these computational efforts, new developments such as machine learned force fields have increased demands for more flexible and programmable workflow solutions. This manuscript introduces atomate2, a comprehensive evolution of our original atomate framework, designed to address existing limitations in computational materials research infrastructure. Key features include the support for multiple electronic structure packages and interoperability between them, along with generalizable workflows that can be written in an abstract form irrespective of the DFT package or machine learning force field used within them. Our hope is that atomate2's improved usability and extensibility can reduce technical barriers for high-throughput research workflows and facilitate the rapid adoption of emerging methods in computational material science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Performance Analysis of Large Scale HPC Workflows for Earth System Models

Climate assessment reports, such as the Intergovernmental Panel on Climate Change’s 5th assesment report [IPCC AR5; IPCC et al., 2014] or the United States National Climate Assessment [U.S. Global Change Research Program, 2014], increasingly rely on many-ensemble climate projections from fully-coupled Earth system models (ESM) to asses possible future climate states. ESM participate in a large set of Coupled Model Intercomparson Projects (CMIP) to better understand their models and the climate system, so that they may provide credible climate projections. However, fully participating in these intercomparison exercises represents an extremely large computational personnel burden for modeling centers [Eyring et al., 2016, Williams et al., 2016a]. However, there has been little attention within the climate modeling community paid to the in situ performance of ESMs up until now. Recently a computational performance model intercomparison project (CPMIP) for CMIP6 has begun [Balaji et al., 2017], but there has not yet been an effort to gather, process, and analyze the historical performance data that has been saved within archived log files from previous modeling efforts, nor the development of performance analysis system capable of providing detailed statistical analyses or visualizations of this performance data. Therefore, we’ve begun a joint effort to analyze the historical and current performance data on both GFDL and OLCF modeling systems. Here, we have developed a preliminary performance analysis system to captue, analyize, and visualize end-to-end in situ performance data and applied it to NOAA/GFDL’s post-processing system. Through the course of the project a number of performance improvements have been made, with the steps necessary to handle the expected volume of data (Big Data) likely to come out of the CMIP6 exercises (TB to PB) provided. We have also begun developing a statistical workload model that will allow modeling centers to determined their expected scientific throughput in the context of expected resource (computational and personnel) availability and accounting for typical workflow disruptions. This will allow them to develop a CMIP participation strategy and maximize their level of participation in the exercises. We expect this infrastructure, and the data it produces, to ultimately allow us to identify performance issues and workflow errors that exist across Earth system modeling centers, and possibly HPC modeling centers in general. Center-independent solutions to workflow or performance issues would help alleviate the computational burden for CMIP6+ modeling projects and should realize both labor and cost savings benefits throughout the CMIP community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Probability of Pinching Failure in PWR SNF after Long-Term Storage

A probabilistic assessment was performed on pinching failure of PWR SNF rod cladding under accidental SNF cask transport conditions. a set of 3,000 rod pinching scenarios was analyzed, with different geometrical, material, and loading parameters, using a Latin Hypercube Sampling (LHS). The investigated accidents include train derailments leading to cask collisions or cask drops. Pinching failure was related to cladding offset strain, defined as the plastic deformation interval in radial cladding direction normalized with respect to the outer cladding diameter. Finite element (FE) models were used to compute SNF rod deformations and the offset strain demand OSD in each of the assessed pinching scenarios. The numerical models account for the contribution of the degraded fuel pellets to rod pinching stiffness. The offset strain capacity OSC was computed using regression and interpolation models based on ring compression (RCT) test results, and account for hydride-related effects on cladding ductility. The results show a low probability of pinching failure. Hydride-related cladding embrittlement affects the structural integrity of PWR SNF rods after long-term dry storage of up to 300 years. Radial hydride patterns or large amounts of circumferentially oriented hydrides reduce the cladding ductile deformation capability when pinched at low material temperatures. The probability of hydride-related pinching failure of PWR cladding is estimated after long-term dry storage to address: How do PWR SNF rod conditions like hydrogen content, vacuum drying peak cladding hoop stress σ{sub θ}, and cladding material temperature affect cladding ductility? Is hydride-related cladding embrittlement a material degradation mechanism that affects the structural integrity of PWR SNF rods under pinching loading? What are the controlling parameters for deformation demands in PWR SNF rods under pinching loading? Is the pinching load amplitude an important parameter that controls the probability of pinching failure? How do parameters like pellet-cladding gap or fuel pellet degradation affect the pinching loading capacity of PWR SNF rods? Statistical modelling of PWR SNF rod conditions in U.S. inventory: A pseudo rod database was generated based on EIA database. Analysis of train accident scenarios: a probability density function (pdf) of a pinching load distribution P{sub 9m} of a 9-m cask drop is linearly scaled to generate a more representative: distribution P{sub AS}. The new distribution considers train collision velocities, cask drops for on- and off-bridge accidents, and cask impact orientation. LHS generates 3,000 rod pinching scenarios based on nine independent geometrical, material, and loading parameters. Probabilistic FE Analysis: The LHS realizations are used to generate 3,000 FE models. The FE analysis is conducted for P{sub 9m}, and P{sub AS}. The workflow is automatized. The OSC is computed with regression and interpolation models, as a function of the hydrogen content C{sub H}, vacuum drying peak σ{sub θ}, and SNF temperature T{sub FR}: The regression data was obtained in RCTs on PWR cladding. The OSD is derived from the cladding deformations in the FE models. Cladding failure was identified via a comparison of OSC and OSD. A 300 and a 30 years-of-dry-storage duration is investigated: The dry-storage period affects the expected SNF temperature and consequently, the OSC and the OSD. The material temperature LHS population is adapted to account for a SNF temperature variation. The probability of hydride-related, ductile cladding failure during a severe transportation accident is not significantly affected by the pinching load amplitude: The probability is slightly reduced from a range of less than 0.0333% to 0.1667% for P{sub 9m}, to a range of less than 0.0333% to 0.1333% for P{sub AS}, depending on the vacuum drying peak cladding hoop stress modelling assumptions. Further, the SNF dry-storage duration does not significantly affect the probability of ductile pinching failure. Based on available train accidents data, the pinching failure probability for a 5,000 km long transportation route ranges from less than 1.62 E-7 to a value of 6.47 E-7, depending on the vacuum drying peak cladding hoop stress modelling assumptions. The pinching load amplitude is not a controlling parameter for the probability of ductile cladding pinching failure, and does not control the cladding deformations once pellet-cladding contact is established, The pellet-cladding gap width, however, plays a key role. Fuel pellet degradation only affects the deformation capacity, if the pellet stiffness is significantly reduced (more than 90%). Only highly embrittled cladding is likely to fail. The deformation demands on the cladding under pinching loading are small.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2024

IRI Technology Landscape – A survey of re-usable components and methodologies

This document describes technical implementation details on network access schemes connecting API-driven workflows to supercomputer centers. API-driven workflows are a central theme in connected computing, since they bring the terminal-mainframe' access pattern present since the 1970s up to the task of interfacing with modern web browser technologies. Both security (HTTPS/TLS/IPSec/VPNs/public key cryptography/digital signatures) and network protocol stacks (HTTP-REST APIs, tokens, gRPC, SRTP) have evolved to the point where implementing API-driven workflows is possible using stable, secure off-the-shelf software.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Toward Real-Time Analysis of Synchrotron Micro-Tomography Data: Accelerating Experimental Workflows with AI and HPC

ynchrotron light sources are routinely used to perform imaging experiments. In this paper, we review the relevant computational stages, identify bottlenecks, and highlight future opportunities to streamline data acquisition for experimental microscopy workflows. We demonstrate our preliminary exploration with an end-to-end scientific workflow on Summit based on micro-computed tomography data. Computational elements include: 1) reconstruction of volumetric image data; 2) denoising with deep neural networks; and 3) non-local means based segmentation and quantitative analysis.

Mcclure, James↗
DOE OSTI2026

Computational methods in solution-based plastics purification

Plastic waste can be recycled into resins with near-virgin properties by solution-based purification processes that selectively dissolve polymers, remove contaminants, or detach printing residues. Here, in this review, we examine computational methods for predicting the behavior governing solution-based plastic purification, motivated by the vast polymer–solvent–contaminant compositional space. We discuss thermodynamic and machine learning methods for predicting polymer–solvent and polymer–contaminant interaction and review physics-based molecular dynamics simulations that resolve molecular-scale phenomena within polymer matrices inaccessible to screening methods. We highlight how these methods have informed experimental design for dissolution-based recycling and solvent-based contaminant removal. Finally, we discuss the prospective role of agentic AI in integrating these computational tools with real-time sorting data to adapt purification conditions to the compositional variability of real post-consumer feedstocks. This review charts a path toward computationally guided solution-based purification workflows that can respond to the complexity inherent in plastic waste streams.

Altamimi, Ali [Univ. of Wisconsin, Madison, WI (Un↗
DOE OSTI2026

Beyond interpolation: Physics-inspired gating transformers for extrapolating irradiation conditions to novel nuclear fuels

The qualification of advanced nuclear fuels relies on irradiation experiments in test reactors that emulate commercial conditions. Designing these tests requires accurate prediction of key irradiation quantities, particularly heat generation rate and burnup, yet obtaining them typically involves computationally expensive multi-step simulation workflows. We propose a physics-inspired gating transformer (PIGT) that integrates an inverse-square, distance-based attenuation into the encoder representation to bias attention toward physically relevant spatial relationships while retaining data-driven flexibility. Using MiniFuel irradiation data from the High Flux Isotope Reactor at Oak Ridge National Laboratory, we benchmark against ensemble methods, feedforward and recurrent networks, convolutional models, and standard transformers. While baseline models perform well under interpolation, they exhibit a pronounced generalization gap when evaluated on fuels not included in the training set. The proposed model consistently improves extrapolative accuracy and stability, yielding the strongest performance on unseen fuel configurations. These results indicate that a lightweight physics structure embedded within attention mechanisms can substantially improve robustness, enabling more reliable surrogate predictions to accelerate the design of nuclear fuel irradiation experiments.

Fuel qualification↗
DOE OSTI2016

Atomistic modelling of scattering data in the Collaborative Computational Project for Small Angle Scattering (CCP-SAS)

The capabilities of current computer simulations provide a unique opportunity to model small-angle scattering (SAS) data at the atomistic level, and to include other structural constraints ranging from molecular and atomistic energetics to crystallography, electron microscopy and NMR. This extends the capabilities of solution scattering and provides deeper insights into the physics and chemistry of the systems studied. Realizing this potential, however, requires integrating the experimental data with a new generation of modelling software. To achieve this, the CCP-SAS collaboration (http://www.ccpsas.org/) is developing open-source, high-throughput and user-friendly software for the atomistic and coarse-grained molecular modelling of scattering data. Robust state-of-the-art molecular simulation engines and molecular dynamics and Monte Carlo force fields provide constraints to the solution structure inferred from the small-angle scattering data, which incorporates the known physical chemistry of the system. The implementation of this software suite involves a tiered approach in which GenApp provides the deployment infrastructure for running applications on both standard and high-performance computing hardware, and SASSIE provides a workflow framework into which modules can be plugged to prepare structures, carry out simulations, calculate theoretical scattering data and compare results with experimental data. GenApp produces the accessible web-based front end termed SASSIE-web , and GenApp and SASSIE also make community SAS codes available. Applications are illustrated by case studies: (i) inter-domain flexibility in two- to six-domain proteins as exemplified by HIV-1 Gag, MASP and ubiquitin; (ii) the hinge conformation in human IgG2 and IgA1 antibodies; (iii) the complex formed between a hexameric protein Hfq and mRNA; and (iv) synthetic `bottlebrush' polymers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

AdaParse

SF-25-127AdaParse (Adaptive Parallel PDF Parsing and Resource Scaling Engine) enables scalable, high-accuracy PDF parsing. AdaParse is a data-driven strategy that assigns an appropriate parser to each document, offering high accuracy for any computational budget. Moreover, it offers a workflow of various PDF parsing software that includes extraction tools: PyMuPDF, pypdf traditional OCR: Tesseract, modern OCR (e.g., Vision Transformers): Nougat and Marker

Siebenschuh, Carlo↗
DOE OSTI2020

Predicting Execution Times for Disk-based and In-Situ Parallel Data Analytics (Final Technical Report)

In recent years, there has been a significant amount of interests in in-situ analytics on simulation programs. For a variety of reasons, it is desirable to be able to predict the execution time of an analytics program. At the same time, frameworks such as MapReduce have become popular for scientific data analytics. This paper focuses on developing performance models for predicting execution time of parallel data analytics, with a special emphasis on in-situ analytics. We take two distinct approach towards performance prediction. We first expand SKOPE (a SKeleton framewOrk for Performance Exploration) with performance models for disk data read, cache performance, and page fault penalty. Second, an analytical performance model is also developed. We have evaluated our performance prediction framework as well as the analytical model on three hardware setups with well-known data mining algorithms implemented in three programming paradigms, MapReduce, MATE (a MapReduce-like parallel system with an alternate API for multi-core environments) and Smart (a MapReduce-like framework for in-situ analytics). Results show that our performance prediction framework along with the incorporated performance models are capable of accurately predicting execution times for parallel scientific analytics on different hardware setups.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Porting E3SM from LANL’s Open to Classified Computing Networks

This project used turquoise computing time to support a LANL TED project focused on porting E3SM to classified computing resources. Computing time was used to test and confirm a complete and working simulation and analysis workflow prior to porting to yellow and SCI-level computing platforms. The code port and all planned simulations and analysis were successful. E3SM analysis from simulations on classified platforms was published to C2S and will be used to demonstrate and advertise LANL’s classified Earth system modeling and analysis capabilities to current and new customers in the intelligence community. This project also significantly improved library and E3SM analysis support on turquoise, yellow, and classified computing platforms at LANL, which will result in benefits beyond the work discussed here.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Elastic Workflows with PMIx [Slides]

Scientific workflows increasing in complexity. Elastic workflows promise improved application and system performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

First-generation college student forges ahead, now key to Lab's mission

Fatima Woody was just 17 and a student at Pojoaque Valley High School when she first started her career as a Los Alamos Neutron Science Center receptionist. Now she's in a crucial role that keeps plutonium pit production and other mission processes operating with as little interruption as possible. Over nearly four decades, Fatima has gradually advanced from her initial positions as a receptionist and administrative secretary to become a computer technician, then a computer system professional who specializes in project management. Today, she coordinates the workflow for nearly two dozen deployed information technology technicians who keep the computer systems and networks operating at the high-tech complex that houses the Lab's Plutonium Facility.

99 GENERAL AND MISCELLANEOUS↗