Search NASA⌕ Search

SEARCH · Search NASA

Results for “Automated workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Integration of scanning probe microscope with high-performance computing: Fixed-policy and reward-driven workflows implementation

The rapid development of computation power and machine learning algorithms has paved the way for automating scientific discovery with a scanning probe microscope (SPM). The key elements toward operationalization of the automated SPM are the interface to enable SPM control from Python codes, availability of high computing power, and development of workflows for scientific discovery. Here, we build a Python interface library that enables controlling an SPM from either a local computer or a remote high-performance computer, which satisfies the high computation power need of machine learning algorithms in autonomous workflows. We further introduce a general platform to abstract the operations of SPM in scientific discovery into fixed-policy or reward-driven workflows. Furthermore, our work provides a full infrastructure to build automated SPM workflows for both routine operations and autonomous scientific discovery with machine learning.

47 OTHER INSTRUMENTATION↗

express: Extensible, high-level workflows for swifter ab initio materials modeling

In this work, we introduce an open-source Julia project, express, an extensible, lightweight, high-throughput, high-level workflow framework that aims to automate ab initio calculations for the materials science community. express is shipped with well-tested workflow templates, including structure optimization, equation of state (EOS) fitting, phonon spectrum (lattice dynamics) calculation, and thermodynamic property calculation in the framework of the quasi-harmonic approximation (QHA). It is designed to be highly modularized so that its components can be reused across various occasions, and customized workflows can be built on top of that. Users can also track the status of workflows in real-time, and rerun failed jobs thanks to the data lineage feature express provides. Finally, two working examples, i.e., all workflows applied to lime and akimotoite, are also presented in the code and this paper.

36 MATERIALS SCIENCE↗

Leveraging Automated Fiber Placement Computer Aided Process Planning Framework for Defect Validation and Dynamic Layup Strategies

Process planning represents an essential stage of the Automated Fiber Placement (AFP) workflow. It develops useful and efficient machine processes based upon the working material, composite design, and manufacturing resources. The current state of process planning requires a high degree of interaction from the process planner and could greatly benefit from increased automation. Therefore, a list of key steps and functions are created to identify the more difficult and time-consuming phases of process planning. Additionally, a set of metrics must exist by which to evaluate the effectiveness of the manufactured laminate from the machine code created during the Process Planning stage. Layup strategies, in addition to dog ears, stagger shifts, steering constraints, and starting points, represented the group of functions labeled as process optimization and ranked the highest in terms of priority for automation. The laminates resulting from the selected parameters are evaluated through the occurrences of principal defect metrics such as fiber gaps, overlaps, angle deviation and steering violations. This document presents an automated software solution to the layup strategy and starting point selection phase of process planning. A series of ply scenarios are generated with variations of these ply parameters and evaluated according to a set of metrics entered by the Process Planner. These metrics are generated through use of the Analytical Hierarchy Process (AHP), where relative importance between each of the fiber features are defined. The ply scenarios are selected which reduce the overall fiber feature scores based on the defects the Process Planner wishes to minimize.

HiCAM↗

Real-World Experiences Adopting Workflows at Exascale on the ExaAM Project

The purpose of this study is to discuss the experiential lessons associated with adopting scientific workflows in the Exascale Additive Manufacturing project (ExaAM) through the lens of Perceived Characteristic of Innovation (PCI). Besides the implementation, the factors we considered critical to the adoption of the workflow are provenance, sustainable automation, implementation challenges, and integration/compatibility challenges. Through conversations and interviews among the program managers, project leads, and software engineers, we have developed critical insight and strategies to overcome the obstacles and augment the successful adoption and long-term use of these workflows in ExaAM and beyond. We hope our work will pave the way for others in the research community to develop and use workflows in their respective science domains.

Malviya, Addi Thakur↗

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research

Earth observation satellites and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose significant challenges for processing and analysis. In this paper we discuss our experiences in developing and using a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our workflow can be seamlessly integrated and orchestrated across research facilities managed by different federal agencies, thus allowing users to extract new scientific insights from climate datasets. The experimental results indicate that the multi-facility workflow significantly reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Notably, our workflow processes 12,000 high-resolution satellite images in just 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems. Such high throughput is essential for dynamic tokenization and sharding of petascale satellite data for distributed AI model training and inferencing at scale across thousands of GPUs.

Kurihana, Takuya [ORNL] (ORCID:0000000156698565)↗

Heterogeneous Hierarchical Workflow Composition

Workflow systems promise scientists an automated end-to-end path from hypothesis to discovery. However, expecting any single workflow system to deliver such a wide range of capabilities is impractical. A more practical solution is to compose the end-to-end workflow from more than one system. With this goal in mind, the integration of task-based and in situ workflows is explored, where the result is a hierarchical heterogeneous workflow composed of subworkflows, with different levels of the hierarchy using different programming, execution, and data models. Materials science use cases demonstrate the advantages of such heterogeneous hierarchical workflow composition.

distributed workflows↗

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

Panorama 360 (Final Report)

This final technical report from the lead institution, USC grant #DE-SC0012636, serves as the final technical report for collaborative institution UNC-CH grant #DE-SC0012390. The goal was to develop a repository and associated capabilities for data collection, ingestion, and analysis for a broad class of DOE applications that span experimental and simulation science workflows. In particular, this work focuses on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: (1) A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); (2) A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; (3) A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and (4) Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

A Recipe for Streamlining Mission Management

This paper describes a project's design and implementation for streamlining mission management with knowledge capture processes across multiple organizations of a NASA directorate. Thc project's focus is on standardizing processes and reports; enabling secure information access and case of maintenance; automating and tracking appropriate workflow rules through process mapping; and infusing new technologies. This paper will describe a small team's experiences using XML technologies through an enhanced vendor suite of applications integrated on Windows-based platforms called the Wallops Integrated Scheduling and Document Management System (WISDMS). This paper describes our results using this system in a variety of endeavors, including providing range project scheduling and resource management for a Range and Mission Management Office; implementing an automated Customer Feedback system for a directorate; streamlining mission status reporting across a directorate; and initiating a document management, configuration management and portal access system for a Range Safety Office's programs. The end result is a reduction of the knowledge gap through better integration and distribution of information, improved process performance, automated metric gathering, and quicker identification of problem areas and issues. However, the real proof of the pudding comes through overcoming the user's reluctance to replace familiar, seasoned processes with new technology ingredients blended with automated procedures in an untested recipe. This paper shares some of the team's observations that led to better implementation techniques, as well as an IS0 9001 Best Practices citation. This project has provided a unique opportunity to advance NASA's competency in new technologies, as well as to strategically implement them within an organizational structure, while wetting the appetite for continued improvements in mission management.

Mitchell, Andrew E.↗

rabpro: global watershed boundaries, river elevation profiles, and catchment statistics

River and Basin Profiler (rabpro) is a Python package to delineate watersheds, extract river flowlines and elevation profiles, and compute watershed statistics for any location on the Earth’s surface. As fundamental hydrologically-relevant units of surface area, watersheds are areas of land that drain via aboveground pathways to the same location, or outlet. Delineations of watershed boundaries are typically performed on digital elevation models (DEMs) that represent surface elevations as gridded rasters. Depending on the resolution of the DEM and the size of the watershed, delineation may be very computationally expensive. With this in mind, we designed rabpro to provide user-friendly workflows to manage the complexity and computational expense of watershed calculations given an arbitrary coordinate pair. In addition to basic watershed delineation, rabpro will extract the elevation profile for a watershed’s mainchannel flowline. This enables the computation of river slope, which is a critical parameter in many hydrologic and geomorphologic models. Finally, rabpro provides a user-friendly wrapper around Google Earth Engine’s (GEE) Python API to enable cloud-computing of zonal watershed statistics and/or time-varying forcing data from hundreds of available datasets. Altogether, rabpro provides the ability to automate or semi-automate complex watershed analysis workflows across broad spatial extents.

54 ENVIRONMENTAL SCIENCES↗

PyARC Status Report: New Integrations and Upgrades to the Fast Reactor Analysis Workflow Management Tool

PyARC was initially developed as an open source tool to support fast reactor analyses using the Argonne Reactor Computation (ARC) code suite as a part of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) Workbench initiative in FY17. The goal of this initiative is to provide a common user interface for model generation, real-time validation, execution, output processing, and visualization for all integrated codes. This is accomplished through the reliance on tools available in the Workbench framework and runtime environment. While initially developed to support the ARC codes, PyARC was extended in FY22 to wrap other NEAMS and non-ARC codes, including Griffin and OpenMC, in the supported other neutronics workflows, and support users in the adoption of NEAMS-supported high fidelity analysis codes. Most recently, NUBOW-3D, a recently adopted ARC code, was integrated to support reactor bowing calculations as well. Integration of these codes into the NEAMS Workbench directly benefits the advanced reactor modeling community by: • Providing a set of controlled, maintained, documented and validated scripts to generate inputs, which promotes best practices, reduces the learning curve, and facilitates project collaboration. • Improving the user experience: the Workbench interface provides assistance for building an input through auto-completion, real-time validation, document navigation, and geometry and results visualization. • Automating complex calculations and workflows for reactor analysis. • Helping users transition to using high-fidelity NEAMS codes along-side the ARC codes. In FY22, a progress report was published that described the state of each of the tools integrated into PyARC. Since then, there have been many enhancements and upgrades to the existing integrations as well as entirely new code integrations as well. This report details all new integrations and major developments in PyARC since the version 2.0.0 release highlighted in the FY22 report.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Building a Consistent and Reproducible Database for Adsorption Evaluation in Covalent–Organic Frameworks

We present a workflow that traces the path from the bulk structure of a crystalline material to assessing its performance in carbon capture from coal's postcombustion flue gases. This workflow is applied to a database of 324 covalent-organic frameworks (COFs) reported in the literature, to characterize their CO 2 adsorption properties using the following steps: (1) optimization of the crystal structure (atomic positions and unit cell) using density functional theory, (2) fitting atomic point charges based on the electron density, (3) characterizing the pore geometry of the structures before and after optimization, (4) computing carbon dioxide and nitrogen isotherms using grand canonical Monte Carlo simulations with an empirical interaction potential, and finally, (5) assessing the CO 2 parasitic energy via process modeling. The full workflow has been encoded in the Automated Interactive Infrastructure and Database for Computational Science (AiiDA). Both the workflow and the automatically generated provenance graph of our calculations are made available on the Materials Cloud, allowing peers to inspect every input parameter and result along the workflow, download structures and files at intermediate stages, and start their research right from where this work has left off. In particular, our set of CURATED (Clean, Uniform, and Refined with Automatic Tracking from Experimental Database) COFs, having optimized geometry and high-quality DFT-derived point charges, are available for further investigations of gas adsorption properties. We plan to update the database as new COFs are being reported.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Ultra-Fast Sample Preparation for High-Throughput Proteomics

Sample preparation oftentimes can be the Achilles Heel of any analytical process and in the field of proteomics, preparing samples for mass spectrometric analysis is no exception. Current goals, concerning proteomic sample preparation on a large scale, include efforts toward improving reproducibility, reducing the time of processing and ultimately the automation of the entire workflow. This chapter reviews an array of recent approaches applied to bottom-up proteomics sample preparation to reduce the processing time down from hours to minutes. The current state-of-the-art in the field uses different energy inputs like microwave, ultrasound or pressure to perform the four basic steps in sample preparation: protein extraction, denaturation, reduction and alkylation, and digestion. No single energy input for enhancement of proteome sample preparation has become the universal gold standard. Instead, a combination of different energy inputs tend to produce the best results. This chapter further describes the future trends in the field such as the hyphenation of sample preparation with downstream detection and analysis systems. Finally, a detailed protocol describing the combined use of both pressure cycling technology and ultrasonic energy inputs to hasten proteomic sample preparation is presented.

59 BASIC BIOLOGICAL SCIENCES↗

Accelerating Next-Generation Cybersecurity R&D Using AI Workflows: BADGER Project Development

The Broadband Automation for Distributed Grid Efficiency and Resilience (BADGER) project aligns with national strategic priorities for integrating emerging wireless technologies and advancing AI-driven security. As critical infrastructure modernizes toward increasingly software-defined and interconnected systems, the ability to leverage 5G/NextG networks and AI-enabled control becomes essential. This report outlines work at the National Laboratory of the Rockies (NLR) to develop a NextG-native security architecture powered by AI-RAN concepts and evaluate workflows that enable efficient and reliable architectures. Together, these efforts position the laboratory to accelerate innovation while directly supporting national security and resilience objectives.

5G/6G↗

Massive all-atom analysis of 2D materials with quantum properties (Final report)

Improvements in microscopy have enabled the acquisition of data at a scale that is difficult to process manually, making automated machine learning approaches to analyzing experimental images essential. In this project, we developed and applied machine learning (ML) workflows for atomic resolution scanning transmission electron microscopy (STEM) images. This development included improving both methodology as well as generating user-friendly codes. We developed machine learning architectures which, after training, automatically identify the location and types of defects throughout a material. We used these data to produce class-averaged images of 2D atomic coordinates with up to 0.3 pm precision, uncovering the structure and oscillations of long-range strain fields around point defects in WSe 2-2x Te 2x . We also resolved a long-standing problem in this field in the training of ML models, a lack of labeled experimental data, by developing a cycle-GAN that transformed simulated-generated labeled data into labeled data indistinguishable from experiment and therefore suitable for training. This removed the remaining parts of the ML data processing workflow where human intervention was still critical and therefore a bottleneck to working at scale. Codes have been developed and released for this full machine learning workflow. ML approaches to partially automate STEM acquisition were also developed. Finally we applied ML and other advanced data processing methods to several materials science problems in two-dimensional materials, including studying the evolution of hyperuniformity with defect concentration in WSe2, understanding phase transformations in transition metal dichalcogenides during in-situ heating in the STEM, and exploring how 2D interfaces transform from twisted into aligned structures.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

nmRanalysis: An Open-Source Web Application for Semi-automated NMR Metabolite Profiling

Though data acquisition and initial signal pre-processing of nuclear magnetic resonance (NMR) spectra have achieved high degrees of automation, downstream processing - specifically the profiling of spectra - has bottlenecked the overall NMR analysis workflow. Several efforts have been made to mitigate this bottleneck, but these solutions often trade an increase in automation for limitations elsewhere. Here, in this technical note, we introduce nmRanalysis, a user-friendly web-application that integrates the strengths of existing profiling tools for a more automated profiling workflow. nmRa-nalysis additionally incorporates novel features, including a machine-learning-driven recommender system for me-tabolite identification, further increasing the utility of nmRanalysis over the individual tools that it incorporates.

Flores, Javier E. [Pacific Northwest National Labo↗

EARTHDATA PUB: A Data Publication Workflow Solution for NASA’s EOSDIS

Each NASA Distributed Active Archive Center (DAAC) faces the challenge of dealing with an increasingly diverse number of publishable data products from diverse data producers. Data producers, on the other hand, may experience pain points when interacting with the EOSDIS for the first time or when publishing different data at different DAACs. As a result, there has been a growing need to develop a common software framework that serves as a common interface for data producers, rigorously defines the data publication procedure for DAAC staff, facilitates the management of various data publication processes, and tracks the progress of data publication. This software should also account for the different configurations at different DAACs. Currently, two primary data publication workflow and tracking tools exist in operation at the EOSDIS: Semi-Automated ingest System (SAuS) and Data Publication workflow Portal (DAPPeR). However, neither tool is cloud-ready. Automated data processing could be managed by Cumulus, an EOSDIS cloud-based data ingest, archive and management system. However, Cumulus does not support manual tasks or on-premise implementations. We propose to develop the Earthdata Publication Minimum Viable Product (Earthdata Pub MVP) -- a cloud-hosted solution that works with both cloud and on-premise systems and implements the communications and exchange requirements generated by the Earthdata Pub information architecture team.

Rice, Justin L.↗

Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument Ecosystems

Science ecosystems are being built by federating computing systems and instruments located at geographically distributed sites over wide-area networks. These computing-instrument ecosystems are expected to support complex workflows that incorporate remote, automated AI-driven science experiments. Their realization, however, requires various designs to be explored and software components to be developed, in order to support the orchestration of distributed computations and experiments. It is often too expensive, infeasible, or disruptive for the entire ecosystem to be available during the typically long software development and testing periods. We propose a Virtual Infrastructure Twin (VIT) of the ecosystem that emulates its network and computing components, and incorporates its instrument software simulators. It provides a software environment nearly identical to the ecosystem to support early development and testing, and design space exploration. We present a brief overview of previous digital infrastructure twins that culminated in the VIT concept, including (i) the virtual science network environment for developing software-defined networking solutions, and (ii) the virtual federated science instrument environment for testing the federation software stack and remote instrument control software. We briefly describe VITs for Nion microscope steering and access to GPU systems.

Rao, Nageswara↗