Search NASA⌕ Search

SEARCH · Search NASA

Results for “computational workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

AiiDA: automated interactive infrastructure and database for computational science

Computational science has seen in the last decades a spectacular rise in the scope, breadth, and depth of its efforts. Notwithstanding this prevalence and impact, it is often still performed using the renaissance model of individual artisans gathered in a workshop, under the guidance of an established practitioner. Great benefits could follow instead from adopting concepts and tools coming from computer science to manage, preserve, and share these computational efforts. We illustrate here our paradigm sustaining such vision, based around the four pillars of Automation, Data, Environment, and Sharing. We then discuss its implementation in the open-source AiiDA platform (http://www.aiida.net), that has been tuned first to the demands of computational materials science. AiiDA’s design is based on directed acyclic graphs to track the provenance of data and calculations, and ensure preservation and searchability. Remote computational resources are managed transparently, and automation is coupled with data storage to ensure reproducibility. Last, complex sequences of calculations can be encoded into scientific workflows. We believe that AiiDA’s design and its sharing capabilities will encourage the creation of social ecosystems to disseminate codes, data, and scientific workflows.

Materials Science↗
DOE OSTI2020

Offsite Data Processing for the GlueX Experiment

The Thomas Jefferson National Accelerator Facility (JLab) 12GeV accelerator upgrade completed in 2015 is now producing data at volumes unprecedented for the lab. The resources required to process this data now exceed the capacity of the onsite farm necessitating the use of offsite computing resources for the first time in the history of JLab. GlueX is now utilizing NERSC and PSC for raw data production. Details of the workflow are presented.

Lawrence, David↗
DOE OSTI2022

Pseudonymization at Scale: OLCF’s Summit Usage Data Case Study

The analysis of vast amounts of data and the processing of complex computational jobs have traditionally relied upon high performance computing (HPC) systems, which offer reliable and efficient management of large-scale computational and data resources. Understanding these analyses’ needs is paramount for designing solutions that can lead to better science, and similarly, understanding the characteristics of the user behavior on those systems is important for improving user experiences on HPC systems. A common approach to gathering data about user behavior is to extract workload characteristics from system log data available only to system administrators. Recently at Oak Ridge Leadership Computing Facility (OLCF), however, we unveiled user behavior about the Summit supercomputer by collecting data from a user’s point of view with ordinary Unix commands.In this paper, we discuss the process, challenges, and lessons learned while preparing this dataset for publication and submission to an open data challenge. The original dataset contains personal identifiable information (PII) about the users of OLCF which needed be masked prior to publication, and we determined that anonymization, which scrubs PII completely, destroyed too much of the structure of the data to be interesting for the data challenge. We instead chose to pseudonymize the dataset, which reduced the linkability of the dataset to the users’ identities. Pseudonymization is significantly more computationally expensive than anonymization, and the size of our dataset, which is approximately 175 million lines of raw text, necessitated the development of a parallelized workflow that could be reused on different HPC machines. We demonstrate the scaling behavior of the workflow on two leadership class HPC systems at OLCF, and we show that we were able to bring the overall makespan time from an impractical 20+ hours on a single node down to around 2 hours. As a result of this work, we release the entire pseudonymized dataset and make the workflows and source code publicly available.

Maheshwari, Ketan↗
DOE OSTI2020

Enabling human–infrastructure interfaces for inspection using augmented reality

Decaying infrastructure maintenance cost allocation depends heavily on accurate and safe inspection in the field. New tools to conduct inspections can assist in prioritizing investments in maintenance and repairs. The industrial revolution termed as “Industry 4.0” is based on the intelligence of machines working with humans in a collaborative workspace. Contrarily, infrastructure management has relied on the human for making day-to-day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today’s owners agree in trusting the inspector’s decision in the field over data collected with sensors. If data collected in the field is accessible during the inspections, the inspector decisions can be improved with sensors. New research opportunities in the human–infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This article studies the role of Augmented Reality (AR) technology as a tool to increase human awareness of infrastructure in their inspection work. The domains of interest of this research include both infrastructure inspections (emphasis on the collection of data of structures to inform management decisions) and emergency management (focus on the data collection of the environment to inform human actions). This article describes the use of a head-mounted device to access real-time data and information during their field inspection. The authors leverage the use of low-cost smart sensors and QR code scanners integrated with Augmented Reality applications for augmented human interface with the physical environment. This article presents a novel interface architecture for developing Augmented Reality–enabled inspection to assist the inspector’s workflow in conducting infrastructure inspection works with two new applications and summarizes the results from various experiments. The main contributions of this work to computer-aided community are enabling inspectors to visualize data files from database and real-time data access using an Augmented Reality environment.

Maharjan, D.↗
NASA NTRS2023

Automated Fiber Placement Through Thickness Defect Stacking Optimization

In its 2022 commercial market outlook, Boeing forecasted an 80% increase in the global fleet through 2041 compared to 2019 pre-pandemic levels. This sharp rise in demand will drive pressure onto airframe manufacturers to ramp up production and find more efficient ways to design and manufacture airplanes. Complicating this challenge is the industry’s recent transformation from traditional metal-based airframes towards hybrid composite-metal aircraft. While composites have been used in aviation for decades, aircraft manufacturers are still struggling to design and manufacture quality parts at a high rate. Automated Fiber Placement (AFP) is one of the main manufacturing techniques used to produce large-scale composite parts. After a design has been created, a manufacturing strategy has to be developed based on the working material, part geometry, and machine capabilities. This process planning stage is essential to the AFP workflow and currently requires a high level of manual input from an experienced process planner. In an effort to automate and optimize this stage, the Computer Aided Process Planning (CAPP) module was developed. CAPP assists process planners in identifying optimal starting point location and layup strategy for each ply of a laminate. This Ply-Level Optimization (PLO) phase operates on the quantification of ply quality through predictable geometry-based defects such as gaps, overlaps, angle deviation, and steering. As you move from PLO to Laminate-Level Optimization (LLO) the design space grows exponentially, emphasizing the need for automated optimization. The work presented in this thesis expands CAPP’s functionality by comparing the planned fiber paths through the thickness of the laminate to mitigate stacked area defects and achieve an optimal laminate-level manufacturing strategy. Within CAPP, predicted gap and overlap defects are imported from Vericut Composites Programming (VCP) and then discretized to streamline the through-thickness comparison. Two objective functions are used to score different combinations of ply layup strategies based on defect stacking both globally and locally. Four combinatorial optimization algorithms were coupled with these objective functions to investigate the laminate-level manufacturing strategy design space and converge on the optimal plan. These algorithms were evaluated based on accuracy and efficiency through virtual testing on a complex tool surface. A separate LLO approach was developed to achieve near-optimal laminates in significantly less time. The end result is a software package which greatly reduces the required input from process planners, shortening the design-build cycle time and improving part quality.

AFP↗
DOE OSTI2022

Computational synthesis of 2D materials: A high-throughput approach to materials design

2D materials find promising applications in next-generation devices, however, large-scale, low-defect, and reproducible synthesis of 2D materials remains a challenging task. Here, to assist in the selection of suitable substrates for the synthesis of as-yet hypothetical 2D materials, we have developed an open-source high-throughput workflow package, Hetero2d, that searches for low-lattice mismatched substrate surfaces for any 2D material and determines the stability of these 2D-substrate heterostructures using density functional theory (DFT) simulations. Hetero2d automates the generation of 2D-substrate heterostructures, the creation of DFT input files, the submission and monitoring of computational jobs on supercomputing facilities, and the storage of relevant parameters alongside the post-processed results in a MongoDB database. We demonstrate the capability of Hetero2d in identifying stable 2D-substrate heterostructures for four 2D materials, namely 2H-MoS 2 , 1T- and 2H-NbO 2 , and hexagonal-ZnTe, considering 50 cubic elemental substrates. We find Cu, Hf, Mn, Nd, Ni, Pd, Re, Rh, Sc, Ta, Ti, V, W, Y, and Zr substrates sufficiently stabilize the formation energies of these 2D materials, with binding energies in the range of ~0.1–0.6 eV/atom. Upon examining the z-separation, the charge transfer, and the electronic density of states at the 2D-substrate interface, we find a covalent type bonding at the interface which suggests that these substrates can be used as contact materials for the 2D materials. Hetero2d is available on GitHub as an open-source package under the GNU license.

36 MATERIALS SCIENCE↗
DOE OSTI2019

BigPanDA: PanDA Workload Management System and its Applications beyond ATLAS

Modern experiments collect peta-scale volumes of data and utilize vast, geographically distributed computing infrastructure that serves thousands of scientists around the world. Requirements for rapid, near real-time data processing, fast analysis cycles and need to run massive detector simulations to support data analysis pose special premium on efficient use of available computational resources. A sophisticated Workload Management System (WMS) is needed to coordinate the distribution and processing of data and jobs in such environment. The ATLAS experiment at CERN uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. While PanDAcurrently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, it runs around 2 million jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. In 2017 about 1.5 exabytes of data were processed with PanDA.In 2012 BigPanDA project project was started with aim to introduce new types of computing resources into ATLAS computing infrastructure, but also to offering PanDA features to different data-intensive applications for projects and experiments outside of ATLAS and High-Energy and Nuclear Physics. In this article we will present accomplishments and discuss possible directions for future work.

Svirin, Pavlo↗
DOE OSTI2024

HPC-FAIR: A Framework Managing Data and AI Models for Analyzing and Optimizing Scientific Applications

The increasing reliance on machine learning (ML) to analyze and optimize large-scale scientific applications on supercomputers faces a significant bottleneck: the lack of readily available, high-quality training datasets and the difficulty in reusing existing AI models. This project was motivated by the urgent need to address the “FAIR” principles (Findability, Accessibility, Interoperability, Reusability) for both training datasets and AI models in the high-performance computing (HPC) domain. The project developed HPC-FAIR, a high-performance computing data management framework designed to centralize HPC-related datasets and AI models within a unified hub. To ensure interoperability, the framework established a standardized representation and vocabulary (ontology) for both data and models. HPC-FAIR also implemented automated workflows to streamline data processing, model access, and benchmarking. Additionally, the project focused on optimizing data harnessing efficiency through advanced techniques like deep reuse and compression-based analytics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Autonomy Loops for Monitoring, Operational Data Analytics, Feedback, and Response in HPC Operations

Many High Performance Computing (HPC) facilities have developed and deployed frameworks in support of continuous monitoring and operational data analytics (MODA) to help improve efficiency and throughput. Because of the complexity and scale of systems and workflows and the need for low-latency response to address dynamic circumstances, automated feedback and response have the potential to be more effective than current human-in-the-loop approaches which are laborious and error prone. Progress has been limited, however, by factors such as the lack of infrastructure and feedback hooks, and successful deployment is often site- and case-specific. In this position paper we report on the outcomes and plans from a recent Dagstuhl Seminar, seeking to carve a path for community progress in the development of autonomous feedback loops for MODA, based on the established formalism of similar (MAPE-K) loops in autonomous computing and self-adaptive systems. By defining and developing such loops for significant cases experienced across HPC sites, we seek to extract commonalities and develop conventions that will facilitate interoperability and interchangeability with system hardware, software, and applications across different sites, and will motivate vendors and others to provide telemetry interfaces and feedback hooks to enable community development and pervasive deployment of MODA autonomy loops.

autonomy loops↗
DOE OSTI2023

QuantumScents: Quantum-Mechanical Properties for 3.5k Olfactory Molecules

Quantitative structure–odor relationships are critically important for studies related to the function of olfaction. Current literature data sets contain expert-labeled molecules but lack feature data. This paper introduces QuantumScents, a quantum mechanics augmented derivative of the Leffingwell data set. QuantumScents contains 3.5k structurally and chemically diverse molecules ranging from 2 to 30 heavy atoms (CNOS) and their corresponding 3D coordinates, total PBE0 energy, molecular dipole moment, and per-atom Hirshfeld charges, dipoles, and ratios. The authors demonstrate that Hirshfeld charges and ratios contain sufficient information to perform molecular classification by training a Message Passing Neural Network with chemprop (Heid, E.; et al. ChemRxiv, 2023, DOI: 10.26434/chemrxiv-2023-3zcfl) to predict scent labels. Finally, the QuantumScents data set is freely available on Zenodo along with the authors’ code, example models, and data set generation workflow (https://zenodo.org/doi/10.5281/zenodo.8239853).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Many extreme scale scientific applications have workloads comprised of a large number of individual highperformance tasks. The Pilot abstraction decouples workload specification, resource management, and task execution via job placeholders and late-binding. As such, suitable implementations of the Pilot abstraction can support the collective execution of large number of tasks on supercomputers. We introduce RADICAL-Pilot (RP) as a portable, modular and extensible Pilot enabled runtime system. We describe RP's design, architecture and implementation. We characterize its performance and show its ability to scalably execute workloads comprised of tens of thousands heterogeneous tasks on DOE and NSF leadership-class HPC platforms. Specifically, we investigate RP's weak/strong scaling with CPU/GPU, single/multi core, (non)MPI tasks and python functions when using most of ORNL Summit and TACC Frontera. RADICAL-Pilot can be used stand-alone, as well as the runtime for third-party workflow systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

RAVEN Quality Assurance Activities

This report discusses the quality assurance activities needed to raise the Quality Level of Risk Analysis in a Virtual Environment (RAVEN) from Quality Level 3 to Quality Level 2. This report also describes the general RAVEN quality assurance activities. For improving the quality, reviews of code changes have been instituted, more parts of testing have been automated, and improved packaging has been created. For upgrading the quality level, requirements have been created and the workflow has been improved.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2017

Modeling and Simulation Capability Development Plan

This report provides an assessment of the current status of the modelling and simulation capability for the Nuclear-Renewable Hybrid Energy System (N-R HES). The current capabilities are separated in: • Creation of the system boundary condition. More in detail this refer to the capability to create either demand, renewable supply, price profiles etc.. • Creation of a numerical representation of the physical model of a N-R HES system • Derivation, from the actual performance of the physical system, the cash flow to perform the economical assessment • Optimization of N-R HES configuration and dispatch Those capabilities are the ones which enable the workflow summarize in Figure 1below.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

BigData Express: Toward Predictable, Schedulable, and High-Performance Data Transfer [Slides]

BigData Express: a schedulable, predictable, and high-performance data transfer service. A peer-to-peer, scalable, and extensible data transfer model; A visually appealing, easy-to-use web portal; A high-performance data transfer engine; On-demand provisioning of end-to-end network paths with guaranteed QoS; Robust and flexible error handling; CILogon-based security (certificate version); A rich set of REST APIs to support scientific workflows

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Real-Time Optimization Workflow Status Update

Economically optimal and safe operation of integrated energy systems (IES) requires optimization at many different time scales. A real-time optimization (RTO) workflow will attempt to maximize revenue and minimize operational costs on a time scale of minutes to hours. Such a workflow requires the use of a digital twin (DT), which is a virtual representation of a physical system. The DT is updated using real-time data from the physical system, and serves as a model in an optimization framework. The optimization results are then sent back to the physical system to complete the loop. This report details the progress made in developing building blocks for a DT/RTO framework. The Risk Analysis Virtual Environment (RAVEN) platform within the Framework for Optimization of Resources and Economics (FORCE) tool suite can perform many of the tasks required for building a DT and performing RTO. The first item of this report details RAVEN enhancements that enable RAVEN workflows to be run in various environments. Data communication between the physical system and its DT is essential for successful RTO. This includes preprocessing real-time data, loading data into a data warehouse, and querying the stored data. The second section of this report describes the progress made in implementing an adapter in Python in order for Deep Lynx to handle the data communication. Typical dispatch optimization frameworks are built on linear programming (LP). The prototype RTO workflow developed in this report uses an LP problem as a part of a receding-horizon- or economic model predictive control (EMPC) based optimization. The third section of this report details the framework of an RTO workflow in which the system consists of a simple electrical storage device. A DT can be built from a reduced-order model (ROM). Integrating a ROM into a typical LP optimization framework has been challenging because most optimization packages require the user to write algebraic expressions for the system model. The final section of this report shows how an externally built RAVEN ROM can be integrated in an RTO framework by using the Python package Pyomo. This demonstrates the RTO workflow capability from a software-only perspective and is an important step in demonstrating the capability to implement an RTO workflow for a physical system.

97 MATHEMATICS AND COMPUTING↗