Search NASA⌕ Search

SEARCH · Search NASA

Results for “Workflow Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

Analysis of soil eDNA functional genes: potential to increase profitability and sustainability of pastoral agriculture

Management of soil biological resources to optimise plant production, efficiency of nutrient inputs, and system sustainability is an emerging opportunity for pastoral agriculture. To attain these goals, suitable tools that can assess the functional state of the soil ecosystem must be developed and standardised approaches to their application adopted. Towards this end, we have undertaken comprehensive, high-density functional-gene microarray analysis (GeoChip5) of environmental DNA (eDNA) extracted from 50 pastoral soils. When combined with soil, environmental and management metadata, the information can be used to provide insights into soil biological processes spanning greenhouse gas emissions, through to natural suppression of plant root diseases. To give an example of a structured workflow of analysis in a pastoral system context, we analysed the GeoChip data using a combination of approaches spanning routine univariate methods through to more complex multivariate and network-based analysis. Analyses were restricted to comparing effects of land-use (dairy or ‘other’ farming systems), and exploring relationships of the GeoChip data with the soil properties from each sample. These exemplar analyses introduce a pathway for the application of eDNA approaches (GeoChip or others) to deliver outcomes for pastoral agricultural in New Zealand.

59 BASIC BIOLOGICAL SCIENCES↗

YoDawg CI Runner

SAND2022-15186 O YoDawg CI Runner builds a container that hosts a Gitlab runner as a service. It provides the ability to run Podman in a nested fashion as part of a CI workflow (i.e., "Podman-in-Podman”). This allows CI jobs to be run on HPC resources by standard users through an on-demand batch scheduler. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Curry, Matthew↗

SmoQyDQMC.jl: A flexible implementation of determinant quantum Monte Carlo for Hubbard and electron-phonon interactions

We introduce the SmoQyDQMC.jl package, a Julia implementation of the determinant quantum Monte Carlo algorithm. SmoQyDQMC.jl supports generalized tight-binding Hamiltonians with on-site Hubbard and generalized electron-phonon ( e e -ph) interactions, including non-linear e e -ph coupling and anharmonic lattice potentials. Our implementation uses hybrid Monte Carlo methods with exact forces for sampling the phonon fields, enabling efficient simulation of low-energy phonon branches, including acoustic phonons. The SmoQyDQMC.jl package also uses a flexible scripting interface, allowing users to adapt it to different workflows and interface with other software packages in the Julia ecosystem. The code for this package can be downloaded from our GitHub repository at https://github.com/SmoQySuite/SmoQyDQMC.jl or installed using the Julia package manager. The online documentation, including examples, can be obtained from our document page at https://smoqysuite.github.io/SmoQyDQMC.jl/stable/.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Codebase release r0.3 for SmoQyDQMC.jl

We introduce the SmoQyDQMC.jl package, a Julia implementation of the determinant quantum Monte Carlo algorithm. SmoQyDQMC.jl supports generalized tight-binding Hamiltonians with on-site Hubbard and generalized electron-phonon ( e e -ph) interactions, including non-linear e e -ph coupling and anharmonic lattice potentials. Our implementation uses hybrid Monte Carlo methods with exact forces for sampling the phonon fields, enabling efficient simulation of low-energy phonon branches, including acoustic phonons. The SmoQyDQMC.jl package also uses a flexible scripting interface, allowing users to adapt it to different workflows and interface with other software packages in the Julia ecosystem. The code for this package can be downloaded from our GitHub repository at https://github.com/SmoQySuite/SmoQyDQMC.jl or installed using the Julia package manager. The online documentation, including examples, can be obtained from our document page at https://smoqysuite.github.io/SmoQyDQMC.jl/stable/.

Cohen-Stead, Benjamin (ORCID:0000000279156280)↗

BEE - FY20 P6-3: Release BEEWorkflowManager, BEETaskManager, and client application 2.3.6.01 – LANL ATDM ST / STNS01-4 P6 Milestone Completion Documentation

Release BEEWorkflowManager, BEETaskManager, and client software. The BEEWorkflowManager daemon runs on the HPC cluster login node. It accepts workflows submitted by the BEE client. These workflows are specified using the Common Workflow Language (CWL) standard. The BEEWorkflowManager loads workflows into the Neo4j graph database to create the workflow directed acyclic graph (DAG), and submits the workflow tasks to the BEETaskManager for execution. The BEEWorkflowManager records the state of the workflow and its tasks, and communicates this state to the BEE client. The BEEWorkflowManager will start, pause, and cancel a running workflow and its tasks at the command of the BEE client. The BEETaskManager daemon runs on the HPC cluster login node. It accepts tasks from the BEEWorkflowManager, turns those tasks into HPC resource manager jobs (e.g. a slurm job script), and submits the job to the cluster resource manager. The BEETaskManager then tracks the status of the job (pending, running, complete) and updates the BEEWorkflowManager. The BEETaskManager will also cancel a queued or running job when commanded to do so by the BEEWorkflowManager. The first release of the BEETaskManager will support the Slurm resource manager and the Charliecloud linux container runtime.

97 MATHEMATICS AND COMPUTING↗

Improving the User Interface of the DeepLynx Data Warehouse

DeepLynx is an open-source ontology-based data warehouse created by INL to support the creation and life cycle of digital engineering projects, with a particular emphasis on digital twins [1]. Digital twins are systems that represent physical assets and process in a real-time digital environment [1]. Most well-known commercial data warehouses use Graphical User Interfaces (GUIs) for users to interact with their systems [3]. Limited publications have addressed the design of these interfaces and understanding of their target users. The current users and development team acknowledge the need to improve the current UI, not just for aesthetics but to improve functionality and workflow of DeepLynx. Traditional data warehouse users are developers, data scientists and business analysts [2]. DeepLynx users have a vast range of experience using data warehouses, and diverse roles, including engineers, scientists and management positions. Because there is a broader audience of target users for DeepLynx than a typical data warehouse, it is essential that DeepLynx has a useable and intuitive user interface. To achieve this the team performed human-computer interaction methods, including a Heuristic Evaluation of current UI using Neilsen’s Usability Heuristic, create personas based on current users by designing a user survey, data analysis and develop of personas. Followed by a redesign of the UI following using Neilsen’s Usability Heuristic and Norman’s Principles of Interactive Design in industry standard software Figma. Lastly a Heuristic Evaluation of new UI design, using Neilsen’s Usability Heuristic and User testing of redesign UI and have a group of users complete a Thinking Aloud Test of the new UI. Preliminary results of the Heuristic Evaluation of current UI arise issue with Consistency and Standards, Visibility of System Status, Match System and Real World and Recognition Rather than Recall. These issues were addressed in the proposed redesign by applying Neilsen’s Usability Heuristic and Norman’s Principles of Interactive Design. Next steps include formalized list of lessons learned and design implications for future publications.

97 MATHEMATICS AND COMPUTING↗

Evolution of International Space Station Program Safety Review Processes and Tools

The International Space Station Program at NASA is constantly seeking to improve the processes and systems that support safe space operations. To that end, the ISS Program decided to upgrade their Safety and Hazard data systems with 3 goals: make safety and hazard data more accessible; better support the interconnection of different types of safety data; and increase the efficiency (and compliance) of safety-related processes. These goals are accomplished by moving data into a web-based structured data system that includes strong process support and supports integration with other information systems. Along with the data systems, ISS is evolving its submission requirements and safety process requirements to support the improved model. In contrast to existing operations (where paper processes and electronic file repositories are used for safety data management) the web-based solution provides the program with dramatically faster access to records, the ability to search for and reference specific data within records, reduced workload for hazard updates and approval, and process support including digital signatures and controlled record workflow. In addition, integration with other key data systems provides assistance with assessments of flight readiness, more efficient review and approval of operational controls and better tracking of international safety certifications. This approach will also provide new opportunities to streamline the sharing of data with ISS international partners while maintaining compliance with applicable laws and respecting restrictions on proprietary data. One goal of this paper is to outline the approach taken by the ISS Progrm to determine requirements for the new system and to devise a practical and efficient implementation strategy. From conception through implementation, ISS and NASA partners utilized a user-centered software development approach focused on user research and iterative design methods. The user-centered approach used on the new ISS hazard system utilized focused user research and iterative design methods employed by the Human Computer Interaction Group at NASA Ames Research Center. Particularly, the approach emphasized the reduction of workload associated with document and data management activities so more resources can be allocated to the operational use of data in problem solving, safety analysis, and recurrence control. The methods and techniques used to understand existing processes and systems, to recognize opportunities for improvement, and to design and review improvements are described with the intent that similar techniques can be employed elsewhere in safety operations. A second goal of this paper is to provide and overview of the web-based data system implemented by ISS. The software selected for the ISS hazard systemMission Assurance System (MAS)is a NASA-customized vairant of the open source software project Bugzilla. The origin and history of MAS as a NASA software project and the rationale for (and advantages of) using open-source software are documented elsewhere (Green, et al., 2009).

Ratterman, Christian D.↗

AI-Science for Performance Optimization and Diagnosis of Science Instrument Federations

Next generation of science workflows are expected to be executed over complex federations composed of supercomputers, science instruments, storage systems and networks, with new additions of the edge and cloud systems and services. The sheer complexity of these multi-domain federations makes it hard to manage them and optimize their performance, as small impedance mismatches (that can dynamically develop between systems) could drastically degrade the entire federation performance. Recent proliferation of Software Defined Everything (SDX) technologies combined with containerization frameworks provide custom instruments that can monitor and collect critical measurements at various levels to support diagnoses and performance optimization; but their data too enormous for human operators and analysts to process and generate decisions. Machine Learning (ML) methods that extract critical parameters, relationships and trends from the data offer general solutions. Artificial Intelligence (AI) and ML methods must be custom-developed for these problems based on solid, rigorous foundations, since black-box approaches are often ineffective and unsound.We propose to develop comprehensive AI-Science for the performance of science federations to (i) monitor and control storage, networks, experiments, and computing systems across multiple domains via softwarization layers, at speeds and scales orders of magnitude superior to current practice, (ii) optimally realize and orchestrate complex workflows with high performance by using dynamic state and performance estimation methods, and (iii) aggregate measurements across sites and time to develop infrastructure-level profiles, optimizations and diagnoses using AI-Science based on foundational principles from ML, game theory, and information fusion areas.

Rao, Nageswara↗

SU-F-R-06: Traumatic Brachial Plexus Injury Imaging, Developing a Coherent Clinical Protocol From Literature Review Through Practice

Purpose: Medical Physics teams can now play a critical role to help plan and provide studied approaches for traumatic brachial plexus MR imaging (tbpMRI). This is especially important for coordination with uncommon applications, since it is challenging to select the right modality, parameters, and train technologists on the essential components. For this work, we started with a review of the medical literature, performed crossover/volunteer studies to bring tbpMRI to practice with greater image QC and protocol management. Methods: To the best of our knowledge, we reviewed the known searchable domain for tbpMRI. We found 69 total articles since 2000. Articles were evaluated with our published protocol for literature management (LIMES3). Two physicists and two radiologists condensed the information from all articles into a knowledgebase. Results: The initial literature demonstrated great heterogeneity, which was a sign that this area needed greater consistency. Despite inconsistency and imprecision, we extracted the most relevant targets using our long-term experience with protocol development in MSK. We ran volunteers on six different magnets of various field strengths with multiple receiver coils, and rebuilt a coherent protocol for tbpMRI. Our radiologists rated LIMES3 work as superior. We have received referrals from the ER and have conducted four patient evaluations. Conclusion: Traumatic brachial plexus MRI has great possible benefits for patients. This work supports the complexity of tbpMRI scanning. As this is rarely performed, it requires a more diligent protocol workflow, coordination of caregivers, and education within multiple clinical departments. Choosing the correct imaging exam can be critical, as patients can have significant neuropathy and/or paralysis. The LIMES3 protocol is well liked at our institution, and forms the cornerstone of understanding for our work. Our literature management led to a better clinical protocol creation despite the diffuse and inconclusive information presented in the existing medical literature.

60 APPLIED LIFE SCIENCES↗

Technosocial Predictive Analytics for Illicit Nuclear Trafficking

Illicit nuclear trafficking networks are a national security threat. These networks can directly lead to nuclear proliferation, as state or non-state actors attempt to identify and acquire nuclear weapons-related expertise, technologies, components, and materials. The ability to characterize and anticipate the key nodes, transit routes, and exchange mechanisms associated with these networks is essential to influence, disrupt, interdict or destroy the function of the networks and their processes. The complexities inherent to the characterization and anticipation of illicit nuclear trafficking networks requires that a variety of modeling and knowledge technologies be jointly harnessed to construct an effective analytical and decision making workflow in which specific case studies can be built in reasonable time and with realistic effort. In this paper, we explore a solution to this challenge that integrates evidentiary and dynamic modeling with knowledge management and analytical gaming, and demonstrate its application to a geopolitical region at risk.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Multi-core processing and scheduling performance in CMS

Commodity hardware is going many-core. We might soon not be able to satisfy the job memory needs per core in the current single-core processing model in High Energy Physics. In addition, an ever increasing number of independent and incoherent jobs running on the same physical hardware not sharing resources might significantly affect processing performance. It will be essential to effectively utilize the multi-core architecture. CMS has incorporated support for multi-core processing in the event processing framework and the workload management system. Multi-core processing jobs share common data in memory, such us the code libraries, detector geometry and conditions data, resulting in a much lower memory usage than standard single-core independent jobs. Exploiting this new processing model requires a new model in computing resource allocation, departing from the standard single-core allocation for a job. The experiment job management system needs to have control over a larger quantum of resource since multi-core aware jobs require the scheduling of multiples cores simultaneously. CMS is exploring the approach of using whole nodes as unit in the workload management system where all cores of a node are allocated to a multi-core job. Whole-node scheduling allows for optimization of the data/workflow management (e.g. I/O caching, local merging) but efficient utilization of all scheduled cores is challenging. Dedicated whole-node queues have been setup at all Tier-1 centers for exploring multi-core processing workflows in CMS. We present the evaluation of the performance scheduling and executing multi-core workflows in whole-node queues compared to the standard single-core processing workflows.

Hernandez, J. M.↗

msdlive-cli-distro

MSD-LIVE, the MultiSector Dynamics – Living, Intuitive, Value-adding, Environment, is a flexible and scalable data and code management system combined with a distributed computational platform that will enable MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and multi-model workflows within a robust Community of Practice. MSD-LIVE will facilitate a new open, collaborative, resource-rich, technology-facilitated, community-driven way of doing MSD research.

Lansing, Carina↗

Department of Energy's Biological and Environmental Research Strategic Data Roadmap for Earth System Science

Rapid advances in experimental, sensor, and computational technologies and techniques are driving exponential growth in the volume, acquisition rate, variety, and complexity of scientific data. This wealth of scientifically meaningful data has tremendous potential to lead to scientific discovery. However, to achieve scientific breakthroughs, these data must be exploitable—they must be analyzed effectively and efficiently and the results shared and communicated easily within the wider Department of Energy’s (DOE’s) Biological and Environmental Research (BER) Climate and Environmental Sciences Division (CESD) community. The explosion in data complexity and scale makes these tasks exceedingly difficult to achieve, particularly given that an increasing number of disciplines are working across techniques, integrating simulation and experimental or observational results (see Table 5 in Appendix 2). Consequently, we need new approaches to data management, analysis, and visualization that provide research teams with easy-to-use and scalable end-to-end solutions. These solutions must facilitate (and where feasible, automate and capture) every stage in the data lifecycle (shown in Figure 1), from collection to management, annotation, sharing, discovery, analysis, and visualization. In addition, the core functionalities are the same across climate science communities, but they require customization to adapt to specific needs and fit into research and analysis workflows. To this end, the mission of CESD’s Data and Informatics Program is to integrate all existing and future distributed CESD data holdings into a seamless and unified environment for the acceleration of Earth system science.

54 ENVIRONMENTAL SCIENCES↗

Extending the Publish/Subscribe Abstraction for High-Performance I/O and Data Management at Extreme Scale

The Adaptable I/O System (ADIOS) represents the culmination of substantial investment in Scientific Data Management, and it has demonstrated success for several important extreme-scale science cases. However, looking towards the exascale and beyond, we see the development of yet more stringent data management requirements that require new abstractions. Therefore, there is an opportunity to attempt to connect the traditional realms of HPC I/O optimization with the Database / Data Management community. As such, in this paper we offer some specific examples from our ongoing work in managing data structures, services, and performance at the extreme scale for scientific computing. Using the publish/subscribe model afforded by ADIOS, we demonstrate a set of services that connect data format, metadata, queries, data reduction, and high-performance delivery. The resulting publish/subscribe framework facilitates connection to on-line workflow systems to enable the dynamic capabilities that will be required for exascale science.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Future Intensity‐Duration‐Frequency Curves of Extreme Precipitation in the Midwest United States From Convection‐Permitting Modeling

Abstract During the last four decades, global warming has statistically significant intensified extreme precipitation events in the Midwestern United States (defined here as the region covering Illinois, Indiana, Ohio, and Kentucky), leading to increased risks to human life, property, and infrastructure. To enable climate change adaptation and resilience across various economic and social sectors in this region, updated information about future climate changes, specifically at finer spatial scales, is essential. Leveraging a new 150‐year dynamical downscaling data set at convection‐permitting resolution, this study introduces a framework to construct the projected future intensity‐duration‐frequency (IDF) curves of heavy precipitation, which are prominent tools for infrastructure design and water resources management. This framework generates IDF curves at both sub‐daily and multi‐day duration utilizing hourly in situ observations as well as quantile‐based statistical techniques in bias‐correction and return levels selection. The assumption of non‐stationarity in the distribution parameter fitting process is also implemented in this workflow. Compared to historical IDF curves for 1980–2022, future projected IDF curves for 2058–2100 under Representative Concentration Pathway (RCP) 4.5 and RCP 8.5 scenarios indicate an average intensity increase of approximately 15% and 25%, respectively, across 74 stations, considering both annual and seasonal timescales. Future projections suggest that extreme precipitation events may become more severe across six investigated return periods, with longer return periods showing a greater increase. The frequency of future extreme precipitation events in the Midwest region is also projected to double. Furthermore, current results reveal spatial heterogeneity of future trends across stations owing to the high‐resolution input data set. Plain Language Summary This study investigates the evolving nature of extreme precipitation events in the Midwestern United States under a changing climate. By leveraging a high‐resolution dynamical downscaling data set, we construct projected intensity‐duration‐frequency (IDF) curves for future extreme rainfall events. These curves serve as vital tools for infrastructure planning and water resource management. Our analysis reveals a significant increase in both the intensity and frequency of extreme precipitation events in the region. Future projected IDF curves for the late century indicate an average intensity increase of approximately 15%–25% compared to historical values. Moreover, the frequency of such events is expected to double. Spatial heterogeneity in future trends is observed across different stations within the Midwest, highlighting the importance of high‐resolution modeling in capturing localized climate variability. These findings underscore the urgent need for climate adaptation strategies to mitigate the increasing risks associated with extreme precipitation events in the region. Key Points This study introduces a workflow to construct future intensity‐duration‐frequency (IDF) curves over the Midwest United States using a new convection‐permitting modeling data set The current IDF construction workflow reproduces well the historical observed IDF 30 curves in summer months with median relative errors of 2.4% among 74 stations and 6 investigated durations The projected IDF curves show diverse future trends of extreme precipitation across stations, with intensity increases of approximately 15% and 25% under RCP4.5 and RCP8.5 climate scenarios, respectively, and a doubling of frequency on average

Nguyen, Trung↗

An Analysis Workflow-Aware Storage System for Multi-Core Active Flash Arrays

Here, the need for novel data analysis is urgent in the face of a data deluge from modern applications. Traditional approaches to data analysis incur significant data movement costs, moving data back and forth between the storage system and the processor. Emerging Active Flash devices enable processing on the flash, where the data already resides. An array of such Active Flash devices allows us to revisit how analysis workflows interact with storage systems. By seamlessly blending together the flash storage and data analysis, we create an analysis workflow-aware storage system, AnalyzeThis. Our guiding principle is that analysis-awareness be deeply ingrained in each and every layer of the storage, elevating data analyses as first-class citizens, and transforming AnalyzeThis into a potent analytics-aware appliance. To evaluate the AnalyzeThis system, we have adopted both emulation and simulation approaches. In particular, we have evaluated AnalyzeThis by implementing the AnalyzeThis storage system on top of the Active Flash Array's emulation platform. We have also implemented an event-driven AnalyzeThis simulator, called AnalyzeThisSim, which allows us to address the limitations of the emulation platform, e.g., performance impact of using multi-core SSDs. The results from our emulation and simulation platforms indicate that AnalyzeThis is a viable approach for expediting workflow execution and minimizing data movement.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Science Capsule - Capturing the Data Life Cycle

The data generated from scientific workflows often become unusable due to the lack or incompleteness of information required for processing and analyzing the data. Reproducibility of scientific data and workflows facilitates efficient processing and analyses. A key to enabling reproducibility is to capture the end-to-end workflow life cycle, and any contextual metadata and provenance.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Development of Transient Coupled Chemical-Thermal-Fluids Simulation for Unsealed Aluminum-Clad Spent Nuclear Fuel Storage Canister

This study examines a preliminary three-dimensional (3D) multi-physics computational fluid dynamics (CFD) model of the unsealed canisters containing aluminum-clad spent nuclear fuel from INL’s ATR in the irradiated fuel storage facility. This 3D CFD model considers fully resolved fuel plates in an unsealed canister with small mass exchange with surrounding air as time evolves in the simulation and contains a network of chemical reactions to consider due to several of the side reactions occurring in the presence of water vapor, nitrogen, oxygen and carbon dioxide rather than in helium backfilled DOE standard canisters. Prior to the CFD modeling, the full set of chemical reactions under these conditions was analyzed, reduced, and refit to a more manageable number of equations for the computational efficiency of the 3D CFD simulation. To deal with the long time-scale of interest for tracking the possible buildup of trace species such as nitric acid, NOx, and hydrogen, a segregated time stepping algorithm is used, wherein the velocity field is only updated periodically, and assume to be in a pseudo steady state. This study first examines the theory of the model, workflow to establish radiolysis reaction network, and initial simulations of the evolutions of thermal fields, hydrogen gas and nitric acid concentrations within unsealed canisters in conditions relevant to the INL INTEC CPP-603 facility over a long period of time.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗