Search NASA⌕ Search

SEARCH · Search NASA

Results for “Workflow Management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

A Microservices Architecture Toolkit for Interconnected Science Ecosystems

Microservices architecture is a promising approach for developing reusable scientific workflow capabilities for inte- grating diverse resources, such as experimental and observational instruments and advanced computational and data management systems, across many distributed organizations and facilities. In this paper, we describe how the INTERSECT Open Architec- ture leverages federated systems of microservices to construct interconnected science ecosystems, review how the INTERSECT software development kit eases microservice capability develop- ment, and demonstrate the use of such capabilities for deploying an example multi-facility INTERSECT ecosystem.

Brim, Michael↗

Sandia Machine Learning Laboratory

Samlab provides a workflow and data storage for machine learning experiment inputs and outputs using mongodb, and a web service to manage, search, visualize, and analyze the results.

Shead, Timothy↗

CalyxFlow

CalyxFlow is a lightweight agentic artificial intelligent workflow. This workflow demonstrates the use of AI LLMs to generate modeling and simulation inputs for a scientific simulation and manage execution and analysis of a suite of simulations.

Shipman, Galen↗

STREAM2016: Streaming Requirements, Experience, Applications and Middleware Workshop

The Department of Energy (DOE) Office of Science (SC) facilities including accelerators, light sources and neutron sources and sensors that study, the environment, and the atmosphere, are producing streaming data that needs to be analyzed for next-generation scientific discoveries. There has been an explosion of new research and technologies for stream analytics arising from the academic and private sectors. However, there has been no corresponding effort in either documenting the critical research opportunities or building a community that can create and foster productive collaborations. The two-part workshop series, STREAM: Streaming Requirements, Experience, Applications and Middleware Workshop (STREAM2015 and STREAM2016), were conducted to bring the community together and identify gaps and future efforts needed by both NSF and DOE. This report describes the discussions, outcomes and conclusions from STREAM2016: Streaming Requirements, Experience, Applications and Middleware Workshop, the second of these workshops held on March 22-23, 2016 in Tysons, VA. STREAM2016 focused on the Department of Energy (DOE) applications, computational and experimental facilities, as well software systems. Thus, the role of “streaming and steering” as a critical mode of connecting the experimental and computing facilities was pervasive through the workshop. Given the overlap in interests and challenges with industry, the workshop had significant presence from several innovative companies and major contributors. The requirements that drive the proposed research directions, identified in this report, show an important opportunity for building competitive research and development program around streaming data. These findings and recommendations are consistent with vision outlined in NRC Frontiers of Data and National Strategic Computing Initiative (NCSI) [1, 2]. The discussions from the workshop are captured as topic areas covered in this report's sections. The report discusses four research directions driven by current and future application requirements reflecting the areas identified as important by STREAM2016. These include (i) Algorithms, (ii) Programming Models, Languages and Runtime Systems (iii) Human-in-the-loop and Steering in Scientific Workflow and (iv) Facilities.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Illumina Production Sequencing at the DOE Joint Genome Institute - Workflow and Optimizations

The U.S. Department of Energy (DOE) Joint Genome Institute?s (JGI) Production Sequencing group is committed to the generation of high-quality genomic DNA sequence to support the DOE mission areas of renewable energy generation, global carbon management, and environmental characterization and clean-up. Within the JGI?s Production Sequencing group, the Illumina Genome Analyzer pipeline has been established as one of three sequencing platforms, along with Roche/454 and ABI/Sanger. Optimization of the Illumina pipeline has been ongoing with the aim of continual process improvement of the laboratory workflow. These process improvement projects are being led by the JGI?s Process Optimization, Sequencing Technologies, Instrumentation& Engineering, and the New Technology Production groups. Primary focus has been on improving the procedural ergonomics and the technicians? operating environment, reducing manually intensive technician operations with different tools, reducing associated production costs, and improving the overall process and generated sequence quality. The U.S. DOE JGI was established in 1997 in Walnut Creek, CA, to unite the expertise and resources of five national laboratories? Lawrence Berkeley, Lawrence Livermore, Los Alamos, Oak Ridge, and Pacific Northwest ? along with HudsonAlpha Institute for Biotechnology. JGI is operated by the University of California for the U.S. DOE.

59 BASIC BIOLOGICAL SCIENCES↗

Managing a Cray supercomputer as a git branch

Summary Modern software engineering/DevOps techniques applied to Cray XC Series systems management enable higher fidelity translation from test to production environments, reduce administration costs by avoiding duplicate efforts, and increase reliability. This can be done by using a git repository to track and manage all system configurations on the SMW(s) (System Management Workstation) and then adapt a gitflow‐like development methodology. Using git improves change management with peer review of changes, identification of who made a change, knowing why a change was made, facilitating easier reversion of bad changes, and enables a workflow using the ideas from gitflow with individual branches for preparing, maintaining, and recording releases. Using branches enables multiple contributors to work on different types of changes without getting unexpected changes from another feature area. We will describe a process similar to that used at NERSC (National Energy Research Scientific Computing Center) to extract/abstract configuration data, manage it using git, and then re‐apply changes to the SMW(s).

Jacobsen, Douglas M.↗

A Recipe for Streamlining Mission Management

This paper describes a project's design and implementation for streamlining mission management with knowledge capture processes across multiple organizations of a NASA directorate. Thc project's focus is on standardizing processes and reports; enabling secure information access and case of maintenance; automating and tracking appropriate workflow rules through process mapping; and infusing new technologies. This paper will describe a small team's experiences using XML technologies through an enhanced vendor suite of applications integrated on Windows-based platforms called the Wallops Integrated Scheduling and Document Management System (WISDMS). This paper describes our results using this system in a variety of endeavors, including providing range project scheduling and resource management for a Range and Mission Management Office; implementing an automated Customer Feedback system for a directorate; streamlining mission status reporting across a directorate; and initiating a document management, configuration management and portal access system for a Range Safety Office's programs. The end result is a reduction of the knowledge gap through better integration and distribution of information, improved process performance, automated metric gathering, and quicker identification of problem areas and issues. However, the real proof of the pudding comes through overcoming the user's reluctance to replace familiar, seasoned processes with new technology ingredients blended with automated procedures in an untested recipe. This paper shares some of the team's observations that led to better implementation techniques, as well as an IS0 9001 Best Practices citation. This project has provided a unique opportunity to advance NASA's competency in new technologies, as well as to strategically implement them within an organizational structure, while wetting the appetite for continued improvements in mission management.

Mitchell, Andrew E.↗

Evolution of LIBS technology to mobile instrumentation for expediting firearm-related investigations at the laboratory and the crime scene

Gunshot residue (GSR) is one of the few forensic disciplines that lack accurate screening techniques. This study proposes using a mobile LIBS instrument to detect inorganic GSR and compares performance to a previously validated laboratory instrument. The mobile LIBS is designed with advanced configurations specifically for on-site GSR analysis, including a CMOS detector and a sampling chamber that holds up to six typical GSR collection devices with separate gas flow ports to prevent cross-contamination. A significant novelty of the portable instrument is its image magnification, which allows quick searching and visualization of GSR particle morphology. The single-particle imaging and elemental composition capability is one of a kind and offers superior confirmatory features for GSR. The mobile LIBS performance was evaluated for residues collected from the hands of shooters (100 samples) and non-shooters (200 background samples), analyzed sequentially by the mobile instrument and then the laboratory instrument. Accuracies better than 98.8% were obtained by both instruments, demonstrating their suitability for trace IGSR detection from skin specimens. Implementation of this methodology is anticipated to drastically speed up response times (i.e., from several hours per sample by standard SEM-EDS practice to a few minutes by LIBS). The screening methods can be easily incorporated into workflows to improve decision-making processes at the crime scene and laboratory settings, reduce backlogs, and improve case management.

47 OTHER INSTRUMENTATION↗

SU-F-T-223: Radiotherapy Incident Reporting and Analysis System (RIRAS):Early Experience

Background & Purpose: RIRAS is a web-based information system deployed on the Veterans Health Administration intranet in early 2014 to collect adverse events and good catch data; analyze the causes and contributing factors; and find ways to prevent future occurrences. Material and Methods: Incident learning consists of a feedback loop which starts with reporting an event, followed by analysis of contributing factors, and culminates in the development of a patient safety work product (PSWP) to prevent recurrence. RIRAS permits both anonymous and non-anonymous reporting. Each report is analyzed by a team of medical physicists who are independent of the reporting facility. The analysts usually contact the reporting facilities for additional information. We analyzed all reports and held telephonic interviews (when necessary) with the reporters. We then generated PSWPs with corrective/preventive and learning actions. Anonymous reporting is handled in the same manner, except without the ability to further interview the reporter. Results: In a significant number of reports, the causes and recommended preventive actions were considerably altered by the independent analysis and additional information from the facility. 130 reports have been entered in RIRAS; 9 misadministrations, 83 good catches, 3 anonymous good catches, and 35 earlier reported incidents from FY2005-14. 45% of the reported incidents occurred in the treatment delivery stages, 19% in on-treatment management, and 16% in pre-treatment verification. 80% of the good catches were found in the treatment delivery workflow. Majority of these incidents were due to inconsistent patient setup instructions or documentation, nonadherence to policies and procedures, lax time-out policy, distracted RTTs, and inadequate RTT staffing. Conclusion: RIRAS has identified many areas for improvement and elevated the quality and safety of radiation treatments in the VHA. We found that the ability to learn is significantly diminished when the analysts do not have the ability to request additional information.

60 APPLIED LIFE SCIENCES↗

Deep Learning–Assisted Multiobjective Optimization of Geological CO 2 Storage Performance under Geomechanical Risks

In geological CO 2 storage, designing the optimal well control strategy for CO 2 injection to maximize CO 2 storage while minimizing the associated geomechanical risks is not trivial. This challenge arises due to pressure buildup, CO 2 plume migration, the highly nonlinear nature of geomechanical responses to rock-fluid interaction, and the high computational cost associated with coupled flow and geomechanics simulations. In this paper, we introduce a novel optimization framework to address these challenges. The optimization problem is formulated as follows: maximize total CO 2 storage while minimizing geomechanical risks by adjusting the injection schedules within bounded constraints. The geomechanical risks are primarily driven by injection-induced pressure build-up, which is characterized by ground displacement and the induced microseismicity. We used the Fourier neural operator (FNO)-based deep learning model to construct surrogate models, replacing the time-consuming coupled flow and geomechanics simulations for evaluating the aforementioned objective functions. The developed surrogate models have been incorporated into a multiobjective optimization framework through a genetic algorithm to reduce the computational burden. The proposed optimization framework reduces the computational cost from approximately 2,400 hours, when using objective function evaluations based on physics-based simulations, to around 20 minutes. A set of Pareto-optimal solutions of the proposed workflow yields nontrivial optimal decisions, reducing the microseismicity potential and the vertical displacement. This Pareto front highlights the optimal trade-offs between CO 2 storage amount, safety, and ground displacement, emphasizing the need for careful optimization and management of injection strategies to achieve a balanced outcome. The novelty of this work is twofold. First, we demonstrate the importance of incorporating the minimization of the geomechanical risks as objective functions into the CO 2 storage optimization workflow to mitigate the potential risk of induced microseismicity and ground displacement. Second, we leverage the FNO-based surrogate models to optimize a real-field CO 2 storage operation.

42 ENGINEERING↗

Digital Tools for the Preventive Conservation of Built Heritage: The Church of Santa Ana in Seville

Historic Building Information Modelling (HBIM) plays a pivotal role in heritage conservation endeavours, offering a robust framework for digitally documenting existing structures and supporting conservation practices. However, HBIM’s efficacy hinges upon the implementation of case-specific approaches to address the requirements and resources of each individual asset and context. This paper defines a flexible and generalisable workflow that encompasses various aspects (i.e., documentation, surveying, vulnerability assessment) to support risk-informed decision making in heritage management tailored to the peculiar conservation needs of the structure. This methodology includes an initial investigation covering historical data collection, metric and condition surveys and non-destructive testing. The second stage includes Finite Element Method (FEM) modelling and structural analysis. All data generated and processed are managed in a multi-purpose HBIM model. The methodology is tested on a relevant case study, namely, the church of Santa Ana in Seville, chosen for its historical significance, intricacy and susceptibility to seismic action. The defined level of detail of the HBIM model is sufficient to inform the structural analysis, being balanced by a more accurate representation of the alterations, through linked orthophotos and a comprehensive list of alphanumerical parameters. This ensures an adequate level of information, optimising the trade-off between model complexity, investigation time requirements, computational burden and reliability in the decision-making process. Field testing and FEM analysis provide valuable insight into the main sources of vulnerability in the building, including the connection between the tower and nave and the slenderness of the columns.

Chaves, Estefanía↗

Expanding Repository Data Available For Sharing and Knowledge Discovery

Some of the hardest space biology and space health challenges require data-intensive, bioinformatic, meta-analytical, and computer-assisted research approaches. These challenges include examining interdisciplinary space life science research across experiments and across interacting spaceflight hazards (radiation, altered gravity, confinement, hostile-closed environments, distance-duration from Earth). The approaches to confront these challenges involve mining multiple datasets simultaneously from various hierarchical organizations of biological complexity, all while concurrently evaluating how experimental design factors affect endpoints of standard assays. To enable this field, it is essential that principal investigators (PIs) submit data in a structure so it can be maximally re-used. The purpose of the NASA Ames Life Sciences Data Archive (ALSDA) is to collect, curate, and make publicly available all non-human space-relevant biological data. ALSDA must also ensure data are open-access, and maximally findable, accessible, interoperable, and reusable (FAIR). The scope of ALSDA data collected and submitted by PIs include subject and study design metadata, assay metadata parameters, raw and processed assay data, assay imagery/video, and subject-experienced mission data telemetry (radiation, temperature, humidity, acoustics, vibrations, etc.). ALSDA recently integrated into a collaborative group of Open Science projects to facilitate a suite of new tools and workflows that will improve data submission, accessibility, and reusability by implementing digital data submission agreements, and adopting the data management system originally developed by NASA GeneLab. ALSDA intends to bring current biological repository data and all future collected data into this new scientific data reuse reality. This new suite of tools will enable ALSDA to deploy a science curation system using scientific assay configurations for the data submission portal. It will capture essential assay parameters according to established standards in each sub-field within biology. The submission portal expedites data collection by enhancing ease of PI data submission, providing a user interface and specificity for which data is to be submitted. Data submissions can be brought into cutting-edge informatic analysis portals to enable mining of physiological, behavioral, biochemical, and imaging datasets in conjunction with ‘omics-level datasets. As ALSDA datasets are submitted, curated, and published (e.g., micro-computed tomography, histology, pulse oximetry, serum metabolites, magnetic resonance imaging, intraocular pressure, novel object recognition, etc.), the merging together of spaceflight data along this multi-hierarchical complexity of biology will enable informatics and data-intensive approaches resulting in knowledge discoveries across missions, space hazards, and biological disciplines.

Biology↗

Expanding Repository Data Available For Sharing And Knowledge Discovery

Some of the hardest space biology and space health challenges require data-intensive, bioinformatic, meta-analytical, and computer-assisted research approaches. These challenges include examining interdisciplinary space life science research across experiments and across interacting spaceflight hazards (radiation, altered gravity, confinement, hostile-closed environments, distance-duration from Earth). The approaches to confront these challenges involve mining multiple datasets simultaneously from various hierarchical organizations of biological complexity, all while concurrently evaluating how experimental design factors affect endpoints of standard assays. To enable this field, it is essential that principal investigators (PIs) submit data in a structure so it can be maximally re-used. The purpose of the NASA Ames Life Sciences Data Archive (ALSDA) is to collect, curate, and make publicly available all non-human space-relevant biological data. ALSDA must also ensure data are open-access, and maximally findable, accessible, interoperable, and reusable (FAIR). The scope of ALSDA data collected and submitted by PIs include subject and study design metadata, assay metadata parameters, raw and processed assay data, assay imagery/video, and subject-experienced mission data telemetry (radiation, temperature, humidity, acoustics, vibrations, etc.). ALSDA recently integrated into a collaborative group of Open Science projects to facilitate a suite of new tools and workflows that will improve data submission, accessibility, and reusability by implementing digital data submission agreements, and adopting the data management system originally developed by NASA GeneLab. ALSDA intends to bring current biological repository data and all future collected data into this new scientific data reuse reality. This new suite of tools will enable ALSDA to deploy a science curation system using scientific assay configurations for the data submission portal. It will capture essential assay parameters according to established standards in each sub-field within biology. The submission portal expedites data collection by enhancing ease of PI data submission, providing a user interface and specificity for which data is to be submitted. Data submissions can be brought into cutting-edge informatic analysis portals to enable mining of physiological, behavioral, biochemical, and imaging datasets in conjunction with ‘omics-level datasets. As ALSDA datasets are submitted, curated, and published (e.g., micro-computed tomography, histology, pulse oximetry, serum metabolites, magnetic resonance imaging, intraocular pressure, novel object recognition, etc.), the merging together of spaceflight data along this multi-hierarchical complexity of biology will enable informatics and data-intensive approaches resulting in knowledge discoveries across missions, space hazards, and biological disciplines.

life science↗

Enabling On-Demand Scientific Workflows on a Federated Cloud

The Fermilab Grid and Cloud Computing Department and the KISTI Global Science experimental Data hub Center are working on a multi-year Collaborative Research and Development Agreement.With the knowledge developed in the first year on how to provision and manage a federation of virtual machines through Cloud management systems. In this second year, we expanded the work on provisioning and federation, increasing both scale and diversity of solutions, and we started to build on-demand services on the established fabric, introducing the paradigm of Platform as a Service to assist with the execution of scientific workflows. We have enabled scientific workflows of stakeholders to run on multiple cloud resources at the scale of 1,000 concurrent machines. The demonstrations have been in the areas of (a) Virtual Infrastructure Automation and Provisioning, (b) Interoperability and Federation of Cloud Resources, and (c) On-demand Services for ScientificWorkflows.

97 MATHEMATICS AND COMPUTING↗

Exascale workflow applications and middleware: An ExaWorks retrospective

Exascale computers offer transformative capabilities to combine data-driven and learning-based approaches with traditional simulation applications to accelerate scientific discovery and insight. However, these software combinations and integrations are difficult to achieve due to the challenges of coordinating and deploying heterogeneous software components on diverse and massive platforms. Here, we present the ExaWorks project, which addresses many of these challenges. We developed a workflow Software Development Toolkit (SDK), a curated collection of workflow technologies that can be composed and interoperated through a common interface, engineered following current best practices, and specifically designed to work on HPC platforms. ExaWorks also developed PSI/J, a job management abstraction API, to simplify the construction of portable software components and applications that can be used over various HPC schedulers. The PSI/J API is a minimal interface for submitting and monitoring jobs and their execution state across multiple and commonly used HPC schedulers. We also describe several leading and innovative workflow examples of ExaWorks tools used on DOE leadership platforms. Furthermore, we discuss how our project is working with the workflow community, large computing facilities, and HPC platform vendors to address the requirements of workflows sustainably at the exascale.

97 MATHEMATICS AND COMPUTING↗

User-Focused Tools to Enhance IT/OT Cyber Resilience within the Power Grid

The power grid is undergoing several changes that are increasing its complexity as nexuses between electric-gas, transmission-distribution, and energy-communications continue to become increasingly critical. This system is heavily dependent on communication infrastructure and controls, and it relies on humans in operational technology (OT) and information technology (IT) roles to manage the increasing breadth, depth, and speed of data. Many technical challenges have presented themselves and will need to be addressed to provide reliable grid operations. With increased reliance on distributed controls and communication infrastructure, cybersecurity becomes an inherent requirement. When considering current cyber-physical security solutions for the power grid, one can notice a clear divide between information technology and operation technology networks. However, in real-life applications, these networks are interdependent. This work presents results of interviews with key utility cybersecurity personnel, analyzes the results, and makes recommendations towards solution of existing technical and operational challenges realized. Existing workflows are presented, wireframe interviews are discussed, and tool requirements are described. The existence of easy-to-implement solutions, based on existing energy management systems, highlight the potential for real-life applications.

cybersecurity, resilience, user-centered design, p↗

Partnership For Edge Physics Simulation

In this effort, we will extend our prior work as part of CPES (i.e., DART and DataSpaces) to support in-situ tight coupling between application codes that exploits data locality and core-level parallelism to maximize on-chip data exchange and reuse. This will be accomplished by mapping coupled simulations so that the data exchanges are more localized within the nodes. Coupled simulation workflows can more effectively utilize the resources available on emerging HEC platforms if they can be mapped and executed to exploit data locality as well as the communication patterns between application components. Scheduling and running such workflows requires an extended framework that should provide a unified hybrid abstraction to enable coordination and data sharing across computation tasks that run on the heterogeneous multi-core-based systems, and develop a data-locality based dynamic tasks scheduling approach to increase on-chip or intra-node data exchanges and in-situ execution. This effort will extend our prior work as part of CPES (i.e., DART and DataSpaces), which provided a simple virtual shared-space abstraction hosted at the staging nodes, to support application coordination, data sharing and active data processing services. Moreover, it will transparently manage the low-level operations associated with the inter-application data exchange, such as data redistributions, and will enable running coupled simulation workflow on multi-cores computing platforms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Argonne Leadership Computing Facility (2018 Annual Report)

The Argonne Leadership Computing Facility enables breakthroughs in science and engineering by providing supercomputing resources and expertise to the research community. In 2018, we continued to build up programs and efforts that reinforce our overall simulation, learning, and data analysis capabilities as a facility. We also enhanced our services to better meet our users’ needs and workflow preferences.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗