Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data science, data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Predicting future well performance for environmental remediation design using deep learning

Here in this study, we developed a deep learning (DL) framework with a multi-channel three-dimensional convolutional neural network (MC3D-CNN) to predict well performance and thereby assist future environmental remediation design. Such prediction of extraction well performance at designated locations is critical for configuring pump-and-treat (P&T) well network design and operation, setting reasonable target closure dates for overall remedying, and estimating remedy costs. The framework is developed with operational and monitoring data routinely collected during P&T remedy operations, including well extraction and injection rates as well as in situ contaminant concentrations. Traditionally, the collected data were rarely used for purposes other than assessing past well performance and the accuracy of the conceptual site model. However, recent advances in data-driven computational approaches enable better use of the large datasets to inform future well performance, enhance site characterization, and improve remediation planning. In this study, we established a DL framework to integrate transient three-dimensional contaminant plumes and multiple aquifer properties (e.g., hydraulic conductivity and hydrostratigraphic maps) to identify characteristic patterns controlling and representing extraction well mass recovery, aiming at providing future mass recovery estimates for existing wells and candidate wells at any proposed locations. We evaluated our framework by using a realistic synthetic dataset generated from a well-calibrated flow and transport model used in the 200 West Area of the U.S. Department of Energy’s Hanford Site in southeastern Washington state. The multi-channel feature in our framework allows integration of various types and temporal densities of training datasets for DL model development. Overall, we found that the trained DL model achieved an accuracy of over 90% in ranking extraction well performance in validation datasets, and over 80% in predicting high-performance-ranking well locations. This data-informed approach provides a flexible tool to support adaptive site management, streamline decision-making, and potentially reduce remediation time and costs. Our DL framework can be used as a filtering tool to improve the current P&T network optimization design by reducing the number of candidate well locations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

High-performance computing in water resources hydrodynamics

In this work, we present a vision of future water resources hydrodynamics codes that can fully utilize the strengths of modern high-performance computing. The advances to computing power, formerly driven by the improvement of central processing unit processors, now focus on parallel computing and, in particular, the use of graphics processing units (GPUs). However, this shift to a parallel framework requires refactoring the code to make efficient use of the data as well as changing even the nature of the algorithm that solves the system of equations. These concepts along with other features such as the precision for the computations, dry regions management, and input/output data are analyzed in this paper. A 2D multi-GPU flood code applied to a large-scale test case is used to corroborate our statements and ascertain the new challenges for the next-generation parallel water resources codes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

NNSA/CEA Workflow (Workshop Report)

This report, prepared by the NNSA/CEA Workflows Working Group, briefly summarizes the presentations in the areas of domain specific workflows, end user environments, data management, job and resource management, and infrastructure, and then identifies six broad areas for potential collaboration. A key finding is that users could benefit from greater interoperability, compatibility, and composability of the workflow technologies under development and that point-to-point collaboration opportunities should be identified to explore these aspects.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

Sensitive Resources Assessment and Forest Analysis for the Proposed Versatile Test Reactor, Oak Ridge, Tennessee

The US Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) is a leading institution in advanced materials, supercomputing, neutrons, and nuclear science. As a research laboratory managed by UT-Battelle, LLC for DOE, ORNL has national priorities in energy, security, and scientific discovery that necessitate facility improvements and expansions. DOE is also committed to environmental stewardship. The laboratory is located on the ~32,000-acre (~13,000-ha) Oak Ridge Reservation (ORR), much of which is categorized as a National Environmental Research Park (NERP) and a state Wildlife Management Area. DOE works with the Tennessee Wildlife Resources Agency (TWRA), Tennessee Department of Environment and Conservation (TDEC), US Fish and Wildlife Service (USFWS), US Department of Agriculture, and other agencies to serve as an effective steward of the ORR. Accordingly, project managers must conform to environmental regulations, agreements, and policies at the federal, state, and institutional levels. Per 40 CFR (Code of Federal Regulations) 1508.14, potential effects on research and science education also represent potential effects of federal actions on the NERP, and impacts on, e.g., deer harvest, must be considered on the Oak Ridge Wildlife Management Area when other aspects of the human environment are affected. The United States currently has no fast neutron testing capability to support advanced nuclear research and development. The proposed Versatile Test Reactor (VTR) will take advantage of current investments by the US government and private industry in nuclear reactors to expedite the design and construction process, using proven technology to create a world-class scientific infrastructure. The VTR will take advantage of fast neutrons provided by this proven technology, along with a capability to rapidly insert, conduct, and remove state-of-the-art experiments. An advantage of the VTR is that it can support future innovations in experimental capabilities without modifying the facility. The VTR will support progress in a variety of science and technology areas, including testing and qualification of advanced reactor fuels; testing and qualification of innovative structural materials; testing of innovative components and instruments; validation of advanced modeling and simulation tools; and versatility for future technical missions. Through proven technology, the VTR can take advantage of existing reactor designs and operating experience to reduce the risk, cost, and time for design and construction. The top available resources of DOE laboratories, industry, and universities will be used to expedite reactor design and construction toward developing the scientific infrastructure that affords a strong testing capability that can be sustained over many years. This report summarizes current knowledge of natural and cultural resources primarily within the VTR construction area. At the time of this report, the proposed VTR site design includes a construction area of ~150.4 acres (~69.9 ha), which contains an ~51.3-acre (~20.8-ha) operations area, located within forested natural areas of the ORR. The primary goal of the work presented here was to evaluate potential effects on sensitive resources that might result from development and construction activities associated with VTR. In addition to on-the-ground surveys during spring and summer 2020 by the ORNL Natural Resources Management Program and Aquatic Ecology Group staff, this report makes use of historical (pre-1995) and contemporary (1995 to present) data from additional confirmed sources (e.g., TDEC). Likewise, forest conditions were compiled from a 2011 forest inventory and supplemented with limited ground observations in 2020. The individuals who obtained and compiled the data presented here are familiar with and routinely assess sensitive resources on the ORR. Anyone who references this report must consider that the timing of surveys did not permit a complete delineation of the resources that will be affected. If the VTR project proceeds, additional surveys will be required to account for the seasonal patterns of various threatened and endangered species. Data deficiencies and potential resources that likely went undetected are indicated where possible. Accordingly, this report should facilitate more environmentally sound decisions during planning and 2 development of the VTR site, provide a foundation for further assessment of sensitive and cultural resources, and help project managers better address regulatory guidance and DOE policies on sustainable development in compliance with, for example, the US Endangered Species Act (ESA), Migratory Bird Treaty Act (MBTA), Tennessee Rare Plant Protection and Conservation Act of 1985, Tennessee Nongame and Endangered or Threatened Wildlife Species Conservation Act of 1974, several federal and state regulations regarding aquatic resource protection, and site-specific policies as outlined in various ORR management plans developed by ORNL and TWRA for DOE.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

2022 Quantum Materials Young Investigators Workshop Report

The fifth edition of the workshop “Quantum Materials Young Investigators” was held August 18-19, 2022 at the Oak Ridge National Laboratory (ORNL). This workshop followed up on previous meetings that took place at ORNL starting in 2016. This important series of annual workshops, organized by Adam Aczel and Stuart Calder, ensures continuous engagement with the early career principal investigators (PIs) of the quantum materials neutron scattering community. Through a series of short scientific talks presented by the external PIs, members of the Neutron Scattering Division (NSD) learn about their scientific interests and needs. Presentations from ORNL staff members also help to inform the external PIs about the current state of ORNL’s quantum materials neutron scattering program. Topics covered include neutron scattering instruments, sample environment, neutron polarization, data reduction, visualization, and analysis, and future neutron scattering and complementary capabilities. Feedback is solicited from the external PIs in some or all of these key areas and communicated to NSD management in the form of a workshop report. This workshop series also helps to foster collaborations between NSD scientists and the research groups of the external PIs and plays an important role in attracting new neutron scattering users.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Identifying Controlling Variables Related to Mercury Vapor Concentrations in Legacy Facilities: SRNL Technology Development For D&D

A suite of sensor packages has been installed in the Alpha-4 facility at Y-12 and preliminary data analysis to isolate the controlling variables has been performed after two months of data collection. In line with the conceptual model presented in Section 2.0, the preliminary analysis has demonstrated that temperature, barometric pressure, and relative humidity are the key controlling variables for mercury vapor releases within the building. The data analysis will continue to explore these controlling variables across the next several months to capture a more complete picture of the annual change in meteorological conditions at Y-12.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Policy-Driven Sustainable Saline Drainage Disposal and Forage Production in the Western San Joaquin Valley of California

Environmental policies to address water quality impairments in the San Joaquin River of California have focused on the reduction of salinity and selenium-contaminated subsurface agricultural drainage loads from westside sources. On 31 December 2019, all of the agricultural drainage from a 44,000 ha subarea on the western side of the San Joaquin River basin was curtailed. This policy requires the on-site disposal of all of the agricultural drainage water in perpetuity, except during flooding events, when emergency drainage to the River is sanctioned. The reuse of this saline agricultural drainage water to irrigate forage crops, such as ‘Jose’ tall wheatgrass and alfalfa, in a 2428 ha reuse facility provides an economic return on this pollutant disposal option. Irrigation with brackish water requires careful management to prevent salt accumulation in the crop root zone, which can impact forage yields. The objective of this study was to optimize the sustainability of this reuse facility by maximizing the evaporation potential while achieving cost recovery. This was achieved by assessing the spatial and temporal distribution of the root zone salinity in selected fields of ‘Jose’ tall wheatgrass and alfalfa in the drainage reuse facility, some of which have been irrigated with brackish subsurface drainage water for over fifteen years. Electromagnetic soil surveys using an EM-38 instrument were used to measure the spatial variability of the salinity in the soil profile. The tall wheatgrass fields were irrigated with higher salinity water (1.2–9.3 dS m -1 ) compared to the fields of alfalfa (0.5–6.5 dS m -1 ). Correspondingly, the soil salinity in the tall wheatgrass fields was higher (12.5 dS m -1 –19.3 dS m -1 ) compared to the alfalfa fields (8.97 dS mm -1 –14.4 dS mm -1 ) for the years 2016 and 2017. Better leaching of salts was observed in the fields with a subsurface drainage system installed (13–1 and 13–2). The depth-averaged root zone salinity data sets are being used for the calibration of the transient hydro-salinity computer model CSUID-ID (a one-dimensional version of the Colorado State University Irrigation Drainage Model). This user-friendly decision support tool currently provides a useful framework for the data collection needed to make credible, field-scale salinity budgets. In time, it will provide guidance for appropriate leaching requirements and potential blending decisions for sustainable forage production. This paper shows the tie between environmental drainage policy and the role of local governance in the development of sustainable irrigation practices, and how well-directed collaborative field research can guide future resource management.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Vegetation water sources in California's Sierra Nevada (USA) are young and change over time, a multi-isotope ( δ 18 O, δ 2 H, 3 H) tracer approach.

Sierra Nevada forests transpire a significant amount of California's water resources, sparking interest in applying forest management to improve California's water supply. Determining the source water of evapotranspiration enables forest managers to make informed decisions. To this end, a significant interest in critical zone science is to develop new methods to work across time scales to predict subsurface water storage and use. Here, forest vegetation accessed young water and switched sources depending on availability, suggesting that forest drought vulnerability may depend on the range of water sources available (rain, snowmelt and deeply stored water). This finding also suggests that changes in transpiration rates may have immediate effects on water sources in close proximity to vegetation, and delayed effects on storage and runoff. New δ 18 O, δ 2 H and 3 H data were used to track precipitation, runoff, evapotranspiration and storage through the critical zone seasonally, including seasons where evapotranspiration and snowmelt were in phase (winter snowmelt) and out of phase (seasonally dry summer). The main source of this headwater catchment's runoff is derived from its meadow saturated zone water, which was dominated by snowmelt. Water that originated as snowmelt contributed to transpiration, unless other sources, such as recent rain, became available. In cases where xylem δ 18 O and δ 2 H signatures matched those of deeper saturated zone water, 3 H data showed that xylem water was distinctly younger than the deep saturated zone water. During 2016, which experienced relatively normal snowpack in winter and seasonally dry summer conditions, mean summer saturated zone water and vegetation water were similar in δ 18 O, –12.4 ± 0.04 ‰ and – 12.5 ± 0.3 ‰, respectively, but were distinctly different in 3 H, 5.5 ± 0.2 pCi/L and 13.7 ± 1.1 pCi/L, respectively. While δ 18 O shows that vegetation and meadow saturated zone water have similar origins, 3 H shows they have dissimilar ages.

58 GEOSCIENCES↗
DOE OSTI2021

The Plant a Trillion Trees Campaign to Reduce Global Warming – Fleshing Out the Concept

The campaign to plant a trillion trees provided an easily understood approach to reduce the threat of global warming. However, focusing on trees does not consider that a maturing forest releases carbon (C) from dying trees offsetting C intake from growth of other trees, and results in only a one-time carbon storage benefit. Under sustainable management that harvests before forest growth slows down, forest C is transferred to wood products that substitute for fossil fuel (FF)- intensive products, reducing the carbon dioxide (CO 2 ) emissions with every harvest. Sustainably managed forests maintain growth by replanting (or regenerating), sustaining the C stored in products, as well as avoiding FF-intensive product emissions with every harvest. This paper addresses the question: Could planting more trees significantly reduce atmospheric CO 2 ; and if so, how? Survey data from many wood-producing mills show life cycle inventory (LCI) data on C that has been tracked through every stage of processing for many different wood products specific to each United States (USA) supply region, as well as C displacement measurements when substituting wood for FFintensive products. Results show that more and better-managed forest lands and more efficient product uses provide significant opportunities to reduce CO 2 emissions, especially in highly productive forest regions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

EMP - Environmental Radiological Air Monitoring Plan: PNNL Operations in Washington

The Environmental Radiological Air Monitoring Plan (EMP) for Pacific Northwest National Laboratory (PNNL) describes systems/processes/practices related to radiological operations in Richland and Sequim, Washington, that are associated with environmental radiological air monitoring and surveillance activities. The activities described support the lab’s responsibility to maintain safe operations and minimize negative impacts to both onsite and offsite persons and environment. Dose assessments required by regulations and DOE Orders for the public and biota are described. PNNL conducts environmental air surveillance monitoring as part of the PNNL Site Radioactive Air Emissions License (RAEL)-005 for Richland Campus, issued in 2010 with its most recent renewal effective in January 2021. The radioactive air emissions license for the PNNL-Sequim campus (RAEL-014) was issued to the U.S. Department of Energy in 2012 with its most recent renewal effective in January 2023. The EMP is a compilation of the following four documents: - Environmental Radiological Air Monitoring Plan (this main document) (PNNL-20919) - Sampling and Analysis Plan (Attachment 1) (PNNL-20919-1) - Data Management Plan (Attachment 2) (PNNL-20919-2) - Dose Assessment Guidance (Attachment 3) (PNNL-20919-3).

40 CFR 61 Subpart H↗
DOE OSTI2023

Enhanced Hanford High-Fluoride Waste Glass Property Data Development: Phase 1

This study focused on investigating the effects of fluorine concentration on simulated high-level waste glass properties to eventually establish a fluorine limit (as a single-component or multiple-component constraint) for glass formulations for high-fluoride Hanford wastes. This is a first step to provide data to understand the impacts of changing flowsheets on the mission duration and extent. A test matrix of 20 high-fluoride glasses was generated, and the chemical compositions were measured. The following properties were measured and tested against current model predictions: crystal formation after centerline canister cooling, crystallinity as a function of temperature, density, viscosity, electrical conductivity, toxic leaching characteristics using the toxicity characteristic leach profile (TCLP), product consistency using the product consistency test (PCT), and SO 3 solubility. Overall, current models failed to adequately predict most of the properties, possibly due to differences in compositional space used to generate the models and the current test matrix. Additional work is needed to more accurately assess the impacts of high-fluoride wastes on Hanford processing, including additional data collection over a broader composition region and model development for the key models of interest such as PCT and TCLP.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Enhancing Resilience of Urban Systems Against Climate-Induced Floods Using Advanced Data-Driven and Computing Techniques: A Driver-Pressure-State-Impact-Response (DPSIR) Framework

Focal Area: This white paper proposes advanced data-driven and computing techniques to improve the predictability of extreme climate events (precipitation) and focuses on comprehending their impact dynamics to enhance resilience of urban systems. Science Challenge: In urban systems, climate change has presented significant challenges related to accurate spatiotemporal prediction of extreme climate events (heavy precipitation) and their impacts, i.e. flooding. The occurrence of extreme climate events and their compounding repercussions on the built environment make urban systems increasingly more vulnerable. The interactions between extreme climate events, urban systems, and the corresponding impacts are irregular, non-linear, and dynamic, making it difficult to standardize resilience. As it stands, the management of complex urban systems is affected by uncertainties related to the nature of short- and long-term consequences caused by extreme events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Schema Elements for Granta Annual Report: FY2022

Granta: Materials Intelligence, also known as Granta:MI or Granta, is a commercial database software by Ansys, Inc. that is utilized by the Nuclear Security Enterprise (NSE) to organize and store relevant materials data. For a complete discussion of the use of Granta:MI at NSE sites, see the FY21 annual report. Granta:MI is used by five NSE sites locally, and all NSE sites have access to an enterprise instance on the Enterprise Secure Network (ESN) as well as an unclassified development instance. It has been recognized by NNSA management that a shared repository for additive manufacturing (AM) data would not only ensure data and knowledge is not lost but would provide a pool of information relating AM inputs (build parameters, raw materials properties, post-processing information) to the as-built properties of AM parts. Such a pool of data would enable optimization of AM build design and help NNSA achieve the goals of shortening fielding times for new components.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Optimizing Alabama’s CO 2 Storage in Shelby County (Project OASIS) - Deliverable Task 7: Risk Assessment

The “Optimizing Alabama’s CO 2 Storage in Shelby County: Project OASIS” CarbonSAFE Phase II Project seeks to build on regional data sets that demonstrate that the subsurface within Shelby County, Alabama has the potential to store commercial volumes of CO 2 safely, permanently, and economically. The primary target CO 2 Storage Complex is the deep Ketona Dolomite located within a 140 square mile area of the Valley and Ridge Region of Alabama. This deep saline reservoir is beneath a confining system encompassing at least 6,500 ft of shales and other low permeability sediments. Project OASIS drilled a deep stratigraphic test well to confirm the geological properties of the confining system and storage reservoir(s) within the Storage Complex. The geological data was incorporated into numerical models to establish the areal extent of the CO 2 plume and help design the storage site and its monitoring system. The project is managed by the Southern States Energy Board (SSEB), an interstate compact organization consisting of governors and state legislative leaders from sixteen southern states, Puerto Rico, and the U.S. Virgin Islands, as well as an appointee by the President of the United States. The organizational compact provides it with access to state government organizations and legislatures. The Board also maintains an Associate Members program comprised of energy resource companies, utilities, trade groups, academic R&D science and technology experts and energy consultants. Further, the SSEB staff is experienced in managing and coordinating complex energy and environmental programs, from research programs to full-scale design and demonstrations of new and innovative technologies. Project OASIS is a public-private partnership of six entities with multiple principal investigators (PIs). SSEB’s Lead PI and Co-PI are responsible for all aspects of project performance in accordance with the DOE-NETL Cooperative Agreement. SSEB has issued subgrants to Advanced Resources International, Inc., Alabama A&M University, Auburn University, Crescent Resource Innovation, and Oklahoma State University. Advanced Resources International, Inc., issued subgrants to Baker Hughes and Loudon Technical Services for field services

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2023

Warming of the Willamette River, 1850–present: the effects of climate change and river system alterations

Abstract. Using archival research methods, we recovered and combined data from multiple sources to produce a unique, 140-year record of daily water temperature (Tw) in the lower Willamette River, Oregon (1881–1890, 1941–present). Additional daily weather and river flow records from the 1850s onwards are used to develop and validate a statistical regression model of Tw for 1850–2020. The model simulates the time-lagged response of Tw to air temperature and river flow and is calibrated for three distinct time periods: the late 19th, mid-20th, and early 21st centuries. Results show that Tw has trended upwards at 1.1 ∘C per century since the mid-19th century, with the largest shift in January and February (1.3 ∘C per century) and the smallest in May and June (∼ 0.8 ∘C per century). The duration that the river exceeds the ecologically important threshold of 20 ∘C has increased by about 20 d since the 1800s, to about 60 d yr−1. Moreover, cold-water days below 2 ∘C have virtually disappeared, and the river no longer freezes. Since 1900, changes are primarily correlated with increases in air temperature (Tw increase of 0.81 ± 0.25 ∘C) but also occur due to alterations in the river system such as depth increases from reservoirs (0.34 ± 0.12 ∘C). Managed release of water affects Tw seasonally, with an average reduction of up to 0.56 ∘C estimated for September. River system changes have decreased variability (σ) in daily minimum Tw by 0.44 ∘C, increased thermal memory, reduced interannual variability, and reduced the response to short-term meteorological forcing (e.g., heat waves). These changes fundamentally alter the response of Tw to climate change, posing additional stressors on fauna.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

VIBES: a workflow for annotating and visualizing viral sequences integrated into bacterial genomes

Abstract Bacteriophages are viruses that infect bacteria. Many bacteriophages integrate their genomes into the bacterial chromosome and become prophages. Prophages may substantially burden or benefit host bacteria fitness, acting in some cases as parasites and in others as mutualists. Some prophages have been demonstrated to increase host virulence. The increasing ease of bacterial genome sequencing provides an opportunity to deeply explore prophage prevalence and insertion sites. Here we present VIBES (Viral Integrations in Bacterial genomES), a workflow intended to automate prophage annotation in complete bacterial genome sequences. VIBES provides additional context to prophage annotations by annotating bacterial genes and viral proteins in user-provided bacterial and viral genomes. The VIBES pipeline is implemented as a Nextflow-driven workflow, providing a simple, unified interface for execution on local, cluster and cloud computing environments. For each step of the pipeline, a container including all necessary software dependencies is provided. VIBES produces results in simple tab-separated format and generates intuitive and interactive visualizations for data exploration. Despite VIBES’s primary emphasis on prophage annotation, its generic alignment-based design allows it to be deployed as a general-purpose sequence similarity search manager. We demonstrate the utility of the VIBES prophage annotation workflow by searching for 178 Pf phage genomes across 1072 Pseudomonas spp. genomes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Life Cycle Analysis and Life Cycle Costing for Municipal Solid Waste (MSW) Management and Materials Redeployment to Support ARPA-E’s Potential Program on Waste to Energy and Materials

Municipal solid waste infrastructure in the United States contains complex supply chains to transport, sort, and sequester waste. This project developed models to estimate the environmental and economic costs of waste infrastructure in the U.S. using data from several municipalities. Life cycle costing (LCC) models were developed for waste management relevant supply chain steps, including waste collection, sorting, landfilling, waste-to-energy, ash processing, and recycling. A waste-to-materials and energy (WTM&E) life cycle analysis (LCA) and LCC dashboard was developed in collaboration with Argonne National Laboratory (ANL) and an LCC tool was developed for additional analysis.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

DYFLOW: A flexible framework for orchestrating scientific workflows on supercomputers

Modern scientific workflows are increasing in complexity with growth in computation power, incorporation of non-traditional computation methods, and advances in technologies enabling data streaming to support on-the-fly computation. These workflows have unpredictable runtime behaviors, and a fixed, predetermined resource assignment on supercomputers can be inefficient for overall performance and throughput. Inability to change resource assignments further limits the scientists to avail of science-driven opportunities or respond to failures.We introduce DYFLOW, a flexible framework that orchestrates scientific workflows on supercomputers based on user-designed policies. DYFLOW compartmentalizes orchestration stages into simplified constructs, and end-users can program and reuse them according to their workflow requirements through an easy-to-use interface. These constructs hide the intricacies involved in runtime management from end-users, for instance, procurement of information to understand the workflow state, assessment, and supervision of the runtime changes. DYFLOW is designed to work alongside existing workflow management systems and reuse the available (static) support for workflow management. We have integrated DYFLOW with an existing workflow management tool as a demonstration. With experiments performed on use cases from three types of scientific workflows and two different parallel architectures, we show that DYFLOW achieves the desired orchestration incurring a small cost to carry out the runtime changes.

Singhal, Swati↗