SEARCH · Search NASA

Results for “computational workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,765 records · Page 98

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Rapid Simulation of Unconventional Reservoirs by Multidomain Multiresolution Modeling Based on the Diffusive Time of Flight

Summary The fast marching method (FMM)-based rapid flow simulation has been shown to accelerate simulation efficiency by orders of magnitude by transforming 3D simulation to equivalent 1D simulation using the concept of the “diffusive time of flight” (DTOF). However, the 1D transformation does not directly apply to multiwell problems. In this paper, we propose a novel DTOF-based multidomain multiresolution discretization scheme to accelerate multiwell simulation of unconventional reservoirs. Our method formulates multiwell simulation problems based on the DTOF which displays the pressure front propagation in unconventional reservoirs. The DTOF contours are used to partition the reservoir into local and shared domains. A local domain is where the flow is dominated by a single well, and the shared domain is where the fluid flow is influenced by multiple wells. The DTOF contours expand independently in local domains and interfere in the shared domain. After the partitioning, each domain is discretized using a multiresolution scheme whereby the original 3D fine mesh is preserved near the wells to account for detailed physics including gravity, and the rest of the domain is discretized into 1D mesh based on the DTOF contours to alleviate the simulation workload. The power and efficacy of our approach are demonstrated using synthetic and field-scale simulation models with different degrees of geologic and well-completion complexity. The simulation results, number of active cells, and computation time for the proposed discretization scheme are compared with the original high-fidelity 3D model for each case. The results show that the proposed method is suitable for multiwell simulation problems in unconventional reservoirs and can accelerate flow simulations by orders of magnitude with minimal loss of accuracy. The novelty of this work is the creation of DTOF-derived multiresolution discretization with local and shared domains to simplify and accelerate the calculation of subsurface flow problems, especially in unconventional reservoirs. Our workflow can be easily interfaced with commercial simulators, making it suitable for large-scale field applications.

Engineering↗

DOE OSTI2024

Complete Demonstration of a Prototype Version of FORCE User Interface and Conduct Analyst Survey Collecting Feedback on Interface Features and Usability

In 2024 the US Department of Energy (DOE) Office of Nuclear Energy (NE) Integrated Energy System (IES) program continued to develop the Framework for Optimization of Resources and Economics (FORCE) analysis ecosystem into a more traditional toolset with simplified software installation, automated workflows, and interactive results visualization. The DOE-NE Nuclear Energy Advanced Modeling and Simulation (NEAMS) Workbench continued to be leveraged for user input, application workflow and runtime environment, and interactive results visualization capabilities. This report documents the demonstration of a FORCE User Interface (UI) prototype and the results of a survey of analysts’ using the Holistic Energy Resource Optimization Network (HERON) tool in FORCE with the prototype UI.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Numerical Modeling and Optimization of the iProTech Pitching Inertial Pump (PIP) Wave Energy Converter (WEC) (Cooperative Research and Development Final Report, CRADA Number: CRD-22-22968)

This work generated a first-of-its-kind automated workflow to couple time-domain simulations of wave energy converters written in one software language with a set of design generation and evaluation scripts written in another software language. This automated workflow used an existing optimization package to analyze the sensitivity of different design parameters on the power output of a specific WEC, iProTech’s Pitching Inertial Pump (PIP). Geometric, inertial, and power take-off variables were all varied and optimized to find values that produced the highest amount of power generated over varying wave conditions. The findings on these parameter sensitivity studies are used to inform future design iterations of the PIP WEC. Including more design variables in the optimizations will only increase computational run time and further software development is needed to analyze a larger optimization.

16 TIDAL AND WAVE POWER↗

DOE OSTI2025

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speed up job completion, better utilize the available hardware, and visibly reduce costs, especially when renting computers in the cloud. This was demonstrated in earlier studies on HEPCloud. However, the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and it shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. GlideinBenchmark, with the help of the GlideinWMS Factory, controls the benchmark execution. Finally, a scheduler like HEPCloud's Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681↗

NASA NTRS2012

Preliminary Evaluation of MapReduce for High-Performance Climate Data Analysis

MapReduce is an approach to high-performance analytics that may be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. We are particularly interested in the potential of MapReduce to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we are prototyping a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. Our initial focus has been on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. Preliminary results suggest this approach can improve efficiencies within data intensive analytic workflows.

Duffy, Daniel Q.↗

NASA NTRS2013

Hadoop for High-Performance Climate Analytics: Use Cases and Lessons Learned

Scientific data services are a critical aspect of the NASA Center for Climate Simulations mission (NCCS). Hadoop, via MapReduce, provides an approach to high-performance analytics that is proving to be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. The NCCS is particularly interested in the potential of Hadoop to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we prototyped a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. The initial focus was on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. After preliminary results suggested that this approach improves efficiencies within data intensive analytic workflows, we invested in building a cyber infrastructure resource for developing a new generation of climate data analysis capabilities using Hadoop. This resource is focused on reducing the time spent in the preparation of reanalysis data used in data-model inter-comparison, a long sought goal of the climate community. This paper summarizes the related use cases and lessons learned.

analytics↗

NASA NTRSDate not supplied

Expanding Repository Data Available For Sharing and Knowledge Discovery

Some of the hardest space biology and space health challenges require data-intensive, bioinformatic, meta-analytical, and computer-assisted research approaches. These challenges include examining interdisciplinary space life science research across experiments and across interacting spaceflight hazards (radiation, altered gravity, confinement, hostile-closed environments, distance-duration from Earth). The approaches to confront these challenges involve mining multiple datasets simultaneously from various hierarchical organizations of biological complexity, all while concurrently evaluating how experimental design factors affect endpoints of standard assays. To enable this field, it is essential that principal investigators (PIs) submit data in a structure so it can be maximally re-used. The purpose of the NASA Ames Life Sciences Data Archive (ALSDA) is to collect, curate, and make publicly available all non-human space-relevant biological data. ALSDA must also ensure data are open-access, and maximally findable, accessible, interoperable, and reusable (FAIR). The scope of ALSDA data collected and submitted by PIs include subject and study design metadata, assay metadata parameters, raw and processed assay data, assay imagery/video, and subject-experienced mission data telemetry (radiation, temperature, humidity, acoustics, vibrations, etc.). ALSDA recently integrated into a collaborative group of Open Science projects to facilitate a suite of new tools and workflows that will improve data submission, accessibility, and reusability by implementing digital data submission agreements, and adopting the data management system originally developed by NASA GeneLab. ALSDA intends to bring current biological repository data and all future collected data into this new scientific data reuse reality. This new suite of tools will enable ALSDA to deploy a science curation system using scientific assay configurations for the data submission portal. It will capture essential assay parameters according to established standards in each sub-field within biology. The submission portal expedites data collection by enhancing ease of PI data submission, providing a user interface and specificity for which data is to be submitted. Data submissions can be brought into cutting-edge informatic analysis portals to enable mining of physiological, behavioral, biochemical, and imaging datasets in conjunction with ‘omics-level datasets. As ALSDA datasets are submitted, curated, and published (e.g., micro-computed tomography, histology, pulse oximetry, serum metabolites, magnetic resonance imaging, intraocular pressure, novel object recognition, etc.), the merging together of spaceflight data along this multi-hierarchical complexity of biology will enable informatics and data-intensive approaches resulting in knowledge discoveries across missions, space hazards, and biological disciplines.

Biology↗

NASA NTRSDate not supplied

Expanding Repository Data Available For Sharing And Knowledge Discovery

life science↗

DOE OSTI2021

Open Data and Deep Semantic Segmentation for Automated Extraction of Building Footprints

Advances in machine learning and computer vision, combined with increased access to unstructured data (e.g., images and text), have created an opportunity for automated extraction of building characteristics, cost-effectively, and at scale. These characteristics are relevant to a variety of urban and energy applications, yet are time consuming and costly to acquire with today’s manual methods. Several recent research studies have shown that in comparison to more traditional methods that are based on features engineering approach, an end-to-end learning approach based on deep learning algorithms significantly improved the accuracy of automatic building footprint extraction from remote sensing images. However, these studies used limited benchmark datasets that have been carefully curated and labeled. How the accuracy of these deep learning-based approach holds when using less curated training data has not received enough attention. The aim of this work is to leverage the openly available data to automatically generate a larger training dataset with more variability in term of regions and type of cities, which can be used to build more accurate deep learning models. In contrast to most benchmark datasets, the gathered data have not been manually curated. Thus, the training dataset is not perfectly clean in terms of remote sensing images exactly matching the ground truth building’s foot-print. A workflow that includes data pre-processing, deep learning semantic segmentation modeling, and results post-processing is introduced and applied to a dataset that include remote sensing images from 15 cities and five counties from various region of the USA, which include 8,607,677 buildings. The accuracy of the proposed approach was measured on an out of sample testing dataset corresponding to 364,000 buildings from three USA cities. The results favorably compared to those obtained from Microsoft’s recently released US building footprint dataset.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Missing data in multi-omics integration: Recent advances through artificial intelligence

Biological systems function through complex interactions between various ‘omics (biomolecules), and a more complete understanding of these systems is only possible through an integrated, multi-omic perspective. This has presented the need for the development of integration approaches that are able to capture the complex, often non-linear, interactions that define these biological systems and are adapted to the challenges of combining the heterogenous data across ‘omic views. A principal challenge to multi-omic integration is missing data because all biomolecules are not measured in all samples. Due to either cost, instrument sensitivity, or other experimental factors, data for a biological sample may be missing for one or more ‘omic techologies. Recent methodological developments in artificial intelligence and statistical learning have greatly facilitated the analyses of multi-omics data, however many of these techniques assume access to completely observed data. A subset of these methods incorporate mechanisms for handling partially observed samples, and these methods are the focus of this review. We describe recently developed approaches, noting their primary use cases and highlighting each method's approach to handling missing data. We additionally provide an overview of the more traditional missing data workflows and their limitations; and we discuss potential avenues for further developments as well as how the missing data issue and its current solutions may generalize beyond the multi-omics context.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗

DOE OSTI2026

Computer vision-based rock bolt detection in orthomosaic imagery obtained in the Waste Isolation Pilot Plant underground facility

Assessing structural integrity of large underground tunnel facilities is often a time consuming and human-labor intensive task. Thus, research using various modes of sensing and automated detection of key structural components in mines is posed to aid in safety assessments and establishing overall structural health. We propose an approach utilizing off-the-shelf camera and lidar technology fixed to a custom sensing platform, image stitching techniques, fine-tuned object detection models, and specialized model-inference methods to automatically detect, count, and map roof bolts for assessment of structural safety in man-made underground tunnels. Results show a novel workflow for effective object counting in orthomosaic tunnel ceiling images generated from collections in GPS-denied mining environments. Additionally, we demonstrate effective fine-tuning of EfficientDet object detectors utilizing state-of-the-art image augmentation techniques known as the mosaic and mixup transformations. Our work is demonstrated on sensed data and imagery collected from the Department of Energy (DOE) Waste Isolation Pilot Plant (WIPP) where miles of tunnel ceiling must be assessed for structural integrity.

42 ENGINEERING↗

DOE OSTI2022

Drugsniffer: An Open Source Workflow for Virtually Screening Billions of Molecules for Binding Affinity to Protein Targets

The SARS-CoV2 pandemic has highlighted the importance of efficient and effective methods for identification of therapeutic drugs, and in particular has laid bare the need for methods that allow exploration of the full diversity of synthesizable small molecules. While classical high-throughput screening methods may consider up to millions of molecules, virtual screening methods hold the promise of enabling appraisal of billions of candidate molecules, thus expanding the search space while concurrently reducing costs and speeding discovery. Here, we describe a new screening pipeline, called drugsniffer, that is capable of rapidly exploring drug candidates from a library of billions of molecules, and is designed to support distributed computation on cluster and cloud resources. As an example of performance, our pipeline required ~40,000 total compute hours to screen for potential drugs targeting three SARS-CoV2 proteins among a library of ~3.7 billion candidate molecules.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2019

Machine learning coarse grained models for water

An accurate and computationally efficient molecular level description of mesoscopic behavior of ice-water systems remains a major challenge. Here, we introduce a set of machine-learned coarse-grained (CG) models (ML-BOP, ML-BOP dih , and ML-mW) that accurately describe the structure and thermodynamic anomalies of both water and ice at mesoscopic scales, all at two orders of magnitude cheaper computational cost than existing atomistic models. In a significant departure from conventional force-field fitting, we use a multilevel evolutionary strategy that trains CG models against not just energetics from first-principles and experiments but also temperature-dependent properties inferred from on-the-fly molecular dynamics (~ 10’s of milliseconds of overall trajectories). Our ML BOP models predict both the correct experimental melting point of ice and the temperature of maximum density of liquid water that remained elusive to-date. Our ML workflow navigates efficiently through the high-dimensional parameter space to even improve upon existing high-quality CG models (e.g. mW model).

42 ENGINEERING↗

DOE OSTI2018

Probability of Encounter Between Ports of Entry (PrEP): Aversion Force-Field “Early Validation” Report

To enhance detection and interdiction of radiological or nuclear material trafficking between primary POEs into the US, The Department of Homeland Security is faced with highly variable smuggling pathways through expansive geographic regions. The goal of this project is to test an approach to estimating likely pathways, as well as the probability of encounter along any given route. Our goal is to test the feasibility of a novel approach based on assessing the adversary’s perception of discovery likelihood over a broad geographic region, and using that result to estimate an “aversion force field” over the same region. The aversion force field is derived from analogies based on the physics of electric charges. Combining the aversion force with the intention of an adversary to reach their destination, generates a route that is consistent with both the adversary’s intention and their aversion to deterrent measures. Phase I is exploratory. Its goal to demonstrate the feasibility of the approach. This report describes progress in an essential part of the approach, specifically, to implement and couple two principle model components: (1) the Bayesian network model of an adversary’s perception of discovery likelihood; and (2) the physics-inspired aversion field model. The objective is to show that the field produced from the two components is plausible and consistent with the assumptions and the data. Figure 1 illustrates the workflow of this phase to the present stage. After collecting preliminary data and implementing each of the two model components, they were separately evaluated. Following this step, the output from the Bayesian network was input to the aversion field solver, and the combined results evaluated as reported here.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Guide to Using Sierra

Sierra is an engineering mechanics simulation code suite supporting the Nation's Nuclear Weapons mission as well as other customers. It has explicit ties to Sandia National Labs' workflow, including geometry and meshing, design and optimization, and visualization. Distinguishing strengths include "application aware" development, scalability, SQA and V&V, multiple scales, and multi-physics coupling. This document is intended to help new and existing users of Sierra as a user manual and troubleshooting guide.

42 ENGINEERING↗

DOE OSTI2024

Analyzing Risks of Virtual Private Network Connections

The use of Splunk for analyzing VPN logs is an effective approach for identifying vulnerabilities in network endpoints. Splunk, a powerful platform for searching, monitoring, and analyzing machine-generated data, enables organizations to aggregate VPN logs in real-time, providing insights into network activity, user behavior, and potential security risks. By indexing VPN traffic and authentication logs, security teams can track abnormal patterns such as multiple failed login attempts, unusual IP addresses, or unexpected changes in bandwidth usage, all of which could indicate potential vulnerabilities or breaches. With Splunk’s advanced search and reporting capabilities, users can create custom dashboards and alerts to detect suspicious activities. Automated searches can flag endpoints exhibiting unusual behavior, while correlation analysis can identify links between compromised devices and broader network vulnerabilities. In particular, Splunk's machine learning capabilities can be leveraged to predict and prevent threats by identifying trends that might otherwise be missed in traditional log analysis. This proactive approach to monitoring VPN logs allows for the early detection of security weaknesses, enabling rapid response and minimizing potential damage to network integrity. By enhancing endpoint visibility, Splunk plays a crucial role in securing remote connections and safeguarding sensitive information. Additionally, Splunk’s automation and alerting features allow teams to create custom workflows that notify them of vulnerable or misconfigured endpoints identified through Shodan. This synergy between Splunk’s log analysis and Shodan’s device intelligence enhances an organization’s ability to proactively identify and mitigate security risks, improving the overall resilience of their VPN infrastructure.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Closing the Loop between In Situ Stress Complexity and EGS Fracture Complexity

We present an agent-guided approach to CAD geometry decomposition that automates hex/hybrid meshing with graph neural networks (GNNs) to accelerate next-generation ModSim workflows. Our end-to-end pipeline (i) reduces 3D boundary-representation (B-Rep) models to a 2D chordal axis skeleton (CAT) and then to a 1D bipartite graph of surface and curve nodes, (ii) assigns per node labels as Cubit® WebCut actions, (iii) trains a multi-action GNN under supervised learning, and (iv) predicts five surface-node and three curve-node actions on out-of-distribution test geometries. Each graph node carries geometric, topological, and meshing attributes drawn from the B-Rep “skin” and CAT “skeleton,” with two-way mappings across 3D↔2D↔1D representations to maintain traceability back to 3D CAD. The supervised learning model exhibits stable convergence of the binary cross-entropy loss and achieves 98.7% accuracy on unseen lattice models. To operationalize decision-making, we rank predicted commands by geometric significance and prototyped the agent-guided workflow through the Cubit® Meshing PowerTool GUI. As a stretch goal, we explore reinforcement learning (RL) to reduce or remove label requirements and to learn policies for action sequences that maximize total reward (e.g., size of hex-meshable regions and resulting hex mesh quality). When all-hex meshing is not feasible, the agent assists in producing hybrid meshes—prioritizing hex in critical regions and transitioning to tetrahedral elements (tets) elsewhere—maintaining fidelity while ensuring robustness. The overarching objective is to replace manual, heuristics-based decomposition with data-driven, reproducible automation, cutting meshing turnaround time by orders of magnitude. We anticipate direct impact on simulation workflows through intelligent, scalable decomposition of complex CAD models into hex-meshable subdomains.

42 ENGINEERING↗