Search NASA⌕ Search

SEARCH · Search NASA

Results for “analysis workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Exploring the potential of using L-Band InSAR for the mapping of flooded vegetation in tropical wetlands

Wetlands play a critical role in global water and carbon cycles, yet monitoring their water extent remains difficult, particularly beneath dense vegetation. SAR-based techniques such as backscatter thresholding are limited by complex scattering mechanisms, while fully polarimetric SAR (PolSAR) data capable of detecting doublebounce scattering remain scarce. To address these challenges, this study evaluates the potential of Interferometric SAR (InSAR) for mapping water surfaces beneath vegetation, termed flooded vegetation, using the Atrato floodplain in Colombia as a case study. We develop an automated workflow combining InSAR fringe detection with local phase homogeneity analysis and random sampling of processing parameters to generate probabilistic flooded vegetation maps. Applied to ALOS PALSAR-1 L-band image pairs from 2007–2011, the workflow captures seasonal fluctuations in flooded extent ranging from 500 to 1,500 km2. Compared to other L-band SAR inundation products, the InSAR-based maps identify broader flooded areas, with ~70% agreement in pairwise comparisons. Around 84% of detections align with existing wetland inventories and seasonal changes correspond with regional hydrological indicators, including terrestrial water storage anomalies and water gauge measurements. PolSAR analysis shows that InSAR complements backscatter-based methods by detecting inundation in areas with weak double-bounce signals. These findings suggest that combining InSAR with backscatter-based methods can improve detection of flooded vegetation, which is especially relevant for the upcoming NISAR mission that will offer frequent global L-band observations.

Coastal inundation↗

Multi-scale Quantitative Risk Analysis of Seabed Minerals: Principles and Application to Seafloor Massive Sulfide Prospects

The potential for mining hydrothermal mineral deposits on the seafloor, such as seafloor massive sulfides, has become technically possible, and some companies (currently not many) are considering their exploration and development. Yet, no present methodology has been designed to quantify the ore potential and assess the risks relative to prospectivity at prospect and regional scales. Multi-scale exploration techniques, similar to those of the play analysis that are used in the oil and gas industry, can help to fulfill this task by identifying the characteristics of geologic environments indicative of ore-forming processes. Such characteristics can represent a combination of, e.g., heat source, pathway, trap and reservoir that all dictate how and where ore components are mobilized from source to deposition. In this study, the understanding of these key elements is developed as a mineral system, which serves as a guide for mapping the risk of the presence or absence of ore-forming processes within the region of interest (the permissive tract). The risk analysis is carried out using geoscience data, and it is paired with quantitative resource estimation analysis to estimate the in-place mineral potential. Resource estimates are simulated stochastically with the help of available data (bathymetric features in this study), conventional grade–tonnage models and Monte Carlo simulation techniques. In this paper, the workflow for a multi-scale quantitative risk analysis, from the definition to the evaluation of a permissive tract and related prospect(s), is described with the help of multi-beam data of a known hydrothermal vent site.

58 GEOSCIENCES↗

The PATRIC Bioinformatics Resource Center: expanding data and analysis capabilities

The PathoSystems Resource Integration Center (PATRIC) is the bacterial Bioinformatics Resource Center funded by the National Institute of Allergy and Infectious Diseases (https://www.patricbrc.org). PATRIC supports bioinformatic analyses of all bacteria with a special emphasis on pathogens, offering a rich comparative analysis environment that provides users with access to over 250 000 uniformly annotated and publicly available genomes with curated metadata. PATRIC offers web-based visualization and comparative analysis tools, a private workspace in which users can analyze their own data in the context of the public collections, services that streamline complex bioinformatic workflows and command-line tools for bulk data analysis. Over the past several years, as genomic and other omics-related experiments have become more cost-effective and widespread, we have observed considerable growth in the usage of and demand for easy-to-use, publicly available bioinformatic tools and services. Here we report the recent updates to the PATRIC resource, including new web-based comparative analysis tools, eight new services and the release of a command-line interface to access, query and analyze data.

59 BASIC BIOLOGICAL SCIENCES↗

Unraveling the impact of initial choices and in-loop interventions on learning dynamics in autonomous scanning probe microscopy

The current focus in Autonomous Experimentation (AE) is on developing robust workflows to conduct the AE effectively. This entails the need for well-defined approaches to guide the AE process, including strategies for hyperparameter tuning and high-level human interventions within the workflow loop. This paper presents a comprehensive analysis of the influence of initial experimental conditions and in-loop interventions on the learning dynamics of Deep Kernel Learning (DKL) within the realm of AE in scanning probe microscopy. We explore the concept of the “seed effect,” where the initial experiment setup has a substantial impact on the subsequent learning trajectory. Additionally, we introduce an approach of the seed point interventions in AE allowing the operator to influence the exploration process. Using a dataset from Piezoresponse Force Microscopy on PbTiO 3 thin films, we illustrate the impact of the “seed effect” and in-loop seed interventions on the effectiveness of DKL in predicting material properties. The study highlights the importance of initial choices and adaptive interventions in optimizing learning rates and enhancing the efficiency of automated material characterization. This work offers valuable insights into designing more robust and effective AE workflows in microscopy with potential applications across various characterization techniques.

47 OTHER INSTRUMENTATION↗

Nova Analysis (Final Technical Report)

As the adoption of solar plus storage technology is rapidly increasing, there is a need for a more wholistic view of homes adopting them. In homes, both energy efficiency (EE) upgrades and DERs provide value not just to the homeowner, but to utilities and even society at large through emissions. Traditionally, EE and DERs are separate sectors, which makes it difficult to understand the co-benefits of their adoption. To address this, we created a novel workflow of tools that allows for a complete analysis of both efficiency and DERs. We also looked at a suite of metrics designed to capture the different benefits provided to different stakeholders, demonstrated with multiple sets of field data. A final report demonstrates the potential of the simulation workflow by simulating hundreds of buildings spread across the U.S. and demonstrating how a variety of factors affect the optimal sizing of DERs along with several other key metrics including energy, utility bills, emissions, and average resilience hours.

14 SOLAR ENERGY↗

Enabling Re-executable Workflows with Near-realtime Visualization, Provenance Capture and Advanced Querying for Mass Spectrometry Data

Mass spectrometry imaging (MSI) enables simultaneous spatially-resolved analysis of numerous ionizable molecules on a sample surface. The current analysis pipeline involves analyzing the data coming off the instrument using a tool called MSI Quickview and saving the results onto a storage drive before moving on to the next experiment. While this works well for single datasets, there is a demand for more scalable, flexible workflows that are re-executable across datasets, support extensive querying and ease collaboration. Here, we present a workflow that moves data analysis from a mere desktop application for single experiments to a more general capability

Re-executable workflow↗

Optimizing mixed cool thermal storage systems across a connected community

A high level of electric demand flexibility must be integrated into our building infrastructure to enable greater renewable energy penetration in the grid. In the U.S., 9% of electricity generated is used to cool buildings in a periodic manner, making this end-use an ideal target for active management through cool thermal energy storage (CTES) technologies. Historic uses for CTES are designed around central chilled water plants, but these systems cool less than 25% of U.S. commercial floorspace. Emerging technologies are under development to serve the many smaller distributed cooling systems, such as rooftop units (RTUs), and have the potential to add CTES to an additional 66% of cooled commercial floorspace. However, these unitary thermal storage systems (UTSS) lack the modeling and analysis tools to evaluate them in the future interactive grid context. Here, this study develops the modeling and optimization tools necessary to simultaneously examine central and distributed ice storage systems within the multi-building, connected community context. An integrated simulation-optimization workflow is created to allow for rapid customized analysis. Results demonstrate the energy and flexibility tradeoffs of various implementations.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

MS Based Metabonomics

Metabonomics is the latest and least mature of the systems biology triad, which also includes genomics and proteomics, and has its origins in the early orthomolecular medicine work pioneered by Linus Pauling and Arthur Robinson. It was defined by Nicholson and colleagues in 1999 as the quantitative measurement of perturbations in the metabolite complement of an integrated biological system in response to internal or external stimuli, and is often used today to describe many non-global types of metabolite analyses. Applications of metabonomics are extensive and include toxicology, nutrition, pharmaceutical research and development, physiological monitoring and disease diagnosis. For example, blood samples from millions of neonates are tested routinely by mass spectrometry (MS) as a diagnostic tool for inborn errors of metabolism. The metabonome encompasses a wide range of structurally diverse metabolites; therefore, no single analytical platform will be sufficient. Specialized sample preparation and detection techniques are required, and advances in NMR and MS technologies have led to enhanced metabonome coverage, which in turn demands improved data analysis approaches. The role of MS in metabonomics is still evolving as instrumentation and software becomes more sophisticated and as researchers realize the strengths and limitations of current technology. MS offers a wide dynamic range, high sensitivity, and reproducible, quantitative analysis. These attributes are essential for addressing the challenges of metabonomics, as the range of metabolite concentrations easily exceeds nine orders of magnitude in biofluids, and the diversity of molecular species ranges from simple amino and organic acids to lipids and complex carbohydrates. Additional challenges arise in generating a comprehensive metabolite profile, downstream data processing and analysis, and structural characterization of important metabolites. A typical workflow of MS-based metabonomics is shown in Figure 1. Gas chromatography-(GC)-MS was the most commonly used MS-based method for small molecule analysis in the 1970s and 1980s. It is still used today for the detection of many metabolic disorders and plays a strong role in plant metabonomics. Liquid chromatography (LC)-MS approaches have grown in popularity for metabolite studies, due to simpler sample preparation, reduced analysis times through the introduction of ultra-high performance liquid chromatography (UPLC)-MS and the ability to observe a wider range of metabolites. This chapter will discuss the role of MS in metabonomics, the techniques involved in this exciting area, and the current and future applications of the field. The various bioinformatics tools and multivariate analysis techniques used to maximize information recovery and to aid in the interpretation of the very large data sets typically obtained in metabonomics studies will also be discussed. While there are many different MS-based approaches utilized in metabonomics studies, emphasis will be placed on more established methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Exploring the Optimal Platform Configuration for Power-Constrained HPC Workflows

In high-performance computing (HPC) workflows, data analytics is typically utilized to gain insights from scientific simulations. Approaching the era of exascale, online analysis is gaining popularity due to the savings of I/O to persistent storage. As computing capability keeps growing, power consumption is becoming critical to HPC facilities. Enforcing power limits is emerging as a practical trend for power-constrained HPC facilities. However, it remains unclear how to choose the appropriate power limits for various HPC workflows and how to distribute the power limit of a workflow between simulation and analysis. In addition, given a power limit, it is unclear what the optimal scales and power capping levels are for various workflows, especially when taking reliability into account. In order to resolve these issues in power-constrained HPC, in this paper, we propose a reliability-aware model to determine the aforementioned platform configurations for HPC workflows. We also validate our model and present model-driven studies for a wide range of real-system scenarios. Our study reveals interesting insights about how platform configuration affects the performance and energy efficiency of HPC workflows under power constraints.

Tang, Kun↗

Performance Analysis Tool for HPC and Big Data Applications on Scientific Clusters

Big data is prevalent in HPC computing. Many HPC projects rely on complex workflows to analyze terabytes or petabytes of data. These workflows often require running over thousands of CPU cores and performing simultaneous data accesses, data movements, and computation. It is challenging to analyze the performance involving terabytes or petabytes of workflow data or measurement data of the executions, from complex workflows over a large number of nodes and multiple parallel task executions. To help identify performance bottlenecks or debug the performance issues in large-scale scientific applications and scientific clusters, we have developed a performance analysis framework, using state-ofthe- art open-source big data processing tools. Our tool can ingest system logs and application performance measurements to extract key performance features, and apply the most sophisticated statistical tools and data mining methods on the performance data. It utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of the big data analysis framework, we conduct case studies on the workflows from an astronomy project known as the Palomar Transient Factory (PTF) and the job logs from the genome analysis scientific cluster. Our study processed many terabytes of system logs and application performance measurements collected on the HPC systems at NERSC. The implementation of our tool is generic enough to be used for analyzing the performance of other HPC systems and Big Data workows.

97 MATHEMATICS AND COMPUTING↗

Numerical Validation of an Algorithm for Combined Soiling and Degradation Analysis of Photovoltaic Systems

We describe and demonstrate an open-source algorithm for simultaneously quantifying degradation and soiling of photovoltaic (PV) systems from energy-production time series data. The new analysis is based on year-on-year degradation rate analysis combined with stochastic rate and recovery soiling analysis. The algorithm is designed to fit into the workflow provided by RdTools, a Python module maintained by NREL and collaboratively developed with the community, which provides a framework and functions for degradation and loss-factor analysis of PV field data. We demonstrate the method on numerically simulated PV data sets and show that it reduces the root-mean-square error of the P50 degradation rate estimate when soiling is present.

14 SOLAR ENERGY↗

Lambda-PFLOTRAN 1.0: a workflow for incorporating organic matter chemistry informed by ultra high resolution mass spectrometry into biogeochemical modeling

Abstract. Organic matter (OM) composition plays a central role in microbial respiration of dissolved organic matter and subsequent biogeochemical reactions. Here, a direct connection of organic matter chemistry and thermodynamics to reactive transport simulators has been achieved through the newly developed Lambda-PFLOTRAN workflow tool that succinctly incorporates carbon chemistry data generated from Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) into reaction networks to simulate organic matter degradation and the resulting biogeochemistry. Lambda-PFLOTRAN is a Python-based workflow, executed through a Jupyter notebook interface, that digests raw FTICR-MS data, develops a representative reaction network based on substrate-explicit thermodynamic modeling (also termed lambda modeling due to its key thermodynamic parameter λ used therein), and completes a biogeochemical simulation with the open source, reactive flow and transport code PFLOTRAN. The workflow consists of the following five steps: configuration, thermodynamic (lambda) analysis, sensitivity analysis, parameter estimation, and simulation output and visualization. Two test cases are provided to demonstrate the functionality of the Lambda-PFLOTRAN workflow. The first test case uses laboratory incubation data of temporal oxygen depletion to fit lambda parameters (i.e., maximum utilization rate and microbial carrying capacity). A slightly more complex second test case fits multiple lambda formulation and soil organic matter release parameters to temporal greenhouse gas generation measured during a soil incubation. Overall, the Lambda-PFLOTRAN workflow facilitates upscaling by using molecular-scale characterization to inform biogeochemical processes occurring at larger scales.

58 GEOSCIENCES↗

AToM: Advanced Tokamak Modeling Environment

Stability constraints play an important role in integrated modeling. The global plasma stability and macro-instabilities such as internal kink modes, neoclassical tearing modes, and edge localized perturbations limit the plasma performance and can result in large-scale transient events and plasma disruptions. These macro-instabilities can be also beneficial to the plasma performance. For example, the peeling models often lead to the edge localized modes (ELMs). However, the formation of stationary edge harmonic oscillations (EHOs) due to nonlinear interaction of peeling modes can result in a transition to the Quiescent H-mode (QH-mode) without ELMs. The profiles predicted with transport models need to be refined using the MHD stability constraints. This is important especially for transient and nonlinear stages of discharges such as ramp-up access to hybrid and steady state operation, and L- to H-mode transition as well as transition to the QH-mode. Several stability and MHD codes are already included in the OMFIT framework. These codes include BALOO, ELITE, GATO, M3DC1, MARS, and NIMROD. However, the verifications of stability constraints are currently mostly excluded from transport modeling workflows. The only exception is the EPED module which includes the ELITE predictions to limit the pedestal height. Here, we improved the OMFIT workflow to include stability calculations. Including the stability conditions to the integrated modeling workflow improved the robustness of the predictive modeling discharges. Being implemented in the workflow, the stability conditions can be also used for the stability analysis of experimental data. This improved the physics understanding of various discharge scenarios and can be used in experiment planning.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows

Here we report that computational science is well established as the third pillar of scientific discovery and is on par with experimentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computations alike, the complexity of managing the compute and data analysis tasks has grown beyond the capabilities of domain scientists. Therefore, workflow management systems are absolutely necessary to ensure current and future scientific discoveries. A key research question for these workflow management systems concerns the performance optimization of complex calculation and data analysis tasks. The central contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific workflows. This approach integrates extreme-scale systems testbed experimentation, structured analytical modeling, and parallel systems simulation into a comprehensive workflow framework called Pegasus for understanding and improving the overall performance of complex scientific workflows.

97 MATHEMATICS AND COMPUTING↗

SeekAView: An Intelligent Dimensionality Reduction Strategy for Navigating High-Dimensional Data Spaces

Dealing with the curse of dimensionality is a key challenge in high-dimensional data visualization. We present SeekAView to address three main gaps in the existing research literature. First, automated methods like dimensionality reduction or clustering suffer from a lack of transparency in letting analysts interact with their outputs in real-time to suit their exploration strategies. The results often suffer from a lack of interpretability, especially for domain experts not trained in statistics and machine learning. Second, exploratory visualization techniques like scatter plots or parallel coordinates suffer from a lack of visual scalability: it is difficult to present a coherent overview of interesting combinations of dimensions. Third, the existing techniques do not provide a flexible workflow that allows for multiple perspectives into the analysis process by automatically detecting and suggesting potentially interesting subspaces. In SeekAView we address these issues using suggestion based visual exploration of interesting patterns for building and refining multidimensional subspaces. Compared to the state-of-the-art in subspace search and visualization methods, we achieve higher transparency in showing not only the results of the algorithms, but also interesting dimensions calibrated against different metrics. We integrate a visually scalable design space with an iterative workflow guiding the analysts by choosing the starting points and letting them slice and dice through the data to find interesting subspaces and detect correlations, clusters, and outliers. We present two usage scenarios for demonstrating how SeekAView can be applied in real-world data analysis scenarios.

Krause, Josua↗

Next Generation Workload Management System For Big Data on Heterogeneous Distributed Computing

The Large Hadron Collider (LHC), operating at the international CERN Laboratory in Geneva, Switzerland, is leading Big Data driven scientific explorations. Experiments at the LHC explore the fundamental nature of matter and the basic forces that shape our universe, and were recently credited for the discovery of a Higgs boson. ATLAS and ALICE are the largest collaborations ever assembled in the sciences and are at the forefront of research at the LHC. To address an unprecedented multi-petabyte data processing challenge, both experiments rely on a heterogeneous distributed computational infrastructure. The ATLAS experiment uses PanDA (Production and Data Analysis) Workload Management System (WMS) for managing the workflow for all data processing on hundreds of data centers. Through PanDA, ATLAS physicists see a single computing facility that enables rapid scientific breakthroughs for the experiment, even though the data centers are physically scattered all over the world. The scale is demonstrated by the following numbers: PanDA manages O(10 2 ) sites, O(10 5 ) cores, O(10 8 ) jobs per year, O(10 3 ) users, and ATLAS data volume is O(10 17 ) bytes. In 2013 we started an ambitious program to expand PanDA to all available computing resources, including opportunistic use of commercial and academic clouds and Leadership Computing Facilities (LCF). The project titled 'Next Generation Workload Management and Analysis System for Big Data' (BigPanDA) is funded by DOE ASCR and HEP. Extending PanDA to clouds and LCF presents new challenges in managing heterogeneity and supporting workflow. The BigPanDA project is underway to setup and tailor PanDA at the Oak Ridge Leadership Computing Facility (OLCF) and at the National Research Center "Kurchatov Institute" together with ALICE distributed computing and ORNL computing professionals. Our approach to integration of HPC platforms at the OLCF and elsewhere is to reuse, as much as possible, existing components of the PanDA system. Finally, we will present our current accomplishments with running the PanDA WMS at OLCF and other supercomputers and demonstrate our ability to use PanDA as a portal independent of the computing facilities infrastructure for High Energy and Nuclear Physics as well as other data-intensive science applications.

97 MATHEMATICS AND COMPUTING↗

Enabling immersive engagement in energy system models with deep learning

Complex ensembles of energy simulation models have become crucial components of renewable energy research in recent years. Often the increasing computational cost, high-dimensional structure, and other complexities hinder researchers from fully utilizing these data sources for knowledge building. Researchers at National Renewable Energy Laboratory have determined an immersive visualization workflow to dramatically improve user engagement and analysis capability through a combination of low-dimensional structure analysis, deep learning, and custom visualization methods. We present case studies for two energy simulation platforms.

97 MATHEMATICS AND COMPUTING↗

iTOUGH2: A multiphysics simulation-optimization framework for analyzing subsurface systems

We report iTOUGH2 is a simulation-optimization framework for the TOUGH suite of nonisothermal multiphase flow models and related simulators of geophysical, geochemical, and geomechanical processes. After appropriate parameterization of subsurface structures and their properties, iTOUGH2 runs simulations for multiple parameter sets and analyzes the resulting output for parameter estimation through automatic model calibration, local and global sensitivity analyses, data-worth analyses, and uncertainty propagation analyses. Development of iTOUGH2 is driven by scientific challenges and user needs, with new capabilities continually added to both the forward simulator and the optimization framework. This paper provides a summary description of methods and features implemented in iTOUGH2, and discusses the usefulness and limitations of an integrated simulation-optimization workflow in support of the characterization and analysis of complex multiphysics subsurface systems.

58 GEOSCIENCES↗