Search NASA⌕ Search

SEARCH · Search NASA

Results for “analysis workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Active workflow system for near real-time extreme-scale science. In: PPAA '14 Proceedings of the first workshop on Parallel programming for analytics applications

In recent years, streaming-based data processing has been gaining substantial traction for dealing with overwhelming data generated by real-time applications, from both enterprise sources and scientific computing. In this work, however, we look at an emerging class of scientific data with Near Real-Time (NRT) requirement, in which data is typically generated in a bursty fashion with the near real-time constraints being applied primarily between bursts, rather than within a stream. A key challenge for this types of data sources is that the processing time per data element is not uniform, and not always feasible to predict. Given the observations on the increasing unpredictability of compute load and system dynamics, this work looks to adapt streaming-based approach to the context of this new class of large experiments and simulations that have complex run-time control and analysis issues. In particular, we deploy a novel two-tier scheme for handling the increasing unpredictability of runtime behaviors: Instead of relying on determining what and where to run the scientific workflows beforehand or partial dynamically, the decision will also be adaptively enhanced online according to system runtime status. This is enabled by embedding workflow along with data streams. Specifically, we break data outputs generated from experiments or simulations into multiple self-describing "chunks", which we call active data objects. As such, if there is a transient hotspot observed, a data object with unfinished workflow pipeline can break its previous schedule and search for a least loaded location to continue the execution. Our preliminary experiment results based on synthetic workloads demonstrate the proposed active workflow system as a very promising solution by outperforming the state-of-the-art semi-dynamic workflow schedulers with an improved workflow completion time, as well as a good scalability.

Zhang, Yanwei↗

Scaling Automatic Vector Data Alignment to Satellite Imagery

Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.

Potnis, Abhishek↗

Decoding the Mechanisms of Phase Transitions from In Situ Microscopy Observations

Abstract Analysis of the temperature‐ and stimulus‐dependent imaging data toward elucidation of the physical transformations is an ubiquitous problem in multiple fields. Here, temperature‐induced phase transition in BaTiO 3 is explored using the machine learning analysis of domain morphologies visualized via variable‐temperature scanning transmission electron microscopy (STEM) imaging data. This approach is based on the multivariate statistical analysis of the time or temperature dependence of the statistical descriptors of the system, derived in turn from the categorical classification of observed domain structures or projection on the continuous parameter space of the feature extraction‐dimensionality reduction transform. The proposed workflow offers a powerful tool for the exploration of the dynamic data based on the statistics of image representation as a function of the external control variable to visualize the transformation pathways during phase transitions and chemical reactions. This can include the mesoscopic STEM data as demonstrated here, but also optical, chemical imaging, etc., data. It can further be extended to the higher dimensional spaces, for example, analysis of the combinatorial libraries of materials compositions.

Valleti, Sai Mani Prudhvi↗

Modular automated bottom-up proteomic sample preparation for high-throughput applications

Manual proteomic sample preparation methods limit sample throughput and often lead to poor data quality when thousands of samples must be analyzed. Automated liquid handler systems are increasingly used to overcome these issues for many of the sample preparation steps. Here, we detail a step-by-step protocol to prepare samples for bottom-up proteomic analysis for Gram-negative bacterial and fungal cells. The full modular protocol consists of three optimized protocols to: (A) lyse Gram-negative bacteria and fungal cells; (B) quantify the amount of protein extracted; and (C) normalize the amount of protein and set up tryptic digestion. These protocols have been developed to facilitate rapid, low variance sample preparation of hundreds of samples, be easily implemented on widely-available Beckman-Coulter Biomek automated liquid handlers, and allow flexibility for future protocol development. By using this workflow 50 micrograms of protein from 96 samples can be prepared for tryptic digestion in under an hour. We validate these protocols by analyzing 47 Pseudomonas putida and Rhodosporidium toruloides samples and show that this modular workflow provides robust, reproducible proteomic samples for high-throughput applications. The expected results from these protocols are 94 peptide samples from Gram-negative bacterial and fungal cells prepared for bottom-up quantitative proteomic analysis without the need for desalting column cleanup and with protein relative quantity variance (CV%) below 15%.

59 BASIC BIOLOGICAL SCIENCES↗

Preserving Command Line Workflow for a Package Management System using ASCII DAG Visualization

Package managers provide ease of access to applications by removing the time-consuming and sometimes completely prohibitive barrier of successfully building, installing, and maintaining the software for a system. A package dependency contains dependencies between all packages required to build and run the target software. Package management system developers, package maintainers, and users may consult the dependency graph when a simple listing is insufficient for their analyses. However, users working in a remote command line environment must disrupt their workflow to visualize dependency graphs in graphical programs, possibly needing to move files between devices or incur forwarding lag. Such is the case for users of Spack, an open source package management system originally developed to ease the complex builds required by supercomputing environments. To preserve the command line workflow of Spack, we develop an interactive ASCII visualization for its dependency graphs. Through interviews with Spack maintainers, we identify user goals and corresponding visual tasks for dependency graphs. We evaluate the use of our visualization through a command line-centered study, comparing it to the system's two existing approaches. We observe that despite the limitations of the ASCII representation, our visualization is preferred by participants when approached from a command line interface workflow.

97 MATHEMATICS AND COMPUTING↗

Scientific Data Management Center for Enabling Technologies

Managing scientific data has been identified by the scientific community as one of the most important emerging needs because of the sheer volume and increasing complexity of data being collected. Effectively generating, managing, and analyzing this information requires a comprehensive, end-to-end approach to data management that encompasses all of the stages from the initial data acquisition to the final analysis of the data. Fortunately, the data management problems encountered by most scientific domains are common enough to be addressed through shared technology solutions. Based on community input, we have identified three significant requirements. First, more efficient access to storage systems is needed. In particular, parallel file system and I/O system improvements are needed to write and read large volumes of data without slowing a simulation, analysis, or visualization engine.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Optimizing Cryo-Focused Pyrolysis GC/MS for Tracing Soil Organic Matter Across Diverse Ecosystems

The cycling of organic matter in terrestrial soils and sediments is central to a range of biogeochemical processes that regulate nutrient cycling, crop productivity, trace gas emissions, and contaminant transport. Pyrolysis-gas chromatography/mass spectrometry (py-GC/MS) is a powerful tool for characterizing bulk soil organic matter (SOM) at the molecular level. In this study, we used a cryo-focused py-GC/MS system to analyze soil samples from seven diverse ecosystems: vernal pool, prairie pothole, temperate forest, tropical forest, tundra, wildfire-affected boreal forest, and grassland. We addressed a key bottleneck in molecular-level SOM characterization by developing an automated data analysis pipeline to optimize py-GC/MS and complementary evolved gas analysis/mass spectrometry (EGA/MS) methods, incorporating advanced tools for peak deconvolution, developing a custom compound class library, and implementing fragmentation spectrum-based molecular networking for the first time. This improved workflow was applied to soil samples from all seven ecosystems, including multiple depths and density fractions. Our findings demonstrate that ecosystem type plays a dominant role in shaping compositional differences in SOM. We also identified trends in the source of SOM compounds (e.g., microbial vs plantderived) across soil depth and density fractions, which are critical for understanding persistence and turnover of SOM. Our molecular networking analysis indicated that although many compounds are widespread across ecosystems, others are restricted to specific environments, such as wetlands. This underscores the utility of molecular-level data in elucidating the complexity of SOM composition and the environmental drivers that shape it. Such molecular-level insights can deepen our knowledge of biogeochemical SOM cycles.

54 ENVIRONMENTAL SCIENCES↗

SU-G-TeP4-02: A Method for Evaluating the Direct Impact of Failed IMRT QAs On Patient Dose

Purpose: We developed a method to calculate patient doses corresponding to IMRT QA measurements in order to determine and assess the actual dose delivered for plans with failed (or borderline) IMRT QA. This work demonstrates the feasibility of automatically computing delivered patient dose from portal dosimetry measurements in the Varian TPS system, which would provide a valuable and clinically viable IMRT QA tool for physicists and physicians. Methods: IMRT QA fluences were measured using portal dosimetry, processed using in-house matlab software, and imported back into Eclipse to calculate dose on the planning CT. To validate the proposed workflow, the Eclipse calculated portal dose for a 5-field sliding window prostate boost plan was processed as described above. The resulting dose was compared to the planned dose and found to be within 0.5 Gy. Two IMRT QA results for the prostate boost plan (one that failed and one that passed) were processed and the resulting patient doses were evaluated. Results: The max dose difference between IMRT QA #1 and the original planned and approved dose is 4.5 Gy, while the difference between the planned and IMRT QA #2 dose is 4.0 Gy. The inferior portion of the PTV is slightly underdosed in both plans, and the superior portion is slightly overdosed. The patient dose resulting from IMRT QA #1 and #2 differs by only 0.5 Gy. With this new information, it may be argued that the evaluated plan alteration to obtain passing gamma analysis produced clinically irrelevant differences. Conclusion: Evaluation of the delivered QA dose on the planning CT provides valuable information about the clinical relevance of failed or borderline IMRT QAs. This particular workflow demonstrates the feasibility of pushing the measured IMRT QA portal dosimetry results directly back onto the patient planning CT within the Varian system.

60 APPLIED LIFE SCIENCES↗

eDNAjoint: An R package for interpreting paired or semi‐paired environmental DNA and traditional survey data in a Bayesian framework

Abstract Environmental DNA (eDNA) sampling is increasingly used in surveys of species distribution as a potentially sensitive and efficient monitoring method. Yet access to modelling tools designed specifically for interpreting this new data type lags behind its ubiquity. While occupancy modelling software has dominated the analytical landscape for eDNA data analysis of single species, this type of model may not always be the most appropriate. The rate of eDNA detection often corresponds to species density, rather than just occupancy, and researchers often have access to observations from non‐genetic sampling methods at the same sites. To provide users access to a modelling framework designed to maximize the use of all available data, we developed an R package, eDNAjoint . The package provides an easy‐to‐use interface for fitting a ‘joint’ model that integrates data from paired or semi‐paired eDNA and traditional surveys in a Bayesian framework. The model can be used to estimate parameters like the probability of a false positive eDNA detection and mean catch rate at a site, and the package allows access to multiple model variations and Bayesian prior customization. Additional functionality can be used for model selection, summarising posteriors and comparing the relative sensitivities of the two survey methods. We demonstrate the use of eDNAjoint by fitting a variation of the model with site‐level covariates that scale the sensitivity of eDNA sampling relative to traditional sampling. The example workflow uses binary eDNA and seine count data for the endangered tidewater goby ( Eucyclogobius newberryi ) from a study by Schmelzle and Kinziger (2016). This use case includes a prior sensitivity analysis and an evaluation of the relationship between detection rates and environmental variables. eDNAjoint has the potential to greatly increase the range of users who will be able to rigorously analyse eDNA and traditional survey data in a Bayesian framework, understand if and how eDNA can improve monitoring practices, and gain confidence in the interpretability of eDNA data.

Keller, Abigail G. [Department of Environment Scie↗

Self-Driving Laboratories for Chemistry and Materials Science

Self-driving laboratories (SDLs) promise an accelerated application of the scientific method. Through the automation of experimental workflows, along with autonomous experimental planning, SDLs hold the potential to greatly accelerate research in chemistry and materials discovery. This review provides an in-depth analysis of the state-of-the-art in SDL technology, its applications across various scientific disciplines, and the potential implications for research and industry. This review additionally provides an overview of the enabling technologies for SDLs, including their hardware, software, and integration with laboratory infrastructure. Most importantly, this review explores the diverse range of scientific domains where SDLs have made significant contributions, from drug discovery and materials science to genomics and chemistry. We provide a comprehensive review of existing real-world examples of SDLs, their different levels of automation, and the challenges and limitations associated with each domain.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

ROOT’s RNTuple I/O Subsystem: The Path to Production

The RNTuple I/O subsystem is ROOT’s future event data file format and access API. It is driven by the expected data volume increase at upcoming HEP experiments, e.g. at the HL-LHC, and recent opportunities in the storage hardware and software landscape such as NVMe drives and distributed object stores. RNTuple is a redesign of the TTree binary format and API and has shown to deliver substantially faster data throughput and better data compression both compared to TTree and to industry standard formats. In order to let HENP computing workflows benefit from RNTuple’s superior performance, however, the I/O stack needs to connect efficiently to the rest of the ecosystem, from grid storage to (distributed) analysis frameworks to (multithreaded) experiment frameworks for reconstruction and ntuple derivation. With the RNTuple binary format soon arriving at its first production release, we present RNTuple’s feature set, integration efforts, and its performance impact on the time-to-solution. We show the latest performance figures of RDataFrame analysis code of realistic complexity, comparing RNTuple and TTree as data sources. We discuss RNTuple’s approach to functionality critical to the HENP I/O (such as multithreaded writes, fast data merging, schema evolution) and we provide an outlook on the road to its use in production.

Blomer, Jakob↗

SU-E-T-261: Development of An Automated System to Detect Patient Identification and Positioning Errors Prior to Radiotherapy Treatment

Purpose: To develop a system that can automatically detect patient identification and positioning errors using 3D computed tomography (CT) setup images and kilovoltage CT (kVCT) planning images. Methods: Planning kVCT images were collected for head-and-neck (H&N), pelvis, and spine treatments with corresponding 3D cone-beam CT (CBCT) and megavoltage CT (MVCT) setup images from TrueBeam and TomoTherapy units, respectively. Patient identification errors were simulated by registering setup and planning images from different patients. Positioning errors were simulated by misaligning the setup image by 1cm to 5cm in the six anatomical directions for H&N and pelvis patients. Misalignments for spine treatments were simulated by registering the setup image to adjacent vertebral bodies on the planning kVCT. A body contour of the setup image was used as an initial mask for image comparison. Images were pre-processed by image filtering and air voxel thresholding, and image pairs were assessed using commonly-used image similarity metrics as well as custom -designed metrics. A linear discriminant analysis classifier was trained and tested on the datasets, and misclassification error (MCE), sensitivity, and specificity estimates were generated using 10-fold cross validation. Results: Our workflow produced MCE estimates of 0.7%, 1.7%, and 0% for H&N, pelvis, and spine TomoTherapy images, respectively. Sensitivities and specificities ranged from 98.0% to 100%. MCEs of 3.5%, 2.3%, and 2.1% were obtained for TrueBeam images of the above sites, respectively, with sensitivity and specificity estimates between 96.2% and 98.4%. MCEs for 1cm H&N/pelvis misalignments were 1.3/5.1% and 9.1/8.6% for TomoTherapy and TrueBeam images, respectively. 2cm MCE estimates were 0.4%/1.6% and 3.1/3.2%, respectively. Vertebral misalignment MCEs were 4.8% and 4.9% for TomoTherapy and TrueBeam images, respectively. Conclusion: Patient identification and gross misalignment errors can be robustly and automatically detected using 3D setup images of two imaging modalities across three commonly-treated anatomical sites.

60 APPLIED LIFE SCIENCES↗

Generator Frequency Response Droop Monitoring Tool

Monitoring and analyzing the frequency response performance of power generation units is essential for maintaining reliable and secure power system operation. To address this need, an automation tool has been developed to provide a pipeline for processing historical power plant generation data, including large-scale SCADA archives. The tool performs end-to-end processing, including event detection, frequency response (FR) analysis in accordance with NERC standards, and estimation of speed governor droop characteristics. The tool is designed with a modular architecture, allowing individual components of the workflow to be extended, customized, or deployed independently. In addition, the tool provides an API that enables seamless integration with other production systems and operational analytics platforms.

Etingov, PavelV [Pacific Northwest National Labora↗

Resilient, Rural, and Revolutionary: Salisbury Square's Direct-Current Affordable Microgrid Community: Preprint

The technology to interconnect buildings with a dedicated direct-current (DC) power distribution network is in place today; what is missing is a turnkey approach to designing a DC microgrid - and the business models allowing such systems to be deployed, owned, and operated at scale. To close this gap, the Salisbury Square Development Team, comprising clean-energy experts, has engineered a resilient community DC microgrid for an affordable housing community in Randolph, Vermont. Ten single-family, occupant-owned residences and 12 multifamily rental units will share locally generated and stored solar energy via a DC power distribution bus capable of operating during extended grid outages. With a DC power distribution network in place, each home will be equipped with high-efficiency DC lighting and appliances, operating alongside alternating current (AC) appliances, even during an islanded mode of operation. To obtain a comprehensive understanding of what is possible and achievable, the Team collaborated with the local utility, regulatory agencies, a national laboratory, energy-as-service providers, and vendors. The collaborators evaluated microgrid typologies, business models, and energy modeling, and analyzed electrification and resilience. Further, the Team applied the URBANoptTM (Urban Renewable Building and Neighborhood optimization, NREL 2022) software development kit (SDK) to Salisbury Square's single-family and multifamily buildings to validate workflows and identify needs for advanced capability. This paper addresses the barriers to entry, scalability, and impact on residents and system ownership. It also examines the analysis that informed the design and engineering of the DC microgrid and the opportunities to streamline the process.

DER↗

Digital Twin for Optimizing Real-time Economy of the Integrated Energy Systems

Economic and safe operation of integrated energy systems (IES) requires real-time optimization (RTO) of the control and actions conducted on each system component. In this regard, digital twins (DTs), which consist of a physical system, a virtual system, and the data communication that occurs between the two, are essential for effective RTO. Through the data warehouse, the virtual system is constantly updated with real-time data from the physical system, and functions as the model in the optimization framework. The reduced-order model of the dynamic process model in the virtual system is used in the optimization framework. The optimization results are then returned, via the data warehouse, as control actions to the physical system. This work demonstrates the software capabilities of DT assets for an IES in the context of preparing a DT for an experimental system comprised of Idaho National Laboratory (INL)’s Thermal Energy Delivery System and battery system. For the virtual demonstration, the DTs encompass (1) a physical system, including the Modelica models of the Thermal Energy Delivery System and the battery system; (2) virtual optimization via the Optimization of Real-Time Capacity Allocation (ORCA) platform; and (3) the open-source data warehouse software DeepLynx. This work assesses the performance of ORCA, which utilizes a reduced-order model built using the Risk Analysis Virtual Environment (RAVEN) and trained on the Modelica models and real-time data pipeline through the graph database hosted in DeepLynx. The proposed optimization workflow will be an RTO model based on DTs and the data they generate.

25 ENERGY STORAGE↗

A Methodology to Evaluate the Grid Reliability Impact of Oscillations Induced by Large Loads

The rapid growth of hyperscale AI data centers is bringing renewed attention to the reliability risk that sustained forced oscillations pose to bulk power systems, with cyclic computational workloads emerging as a new forcing source. Unlike the broadband, stochastic disturbances from traditional industrial loads such as arc furnaces, AI training and inference facilities can inject large active power swings concentrated at specific frequencies over extended durations - characteristics that existing grid planning practices do not account for. While the North American Electric Reliability Corporation (NERC) has recognized this gap and called for system-level studies of large load interconnections, no standardized methodology exists to screen, simulate, and quantify these risks at the planning stage. This report presents the Risk Assessment Tool for Large Load-induced Events (RATLLE), a Python-based, publicly available script suite developed at the Pacific Northwest National Laboratory to evaluate bulk power system reliability risks from data center-induced oscillations. RATLLE implements a three-module workflow: a screening module that identifies vulnerable interconnection locations and excitable system modes; a simulation module that models cyclic data center load behavior using a commercial positive sequence simulation platform; and an analysis module that computes risk metrics and generates interactive visualization dashboards. The risk metrics, formulated around simulation observables, map oscillation impacts to a three-stage severity scale spanning latent equipment fatigue through imminent cascading failure. The methodology is demonstrated on two Western Electricity Coordinating Council (WECC) system models: a publicly available 240-bus reduced representation and a detailed 2031 Heavy Winter planning case. Case studies illustrate that even modest 50 MW forced oscillations at resonant frequencies can produce wide-area power swings, N-1 security constraint violations, and cascading generator trips through protection actions - outcomes that would not occur under normal operating conditions without oscillations present. The results underscore the need for standardized oscillation impact assessment in large load interconnection studies and provide a reproducible, extensible framework for utilities to adopt or customize within their existing planning workflows.

Biswas, Shuchismita↗

Near Real-time Data Analysis of Core-collapse Supernova Simulations with Bellerophon

We present an overview of a software system, Bellerophon, built to support a production-level HPC application called CHIMERA, which simulates core-collapse supernova events at the petascale. Developed over the last four years, Bellerophon enables CHIMERA's geographically dispersed team of collaborators to perform data analysis in near real-time. Its n-tier architecture provides an encapsulated, end-to-end software solution that enables the CHIMERA team to quickly and easily access highly customizable animated and static views of results from anywhere in the world via a web- deliverable, cross-platform desktop application. In addition, Bellerophon addresses software engineering tasks for the CHIMERA team by providing an automated mechanism for performing regression testing on a variety of supercomputing platforms. Elements of the team's workflow management needs are met with software tools that dynamically generate code repository statistics, access important online resources, and monitor the current status of several supercomputing resources.

97 MATHEMATICS AND COMPUTING↗

Enhancements to the RAVEN code in FY16

The RAVEN code has been under development at the Idaho National Laboratory since 2012. Its main goal is to create a multi-purpose platform for the deploying of all the capabilities needed for Probabilistic Risk Assessment, uncertainty quantification, data mining analysis and optimization studies. RAVEN has currently reached a good level of maturity in terms of deployed state-of-art and advanced capabilities. The main subject of this report is to show the activities that have been recently accomplished: • Implementation of ensemble modeling for time-series, and • initial implementation of model validation for surrogate models, and • advanced visualization capability for topology based data analysis The development of ensemble modeling for time-series has been performed in order to begin tackling the needs of those RISMC applications that need to communicate 1-Dimensional information (e.g. power histories, etc.) among different models. In this document the implementation details and an application example is reported. The second subject of this report is about the initial development of methods, within the RAVEN framework, to assess the validity of the predictive capabilities of surrogate models. Indeed, after the construction of a surrogate tight to a certain physical model, it is crucial to assess the goodness of its representation, in order to be confident with its prediction. In this initial activity, a cross-validation technique has been employed. This report will highlight the implementation details and proof its correct implementation by an application example. The final subject of this report is about the implementation of advanced visualization capability in RAVEN, for interactive data analysis. Indeed, RAVEN offers several post-processing capabilities that can structurally decompose data extracted from experimental results offering both data clustering/partitioning and dimensionality reduction techniques. A disadvantage of the workflow available in RAVEN is that it treats these as black box operations and the user is expected to know specific information about their data including the number of partitions to expect in some cases or the “correct” parameter settings for a particular algorithm. In order to overcome this limitation, it has been added an interactive user interface that can be run in RAVEN to explore different parameter settings on the fly specifically for the topological post-processor and allows the user to explore the data interactively. The design of this user interface is generalizable to other algorithms and can be used as a model to generate more dynamic and user-friendly visualization capabilities within RAVEN.

97 MATHEMATICS AND COMPUTING↗