Search NASA⌕ Search

SEARCH · Search NASA

Results for “Multiple data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Solar Performance Insight (Final Report)

The PV Operations and Maintenance (O&M) service industry lacks an affordable, well-documented, intuitive PV modeling and analytics tool to calculate modeled performance from actual data from multiple data acquisition systems (DAS). We envision a performance modeling and analytics platform built on open-source, extensible, community-maintained code. The key innovation is the community-driven development of pvlib python delivered through a lightweight web service to provide configurable, consistent and reproducible PV modeling for O&M providers.

14 SOLAR ENERGY↗
DOE OSTI2024

Integrating Multi-Source Data for Bi-Level Traffic Simulator Calibration: A Literature Review and Highway Case Study

Traffic simulation serves as a powerful tool for pre-evaluating policies and technologies. In this context, simulation-based Dynamic traffic assignment (DTA) models are capable of capturing traffic dynamics. They are well-known as critical tools in controlling and predicting traffic situations. The reliability of simulation results heavily depends on the calibration process. Most studies in the literature formulate and calibrate simulators based on a single source of collected data or multiple data sets with the same spatiotemporal characteristics. However, in practice, traffic data is collected by various tools with usually different spatial and temporal resolutions. This study introduces a novel approach to taking into account diverse input data from a variety of sources. An iterative bi-level solution is proposed. to equally treat traffic flow and speed data. The upper level solves flow calibration with the exact solution method, and the lower level calibrates the speed with the simultaneous perturbation stochastic approximation (SPSA) algorithm. Subsequently, the effectiveness of the proposed model is investigated using data from a six-mile section of Nashville's I-24 highway in Tennessee. The results demonstrate that our proposed model creates an effective feedback loop between the optimizer and the simulator for calibrating flow and speed to reduce the error between simulated and real data.

42 ENGINEERING↗
DOE OSTI2024

TeMPI Shim

This is the manual for the TeMPI Shim library, whose goal is to facilitate scientific apps being loosely coupled through MPI. TeMPI Shim can be used to create and manage Message Passing Interface (MPI, see [MPI]) Communicators for Single Program, Multiple Data (SPMD, see [SPMD]) and Multiple Programs, Multiple Data (MPMD, see [MPMD]) workflows. MPI provides the MPI_APPNUM command number to each of the processes within its context. This number, starting at 0, is the application number being executed. In a case where 3 applications are being run in MPMD mode, there would be MPI_APPNUM values of 0, 1, and 2. TeMPI Shim creates intra- and inter-communicators between each pair of MPI_APPNUM values. In the aforementioned case, application 0 would have intra- and inter-communicators to speak to itself, application 1, and application 2. This is replicated for each of the applications. Additionally, TeMPI Shim creates intra- and inter-communicators for the first MPI rank of each application to directly communicate only with each other. Finally, TeMPI Shim creates its own copy of the default world communicator, i.e., MPI_COMM_WORLD. In the case where there is only a single application, it will have the communicators to only speak with itself. TeMPI Shim is useful in this case since it is considered good practice for MPI applications to copy the default world communicator and reference this copy (see [Duplicate World]_) anyways. Ultimately, it provides value independent of the number of applications present.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

Calibrating Synchronous-Generator-Interfaced DG Models in Microgrids Using Multiple Event Data

Microgrids, consisting of distributed generators (DGs), loads, and energy storage systems, play an important role in future smart grids. In order to evaluate how microgrids operate and their dynamic response, it is necessary to develop accurate dynamic models of DGs in microgrids. This paper proposes a systematic method to calibrate synchronous-generator-interfaced DG models. Challenges associated with model validation in microgrids are analyzed. Underlying parameters are categorized into two groups, i.e., the steady-state parameters and time constants, which are estimated in two successive stages using multiple event data. This methodology ensures that the validated model with the estimated parameters are applicable for all events. A modified unscented Kalman filter (UKF) is proposed to deal with the cases in which measurement of power angle is unavailable. The effectiveness of the proposed method is validated using field test data of a 2.6 MVA diesel generator in a real microgrid.

Wang, Zhiwen↗
DOE OSTI2022

Adaptive elasticity policies for staging-based in situ visualization

In situ processing aims to alleviate the growing gap between computation and I/O capabilities by performing data processing close to the data source. In situ processing is widely used to process data generated by multiple data sources, including observation data from edge devices or scientific observational facilities and the simulation data generated by scientific computation on a high-performance computing (HPC) platform. For a scientific workflow that is run on an HPC platform and composed of a simulation program and an in situ data analytics or visualization (abbreviated as ana/vis) task, there is an implicit assumption that the computing resources assigned to the workflow keep static during the workflow execution. However, with the converging trend between the HPC and cloud computing platform, running the in situ ana/vis task in an elastic way is promising to decrease its overhead and improve its resource utilization rate. Resource elasticity represents the ability to change resource configurations such as the number of computing nodes/processes during workflow execution. An elastic job may dynamically adjust resource configurations; it may use a few resources at the beginning and more resources toward the end of the job when interesting data appear. However, it is hard to predict a priori how many computing nodes/processes need to be added/removed during the workflow execution to adapt to changing workflow needs. How to efficiently guide elasticity operations, such as growing or shrinking the number of processes used for in situ analysis during workflow execution, is an open-ended research question. In this article, we present adaptive elasticity policies that adopt workflow runtime information collected during workflow execution to predict how to trigger the addition/removal of processes in order to minimize in situ processing overhead. Taking in situ visualization tasks as an example, we integrate the presented elasticity policies into a staging-based elastic workflow and evaluate its efficiency in multiple elasticity scenarios. Compared with the situation without elasticity or with a static elasticity policy that uses a fixed number of processes for each rescaling operation, the adaptive elasticity policy can save overhead in finding a proper resource configuration and improve resource utilization efficiency. Furthermore, one experiment illustrates that the adaptive elasticity policy saves 41% of core-hours compared with the situation without the resource elasticity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Low-rank Tensor Completion for PMU Data Recovery

This paper proposes a tensor completion method for the recovery of missing phasor measurement unit (PMU) measurements. Tensor completion as the general case of matrix completion has attracted increasing attention in recent years. The imputation accuracy for the existing matrix completion methods may be significantly reduced when there are consecutive data losses across multiple data channels. To tackle this issue, we explore the multi-way characteristics of PMU measurements by using a tensor model. We leverage the low-rank property of the PMU measurements and formulate the missing PMU data recovery problem as a low-rank tensor completion problem. An efficient algorithm based on alternating direction method of multipliers (ADMM) is developed to solve the tensor completion problem. The experiments using the real PMU dataset show that the proposed method exhibits better imputation accuracy compared with the conventional data recovery methods.

Ghasemkhani, Amir↗
DOE OSTI2021

The LAKE model input dataset for three Arctic lakes

This dataset contains meteorological data collected for three Arctic lakes and compiled to satisfy input requirements of the LAKE 2.0 model. The dataset was generated to act as a benchmarking dataset for future model-data inter-comparisons. The LAKE 2.0 model simulates temperatures within the water later and the sedimentary layer of a lake. The LAKE2.0. is an open-source code and available to download via this weblike http://tesla.parallel.ru/Viktor/LAKE/-/wikis/LAKE-model (last visit July 14, 2021). The meteorological data are required to simulate the surface energy balance at the surface of a lake. This dataset includes a compilation of the meteorological data pulled from multiple data streams, including National Oceanic and Atmospheric Administration (NOAA) climate data, Circumarctic Lakes Observation Network (CALON) data, and the United States Geological Survey (USGS) data. The data were compiled for three Arctic lakes: FoxDen (66.55877, -164.45670), Atqasuk (70.452497, -156.951984), and Toolik (68.63150, -149.60740). Each meteorological data is in comma-delimited format (file extension ‘.dat’) and includes eight columns: Temperature [K], Pressure [Pa], longwave downward radiation [W/m2], shortwave downward radiation [W/m2], “U” wind speed [m/s], ”V” wind speed [m/s], humidity [kg/kg], precipitation [m/s]. In addition to the meteorological data file, we included setup and driver files. The Toolik lake is the deepest out of three lakes and has inflowing and outflowing groundwater data. InflowOutflowREADME.txt has more information about inflow and outflow flies. The other two lakes are much shallower and modeled as a closed system (i.e. no water inflow or outflow).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Suppressing simulation bias in multi-modal data using transfer learning

Abstract Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Virtual Log-Structured Storage for High-Performance Streaming

Over the past decade, given the higher number of data sources (e.g., Cloud applications, Internet of things) and critical business demands, Big Data transitioned from batch-oriented to real-time analytics. Stream storage systems, such as Apache Kafka, are well known for their increasing role in real-time Big Data analytics. For scalable stream data ingestion and processing, they logically split a data stream topic into multiple partitions. Stream storage systems keep multiple data stream copies to protect against data loss while implementing a stream partition as a replicated log. This architectural choice enables simplified development while trading cluster size with performance and the number of streams optimally managed. This paper introduces a shared virtual log-structured storage approach for improving the cluster throughput when multiple producers and consumers write and consume in parallel data streams. Stream partitions are associated with shared replicated virtual logs transparently to the user, effectively separating the implementation of stream partitioning (and data ordering) from data replication (and durability). We implement the virtual log technique in the KerA stream storage system. When comparing with Apache Kafka, KerA improves the cluster ingestion throughput by up to 4x when multiple producers write over hundreds of data streams.

consistent stream ordering↗
DOE OSTI2024

Divide and conquer: using RhizoVision Explorer to aggregate data from multiple root scans using image concatenation and statistical methods

Roots are important in agricultural and natural systems for determining plant productivity and soil carbon inputs. Sometimes, the amount of roots in a sample is too much to fit into a single scanned image, so the sample is divided among several scans, and there is no standard method to aggregate the data. Here, we describe and validate two methods for standardizing measurements across multiple scans: image concatenation and statistical aggregation. We developed a Python script that identifies which images belong to the same sample and returns a single, larger concatenated image. These concatenated images and the original images were processed with RhizoVision Explorer, a free and open-source software. An R script was developed, which identifies rows of data belonging to the same sample and applies correct statistical methods to return a single data row for each sample. These two methods were compared using example images from switchgrass, poplar, and various tree and ericaceous shrub species from a northern peatland and the Arctic. Most root measurements were nearly identical between the two methods except median diameter, which cannot be accurately computed by statistical aggregation. We believe the availability of these methods will be useful to the root biology community.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Evaluation of artificial neural network performance for classification of potato plants infected with potato virus Y using spectral data on multiple varieties and genotypes

Potato virus Y (Potyviridae, PVY) is a plant virus that poses a significant threat to potato producers on a global basis. The pathogen has disrupted seed potato supplies and negatively impacted yield and quality of commercial potato crops. The potato industry currently manages PVY infection levels via insecticide applications, regional seed certification programs that rely on field scouting to visually assess individual plants for infection status, and destructive and costly tissue sampling coupled with laboratory assays. Despite these efforts, PVY continues to confound potato industry stakeholders resulting in economic harm. Remote sensing and machine learning provide for the development of new tools to more accurately detect and spatially quantify PVY-infected plants versus the current state of the art. However, there is a need to understand how the occurrence of many different potato varieties impact the dynamics of developing models to detect potato plants impacted with PVY and their potential effectiveness. This study evaluates classification modelling outcomes using spectral datasets collected in different temporal and spatial environments (greenhouse and a production field) on multiple potato varieties consisting of labelled instances of plants infected with PVY and those not infected with the virus. A modelling framework was developed to support iterative modelling runs using artificial neural network (ANN) architectures configured as binary classifiers to develop sample populations to support statistical analysis on model performance using specific spectral subsets. When using spectral data to detect PVY-infected plants, ANN models achieved the highest mean accuracy of 0.894 on a single variety. Conversely, the same ANN model architecture only achieved a mean accuracy of 0.575 on a spectral data set representing 29 potato breeding lines. Additionally, statistical analysis indicates spectral regions including the red edge, near infrared and shortwave infrared contain more important spectral features for the ANN classifier introduced in this research.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

An MPMD approach coupling electromagnetic continuum mechanics approximations in ALEGRA

In this work, two complementary approximations for describing aspects of continuum electromagnetics in moving media are discussed: electroquasistatic and magnetoquasistatic. Each has been implemented in the finite element shock code ALEGRA for modeling dynamic electromechanical phenomena on typical engineering time scales, with fully integrated circuit coupling. The approximations can be obtained by consistent asymptotic balancing of Maxwell’s equations relative to timescales associated with magnetic diffusion, charge relaxation, and electromagnetic wave propagation. In ALEGRA, the electroquasistatic approximation is used for ferroelectric (FE) modeling, while the magnetoquasistatic approximation is used for magnetohydrodynamic (MHD) modeling. In this paper we introduce for the first time a detailed derivation of a useful quasi-steady “low-R m ” variant of the MHD approximation applicable for cases, such as with detonators, where the thermodynamic pressure arising from Joule heating dominates over magnetic forces. An additional purpose of this paper is to present a coupling mode using Multiple Program-Multiple Data (MPMD) message passing communication that allows the user to run 3D FE problems together with 2D and/or 3D MHD problems with the respective simulation domains coupled through a common circuit equation. The MPMD coupling capability is used here to model the dynamic coupling of a notional ferroelectric generator with an RP-87 exploding bridgewire detonator. The simulated bridgewire heats up and bursts under current generated by simulated depoling of the ferroelectric generator, as a demonstration of the MPMD capability.

42 ENGINEERING↗
DOE OSTI2025

MicroFisher: Fungal taxonomic classification for metatranscriptomic and metagenomic data using multiple short hypervariable markers

AbstractProfiling the taxonomic and functional composition of microbes using metagenomic (MG) and metatranscriptomic (MT) sequencing is advancing our understanding of microbial functions. However, the sensitivity and accuracy of microbial classification using genome– or core protein-based approaches, especially the classification of eukaryotic organisms, is limited by the availability of genomes and the resolution of sequence databases. To address this, we propose the MicroFisher, a novel approach that applies multiple hypervariable marker genes to profile fungal communities from MGs and MTs. This approach utilizes the hypervariable regions of ITS and large subunit (LSU) rRNA genes for fungal identification with high sensitivity and resolution. Simultaneously, we propose a computational pipeline (MicroFisher) to optimize and integrate the results from classifications using multiple hypervariable markers. To test the performance of our method, we applied MicroFisher to the synthetic community profiling and found high performance in fungal prediction and abundance estimation. In addition, we also used MGs from forest soil and MTs of root eukaryotic microbes to test our method and the results showed that MicroFisher provided more accurate profiling of environmental microbiomes compared to other classification tools. Overall, MicroFisher serves as a novel pipeline for classification of fungal communities from MGs and MTs.

Wang, Haihua↗
DOE OSTI2026

ndi

The Nuclear Data Interface (NDI) is an application programming interface (API) that allows access to standard nuclear data parameters while hiding the underlying details of the data libraries and their storage. It allows access to multigroup transport data (neutron and gamma), thermonuclear burn data, dosimetry data, production/depletion chain data, radiochemistry data, and secondary neutron multiplicity data. The name NDI refers to both the code and data formats supported by the code.

Saller, Thomas↗
DOE OSTI2021

Evaluation and improvement of the parameterization of aerosol hygroscopicity in global climate models using in-situ surface measurements (Final Report)

Aerosols are tiny particles suspended on the atmosphere that can interact with incoming solar radiation and affect the Earth radiative budget. They do so by scattering and absorbing solar radiation, and these properties vary depending on the aerosol size and chemical composition. Moreover, by taking up water from the surrounding air, hygroscopic aerosol particles will grow in size and change their chemical composition, thus modifying their optical properties (scattering and absorption) and their final impact on radiative forcing calculations. An accurate knowledge of aerosol hygroscopicity is crucial for estimating the net radiative impact of aerosols. We took a three-pronged approach to improve our understanding of aerosol hygroscopicity and how it is implemented in Earth system models. In the first part of our project, we developed a benchmark dataset from existing aerosol hygroscopic growth measurements made by tandem nephelometer humidogram systems. We analyzed, using a standardized methodology, observations from 26 in-situ stations around the globe. Measurement data was collected from multiple data providers, reviewed and harmonized to create a consistent dataset of the scattering enhancement factor due to aerosol water uptake. This dataset is archived in several publicly available databases for use by interested researchers. In the second part of the project, we used the benchmark hygroscopicity dataset to perform a global study on aerosol hygroscopicity and aerosol optical properties. Measurements show a global picture of scattering enhancement with larger values for Arctic and marine sites and lower for urban and desert sites. We assessed the RH dependence of aerosol radiative forcing and showed that the overall effect of aerosol hygroscopicity on DARF is an increase in the absolute forcing effect (negative sign) by a factor of up to 4 compared to dry conditions (RH<40%). Finally, we explored using aerosol single scattering albedo (SSA) and scattering Angstrom exponent (SAE) as possible proxies for aerosol hygroscopicity. SSA showed more promise as a surrogate for the scattering enhancement factor than SAE, but neither was ideal. In the third part of the project we evaluated the output of ten Earth system models (ESMs) against the benchmark hygroscopicity dataset. ESMs utilize various schemes for aerosol hygroscopicity which had not been previously tested against observations on a global scale. We found that ESMs currently overestimate scattering enhancement due to hygroscopic growth. Model parameterizations of hygroscopicity and model chemistry are two main factors driving the observed diversity in hygroscopicity simulations among the models. In addition, our study makes several suggestions for modelers, including improving the parameterizations of organic and sea salt aerosol hygroscopicity. Future hygroscopicity model evaluation experiments should include the model data related to particles size which was not available for our study.

54 ENVIRONMENTAL SCIENCES↗