Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Learning to simulate high energy particle collisions from unlabeled data

In many scientific fields which rely on statistical inference, simulations are often used to map from theoretical models to experimental data, allowing scientists to test model predictions against experimental results. Experimental data is often reconstructed from indirect measurements causing the aggregate transformation from theoretical models to experimental data to be poorly-described analytically. Instead, numerical simulations are used at great computational cost. We introduce Optimal-Transport-based Unfolding and Simulation (OTUS), a fast simulator based on unsupervised machine-learning that is capable of predicting experimental data from theoretical models. Without the aid of current simulation information, OTUS trains a probabilistic autoencoder to transform directly between theoretical models and experimental data. Identifying the probabilistic autoencoder’s latent space with the space of theoretical models causes the decoder network to become a fast, predictive simulator with the potential to replace current, computationally-costly simulators. Here, we provide proof-of-principle results on two particle physics examples, Z-boson and top-quark decays, but stress that OTUS can be widely applied to other fields.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

NBIC Biofeeds: A Digital Tool for Open Source Biosurveillance across Federal Agencies

The National Biosurveillance Integration Center (NBIC) is developing a scalable, flexible open source data collection, analysis, and dissemination tool to support biosurveillance operations by the U.S. Department of Homeland Security (DHS) and its federal interagency partners. The NBIC integrates, analyzes, and distributes key information about health and disease events to help ensure the nation’s responses are well-informed, save lives, and minimize economic impact. NBIC serves as a bridge between Federal, State, Local, Territorial, and Tribal entities to conduct biosurveillance across human, animal, plant, and environmental domains. The integration of information enables early warning and shared situational awareness of biological events to inform critical decisions directing response and recovery efforts. To meet its mission objectives, NBIC utilizes a variety of datasets, including open source information, to provide comprehensive coverage of biological events occurring across the globe. NBIC Biofeeds is a digital tool designed to improve the efficiency of reviewing and analyzing large volumes of open source reporting by biosurveillance analysts on a daily basis; moreover, the system provides a mechanism to disseminate tailored feeds allowing NBIC to better meet the specific information needs of individual, interagency partners. The tool is currently under development by the Department of Energy (DOE), Pacific Northwest National Laboratory (PNNL)and it is in a testing and evaluation phase supported by NBIC biosurveillance subject matter experts. Integration with the Defense Threat Reduction Agency (DTRA), Biosurveillance Ecosystem(BSVE) is also underway. NBIC Biofeeds Version 1 is expected to be fully operational in Fiscal Year 2017. The PNNL is applying agile methodology to streamline the build of NBIC Biofeeds to specifications required for operational use by NBIC and its federal interagency partners. Biosurveillance, analytics, and system engineering subject matter experts provide guidance on the implementation of features in the tool to ensure functionality aligns with operational workflows and production support. PNNL is leveraging software from a previous government effort to repurpose the technology to meet NBIC needs. NBIC Biofeeds incorporates the open source, document-orientated MongoDB database to capture user- and system-generated metadata on hundreds of thousands of records, in part, to establish baselines to aid prospective and retrospective analysis on emerging biological events. NBIC Biofeeds integrates a biosurveillance taxonomy (uniquely developed by NBIC),which includes input from interagency partners to recognize critical characteristics of a biological event. In NBIC Biofeeds Version1, metadata capture of reported events is done manually by NBIC analysts; however, moving forward in Version 2, the tool will be further automated to flag significant reporting on biological events with a human remaining in the loop to confirm the validity of the system-generated tags. To serve as a one-stop tool for open source biosurveillance, NBIC Biofeeds automatically harvests information from thousands of websites, utilizing third party aggregators, paid subscriptions to data feeds, and scraping of high priority sources. Users can develop desired queries for automatic updating, leverage a unique review and curation mechanism, and further analyze data from topical, geographic, and temporal visualization features in the tool. To meet NBIC’s information sharing needs, the tool allows for design of tailored RSS feeds and electronic message-based delivery of analysis on biological events, intended for recipients in the government with unique missions around human, animal, plant, and environmental health. Through current testing and evaluation – underway by biosurveillance subject matter experts – NBIC Biofeeds is demonstrating value in supporting open source biosurveillance by the Center for more rapid recognition and sharing of key event characteristics. Centralizing access and analysis of this data set into a single system is increasing the efficiency of daily, global biosurveillance, while enhancing the value of information identified through use of the querying, curation, and production support features in the tool.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluation of Correction Methods for NASA GeneLab Transcriptomic Datasets

Conducting space biology experiments aboard the International Space Station, particularly those utilizing complex model organisms like mice, is expensive and difficult due to limited crew availability, hardware, and space. As a result, sample numbers from these studies are low, reducing the statistical power of any one experiment. Aggregating spaceflight datasets serves as a method to increase sample numbers, allowing for novel insights through bioinformatic analysis of ‘omics data from merged datasets. However, aggregating datasets can introduce unwanted variation including 1) differences in sample handling, processing, and sequencing platforms between datasets (technical variation) as well as 2) differences in experimental design between datasets. In the present study, NASA GeneLab-hosted RNAseq datasets from mouse liver tissues were used to evaluate several statistical methods to correct for this unwanted variation through two approaches, reference-based and standard. The following correction algorithms were applied with (reference-based) and/or without (standard) considering Universal Mouse RNA Reference samples: ComBat and ComBat_seq from the SVA package, median polish, empirical Bayes, and ANOVA-based algorithms from the MBatch package, and negative binomial regression normalization in the DESeq2 package. For each approach, after the correction algorithm was applied, differential gene expression (DGE) analysis of flight and ground control samples was performed with the combined data. The robustness of each tool was evaluated using BatchQC to determine statistical differences between datasets before and after correction, Principal Component Analysis to evaluate global gene expression in samples before and after correction, and by comparing DGE analysis of individual datasets and combined datasets before and after correction. The results showed that the reference-based approach introduced several additional (and likely artificial) DEGs when compared with the respective standard approach. Of the methods tested, standard ComBat and DESeq2 were identified as the most robust correction methods for combining spaceflight mouse liver RNAseq datasets hosted on GeneLab.

GeneLab↗

Combining RNA-SEQ Datasets from NASA GENELAB: An Evaluation of Correction Methods

Background: Conducting space biology experiments aboard the International Space Station, particularly those utilizing complex model organisms like mice, is expensive and difficult due to limited crew availability, hardware, and space. As a result, sample numbers from these studies are low, reducing the statistical power of any one experiment. Aggregating spaceflight datasets serves as a method to increase sample numbers, allowing for novel insights through bioinformatic analysis of ‘omics data from merged datasets. However, aggregating datasets can introduce unwanted variation including 1) differences in sample handling, processing, and sequencing platforms between datasets (technical variation) as well as 2) differences in experimental design between datasets. Methods: In the present study, NASA GeneLab-hosted RNAseq datasets from mouse liver tissues were used to evaluate several statistical methods to correct for this unwanted variation through two approaches, reference-based and standard. The following correction algorithms were applied with (reference-based) and/or without (standard) considering Universal Mouse RNA Reference samples: ComBat and ComBat_seq from the SVA package, the median polish, empirical Bayes, and ANOVA-based algorithms from the MBatch package, and negative binomial regression normalization in the DESeq2 package. For each approach, after the correction algorithm was applied, differential gene expression (DGE) analysis of flight and ground control samples was performed with the combined data. The robustness of each tool was evaluated using BatchQC to determine statistical differences between datasets before and after correction, Principal Component Analysis to evaluate global gene expression in samples before and after correction, and by comparing DGE analysis of individual datasets and combined datasets before and after correction. Results: The results showed that the reference-based approach introduced several additional (and likely artificial) differentially expressed genes when compared with the respective standard approach. Conclusions: Of the methods tested, standard ComBat_seq and DESeq2 were identified as the most robust correction methods for combining spaceflight mouse liver RNAseq datasets hosted on GeneLab.

Finsam Samson↗

Evaluation of Correction Methods for NASA GeneLab Transcriptomic Datasets

Conducting space biology experiments aboard the International Space Station, particularly those utilizing complex model organisms like mice, is expensive and difficult due to limited crew availability, hardware, and space. As a result, sample numbers from these studies are low, reducing the statistical power of any one experiment. Aggregating spaceflight datasets serves as a method to increase sample numbers, allowing for novel insights through bioinformatic analysis of ‘omics data from merged datasets. However, aggregating datasets can introduce unwanted variation including 1) differences in sample handling, processing, and sequencing platforms between datasets (technical variation) as well as 2) differences in experimental design between datasets such as sex or age of the model organism used. In the present study, NASA GeneLab-hosted RNAseq datasets from rodent liver tissues were used to evaluate several statistical methods to correct for this unwanted variation through two approaches, reference-based and standard. The following correction algorithms were applied with (reference-based) and/or without (standard) considering Universal Mouse RNA Reference samples: ComBat and ComBat_seq from the SVA package, median polish, empirical Bayes, and ANOVA-based algorithms from the MBatch package, and negative binomial regression normalization in the DESeq2 package. For each approach, after the correction algorithm was applied, differential gene expression (DGE) analysis of flight and ground control samples was performed with the combined data. The robustness of each tool was evaluated using BatchQC, to determine statistical differences between datasets before and after correction, Principal Component Analysis, to evaluate global gene expression in samples before and after correction, and by comparing DGE analysis of individual datasets and combined datasets before and after correction. The results showed that the reference-based approach introduced several additional (and likely artificial) DEGs when compared with the standard approach. Thus, the most robust standard correction will be implemented in the GeneLab Visualization 2.0 platform when datasets are combined.

GeneLab, RNA-seq, Batch Correction↗

Performance Assessments of the SNPP and N20 VIIRS DNB Using Observations of Bright Stars

The Visible Infrared Imaging Radiometer Suite (VIIRS) on-board the Suomi-NPP (SNPP) and NOAA-20 (N20) spacecrafts is a multi-spectral Earth-observing instrument with spectral channels ranging from the visible to the long-wave infrared. In addition to these spectral channels, the VIIRS instrument features a panchromatic Day/Night band (DNB) with 3 gain stages (low, mid, and high) which allows for Earth-view (EV) radiance retrievals over 7 orders of magnitude. The DNB is able to collect data ranging from dim city lights during nighttime up to reflected sunlight from cloud-tops during daytime. The DNB data collection is divided into 32 aggregation modes, where sub-pixel aggregation is used to maintain a nearly constant pixel footprint on ground during EV data collection throughout each scan. The DNB is also sensitive enough to observe bright stars through the instrument space-view (SV) port. Data from these bright stars can be used to track the temporal stability of the sensor gain and as a method of intercomparison between the two VIIRS instruments. In this work, we will present several sensor performance assessments of the VIIRS DNB using stars. In addition to the sensor gain trending, these assessments will include a comparison of the differences between the daytime and nighttime data due to the effects of stray-light, a comparison of the measured signal as a function of aggregation mode, a comparison between the two high-gain sub-stages, and half-angle mirror side for both the SNPP and N20 VIIRS DNB.

VIIRS↗

Shifting temporal dynamics of human mobility in the United States

In this paper we analyze the average hourly temporal dynamics of human mobility in the United States from 2019 to 2020. We discuss how large decreases in human mobility nonuniformly effect the daily temporal dynamics of aggregate human behavior. The data used are weekly activity patterns for POIs from 2019 to 2020 in the United States, provided by SafeGraph and made openly available to academic and research institutions. We use clustering methods to create metrics describing how human activity changes throughout the day/week at the county and national levels. In response to significant mobility reductions starting March 2020, daily temporal patterns of human activity changed nonuniformly. Morning activity started later, and evening activity started earlier in 2020 compared to 2019, and temporal behavioral patterns on weekdays began to look more similar to weekends. The changes in daily temporal behavior persisted throughout the year even as total mobility levels recovered. The results provide insights on the changes in human behavior in response covid-19 policies and illustrate influences on social systems, health, and transportation networks.

99 GENERAL AND MISCELLANEOUS↗

Dynamic signal recovery in distribution grids using compressive lossy measurements

Distribution system state estimation requires reliable aggregation of the measured data. However, the large volume of the measured data imposes a significant stress on the underlying communication infrastructure. With the challenges associated with measurement availability, current distribution systems are typically unobservable. To cope with the unobservability issue, compressive sensing theory allows us to recover system state information from a small number of measurements provided the states of the distribution system exhibit sparsity. In this paper, we evaluate the robustness of an updated Kalman filtered modified compressive sensing (KF-ModCS) technique that dynamically estimates the grid states using a small fraction of measured data. In practice, measurements used for sparsity based state estimation may also be intermittent due to communication network induced losses. Further, to understand the effect of packet losses on KF-ModCS, we provide an upper bound for the expected variances of the state estimation error for a given rate of information loss. This upper bound is further improved if the support set of the sparse signal that characterizes the state dynamics does not change over time and/or the reduced model is observable. Simulations based on two practical data sets collected from actual customers in a distribution grid validate the theoretical results.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Rapid Electrochemical Methane Functionalization Involves Pd–Pd Bonded Intermediates

High-valent Pd complexes are potent agents for the oxidative functionalization of inert C-H bonds, and it was previously shown that rapid electrocatalytic methane monofunctionalization could be achieved by electro-oxidation of Pd II to a critical dinuclear Pd III intermediate in concentrated or fuming sulfuric acid. However, the structure of this highly reactive, unisolable intermediate, as well as the structural basis for its mechanism of electrochemical formation, remained elusive. Herein, we use X-ray absorption and Raman spectroscopies to assemble a structural model of the potent methane-activating intermediate as a Pd III dimer with a Pd-Pd bond and a 5-fold O atom coordination by H x SO 4 (x-2) ligands at each Pd center. We further use EPR spectroscopy to identify a mixed-valent M-M bonded Pd 2 II,III species as a key intermediate during the Pd II -to-Pd III 2 oxidation. Combining EPR and electrochemical data, we quantify the free energy of Pd dimerization as <-4.5 kcal/mol for Pd 2 II,III and <-9.1 kcal/mol for Pd III 2 . The structural and thermochemical data suggest that the aggregate effect of metal-metal and axial metal-ligand bond formation drives the critical Pd dimerization reaction in between electrochemical oxidation steps. This work establishes a structural basis for the facile electrochemical oxidation of Pd II to a M-M bonded Pd III dimer and provides a foundation for understanding its rapid methane functionalization reactivity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Implementation of FAIR principles in the IPCC: the WGI AR6 Atlas repository

The Sixth Assessment Report (AR6) of the Intergovernmental Panel on Climate Change (IPCC) has adopted the FAIR Guiding Principles. We present the Atlas chapter of Working Group I (WGI) as a test case. We describe the application of the FAIR principles in the Atlas, the challenges faced during its implementation, and those that remain for the future. We introduce the open source repository resulting from this process, including coding (e.g., annotated Jupyter notebooks), data provenance, and some aggregated datasets used in some figures in the Atlas chapter and its interactive companion (the Interactive Atlas), open to scrutiny by the scientific community and the general public. We describe the informal pilot review conducted on this repository to gather recommendations that led to significant improvements. Finally, a working example illustrates the re-use of the repository resources to produce customized regional information, extending the Interactive Atlas products and running the code interactively in a web browser using Jupyter notebooks.

54 ENVIRONMENTAL SCIENCES↗

A Contextually Supervised Optimization-Based HVAC Load Disaggregation Methodology

This paper presents a novel contextually supervised optimization-based approach for disaggregating heating, ventilation, and air-conditioning (HVAC) loads using smart meter or Supervisory Control and Data Acquisition data. To disaggregate the load into HVAC loads, large and infrequently used loads (LIUL), and base loads, we formulate an optimization problem to minimize a set of five loss terms, consisting of the reconstruction errors of the overall load profile, the ramp rate losses, and three distinct loss functions linked with the HVAC load, base load, and LIUL, respectively. To enhance accuracy, we incorporate two forms of contextual information into the problem formulation. First, we utilize mutual information to estimate HVAC energy consumption. Second, we employ a base load dictionary to constrain HVAC load estimation errors. The obtained HVAC load profiles are fine-tuned by abnormal ramp detection followed by binary hypothesis testing. Here, the proposed method is developed and tested using sub-metered residential and commercial building data. Simulation results show that the proposed method outperforms existing methods across various data resolutions and load aggregation levels, showing excellent transferability and generalizability.

24 POWER TRANSMISSION AND DISTRIBUTION↗

User's Guide for REFoffSpec Version 1.5.4

This document is a user s guide for the IDL software REFoffSpec version 1.5.4 whose purpose is to aggregate for analysis NeXus data files from the magnetism and liquids reflectometer experiments at the Oak Ridge National Laboratory Spallation Neutron Source. The software is used to scale and align multiple data files that constitute a continuous set for an experimental run. The User s Guide for REFoffSepc explains step by step the process using a specific example run. Output screens are provided to orient the user at each step. The guide documents in detail changes made to the original REFoffSpec code during the period November 2009 and January 2011. At the time of the completion of this version of the code it was accessible from the sns_tools interface as a beta version.

Ward, Richard C.↗

National Energy Water Treatment & Speciation (NEWTS): A Water & Critical Minerals Database and Dashboard

The poster, presented at the American Geophysical Union (AGU) 2023 Conference, highlights NETL's National Energy Water Treatment & Speciation (NEWTS) Database & Dashboard. This poster reviews the work done to aggregate disparate water quality data for energy-related wastewater streams, to process and integrate the datasets, and enable easy integration with aqueous chemistry modeling software to support water management and beneficial reuse decision-making.

Wenzlick, Madison↗

Analyzing Risks of Virtual Private Network Connections

The use of Splunk for analyzing VPN logs is an effective approach for identifying vulnerabilities in network endpoints. Splunk, a powerful platform for searching, monitoring, and analyzing machine-generated data, enables organizations to aggregate VPN logs in real-time, providing insights into network activity, user behavior, and potential security risks. By indexing VPN traffic and authentication logs, security teams can track abnormal patterns such as multiple failed login attempts, unusual IP addresses, or unexpected changes in bandwidth usage, all of which could indicate potential vulnerabilities or breaches. With Splunk’s advanced search and reporting capabilities, users can create custom dashboards and alerts to detect suspicious activities. Automated searches can flag endpoints exhibiting unusual behavior, while correlation analysis can identify links between compromised devices and broader network vulnerabilities. In particular, Splunk's machine learning capabilities can be leveraged to predict and prevent threats by identifying trends that might otherwise be missed in traditional log analysis. This proactive approach to monitoring VPN logs allows for the early detection of security weaknesses, enabling rapid response and minimizing potential damage to network integrity. By enhancing endpoint visibility, Splunk plays a crucial role in securing remote connections and safeguarding sensitive information. Additionally, Splunk’s automation and alerting features allow teams to create custom workflows that notify them of vulnerable or misconfigured endpoints identified through Shodan. This synergy between Splunk’s log analysis and Shodan’s device intelligence enhances an organization’s ability to proactively identify and mitigate security risks, improving the overall resilience of their VPN infrastructure.

97 MATHEMATICS AND COMPUTING↗

Solar and Battery Storage Permitting and Siting Requirements - Solar Prize Round 7 (CRADA Final Report)

The purpose of this project is to identify local zoning ordinances, state permitting requirements, federal regulatory bottlenecks throughout the development process, and the potential to use Large Language Models (LLM) and aggregate permitting and interconnection data for commercial and industrial (C&I), community, and utility-scale solar photovoltaic (PV) and battery storage projects.

14 SOLAR ENERGY↗

eMosaic: Electrification Mosaic Platform for Grid Informed Smart Charging Management (Final Scientific/Technical Report)

ABB (Prime Contractor), in collaboration with its partners at the Utah State University (USU), Idaho National Laboratory, Rocky Mountain Power (RMP), and Electric Power Engineers (EPE), have performed research, development, and wide scale demonstration of a scalable and resilient Electrification Mosaic (eMosaic) platform for Smart Charge Management (SCM) for Electric Vehicle Infrastructure. Work was completed under DE EE0009194, titled “eMosaic Electrification Mosaic Platform for Grid Informed Smart Charging Management”, funded by the US Department of Energy. The project members developed algorithms that provide localized and bulk grid services and that reduce and stabilize costs all the way down the supply chain to the PEV owner through SCM. This platform aggregates telemetry from multiple data sources as pieces of the larger picture including personal, private fleet or transportation EVs, fast chargers and other supply equipment, weather service information, and geographically distributed charging sites such as public lots, garage and retail, and private or shared usage depots. ABB and the project team designed, tested, and improved a charging management system at local/edge and cloud levels. The ultimate objective of the project was to convincingly demonstrate that the developed secure eMosaic plat-form can be readily and favorably adopted by diverse utilities and site owners at scale. This was achieved through a demonstration plan with field deployment at several physical sites across 4 states and additional scalable simulation from high fidelity charging models.

24 POWER TRANSMISSION AND DISTRIBUTION↗