Salvaging Data Records with Missing Data: Data Imputation using the Multivariate t Distribution
No abstract provided
DiNola, Michael↗
SEARCH · Search NASA
Results for “data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Automated Resonance Fitting for Nuclear Data Evaluation
Global and national efforts to deliver high-quality nuclear data to users have a wide-ranging impact, affecting applications in national security, reactor operations, basic science, medicine, and more. Cross section evaluation is a major part of this effort, combining theory and experimentation to produce recommended values and uncertainties for reaction probabilities. Resonance region evaluation is a specialized type of nuclear data evaluation that can require significant manual effort and months of time from expert scientists. In this article, non-convex non-linear optimization methods are combined with concepts of inferential statistics to infer a resonance model from experimental data in an automated manner that is not dependent on prior evaluation(s). This methodology aims to enhance the workflow of a resonance evaluator by minimizing time, effort, and the potential for bias from prior assumptions, while enhancing reproducibility and documentation, thereby addressing well-known challenges in the field.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Development of a Data Overflow Protection System for Super-Kamiokande to Maximize Data from Nearby Supernovae
Neutrinos from very nearby supernovae, such as Betelgeuse, are expected to generate more than ten million events over 10 s in Super-Kamokande (SK). At such large event rates, the buffers of the SK analog-to-digital conversion board (QBEE) will overflow, causing random loss of data that are critical for understanding the dynamics of the supernova explosion mechanism. In order to solve this problem, two new data-acquisition (DAQ) modules were developed to aid in the observation of very nearby supernovae. The first of these, the SN module, is designed to save only the number of hit photomultiplier tubes during a supernova burst and the second, the Veto module, prescales the high-rate neutrino events to prevent the QBEE from overflowing based on information from the SN module. In the event of a very nearby supernova, these modules allow SK to reconstruct the time evolution of the neutrino event rate from beginning to end using both QBEE and SN module data. This paper presents the development and testing of these modules together with an analysis of supernova-like data generated with a flashing laser diode. We demonstrate that the Veto module successfully prevents DAQ overflows for Betelgeuse-like supernovae as well as the long-term stability of the new modules. During normal running the Veto module is found to issue DAQ vetos a few times per month resulting in a total dead-time less than 1 ms, and does not influence ordinary operations. Additionally, using simulation data we find that supernovae closer than 800 pc will trigger the Veto module, resulting in a prescaling of the observed neutrino data.
F20 Instrumentation and technique↗
A Data Processing Pipeline To Extract A Knowledge Graph From Heterogeneous Data For Socio-technical Analysis Of Critical Infrastructure Influence
The code is written in Python and consists of the following pipeline that is implemented in Apache Airflow. This pipeline intends to understand the companies that are directly or indirectly involved with a type of critical infrastructure system at some point in that system's lifecycle. The pipeline takes a configuration file that specifies a list of initial companies to consider, a geographic region of interest, and a set of SEC form types as well as other data sources (e.g. CrunchBase) from which to extract entities and relations. There are four main components to this pipeline as currently implemented: Entity Extraction, Network Construction, Analysis, and Visualization. First, Entity Extraction, is implemented as the `topear-extract_organizations` Apache Airflow workflow. Given an initial query that specifies a geographic region of interest and a time interval, the software will extract CI facilities of interest and organizations that have a direct influence relationship to those facilities (e.g. ownership). During the course of the LDRD, we focused on Electric Vehicle charging stations and this information is available via the Department of Energy (DOE) database on fueling stations maintained by NREL. Within the context of the DOE CESER project, we have focused on Battery Energy Storage Systems (BESS). Second, the Network Extraction component will iteratively construct a social network graph given the set of organizations and people extracted in the previous step. Organizations (and eventually People if desired) are then fed as a query to the `topgear-construct_social_network` Apache Airflow workflow which given a set of initial companies and data sets (e.g. SEC EDGAR form types, OpenCorporates, Crunchbase). This Airflow workflow will iteratively query such data sources to discover relationships with new organizations and people. For example, this module can iteratively query SEC EDGAR for metadata that documents the number of each type of form for the given set of companies and their location. This forms metadata represents a catalog of data sources from SEC EDGAR for the extracted social network knowledge graph. The pipeline then downloads these forms from the website and saves them in a build directory for further processing. These documents are then parsed for entities and relations. Again, we note that in additional to SEC data sources, this step can also pull in information on organizations via API services such as CrunchBase and OpenCorporates or bulk data sources. At the end of this step, the resultant social network, the Critical Infrastructure network, and the edges that encode relationships between organizations and CI facilities, form the Adversarial Socio-Technical Network (ASTN) that informs the analysis. Third, the Analysis component processes these generated ASTN. Previously, that has included the ability to compare prevalence of different vendors for a given infrastructure component type across different regions as well as identify common public and private investors across those vendors. This was demonstrated for EV Charging Stations across several different metropolitan areas within an IEEE PES GridEdge publication. More recently, we have looked at ways to identify infrastructure owners and operators of BESS with the most nameplate capacity across different states as well as other indictors of risk resulting from changes in ownership over time. Finally, the Visualization component consists of an HTML/CSS/JS framework by which users can interact geospatial, operational, and organizational relationships across a given portfolio of Critical Infrastructure facilities. The objective is to provide a library of UI/UX modules that can be repurposed for stakeholder-specific dashboards. All of the modules are related via a common event model that enables UI actions in one view to percolate across the other views.
Weaver, Gabriel [Idaho National Laboratory (INL), ↗
Data for Spatial Analysis of Cell Patterning to Aid Genetic and Phenotypic Understanding of Grass Stomatal Density: A Case Study in Maize
Biological processes involve complex hierarchies where composite traits result from multiple component traits. However, holistically understanding of how sets of component traits interact to underpin genotype-to-phenotype relationships is generally lacking. Stomatal density (SD) is a tractable model system for exploring how high-throughput phenotyping (HTP) data could be exploited by a new spatial analysis approach to better understand a developmentally and functionally important trait. SD is a composite trait, resulting from various components related to cell identity and size, which are themselves governed by a series of spatio-developmental processes. Data from 192 recombinant inbred lines of maize [Zea mays (L.)] were analyzed by a new stomatal patterning phenotype (SPP) to (1) describe the average spatial probability distribution of the nearest neighboring stomata; (2) derive a core set of component traits related to cell size, cell packing, and positional probabilities; (3) build a structural equation model of component traits underlying SD; and (4) identify stomatal patterning quantitative trait loci (QTL). The core set of SPP-derived traits explained 74% of the variation in SD. Analyzing SPP component traits allowed some loci previously identified as generic SD QTL to be recognized as specific to lateral versus longitudinal elements of stomatal patterning. Therefore, this study highlights how novel insights can be gained by decomposing a composite trait (e.g., SD) into a set of component traits that were present in HTP data but not previously exploited.
AI/ML↗
Automated analysis of unlabeled PV data with Solar Data Tools software: Overview and feature updates
Distributed rooftop PV systems: ubiquitous, yet commonly have unlabeled data Difficult or impossible to form a performance index We developed Solar Data Tools (SDT), an open-source Python library for analyzing PV power (and irradiance) time-series data SDT enables analysis of unlabeled PV data—no model, no meteorological data, no performance index required Takes a statistical signal processing approach Data processing steps are largely pre-defined and automatic regardless of system type—from utility tracking systems to multi-pitch rooftop systems
Meyers-Im, Bennet E↗
FY 2024 Multidimensional Data Correlation Platform Data Management Infrastructure Progress: Materials Laboratory
This report provides an inventory of the equipment available at the ORNL Manufacturing Demonstration Facility (MDF) for sample preparation and material characterization, including both destructive and non-destructive techniques that generate critical data to support the development of the Multi-Dimensional Data Correlation (MDDC) framework. The success of the MDDC framework depends heavily on the quality and completeness of the data it can access. Therefore, it is essential to establish a comprehensive inventory of the technologies available to the Advanced Materials and Manufacturing Technologies (AMMT) multi-laboratory team. This starts by gathering information about the types of data they produce, the data collection and transfer protocols used, file formats, and data storage requirements for experiments. This information is then carefully evaluated to create the operations and trackables elements of the Damara Tern platform, which is the foundation of the MDDC framework.
36 MATERIALS SCIENCE↗
FY 2025 Multidimensional Data Correlation Platform: Unified Software Architecture for Advanced Materials and Manufacturing Technologies Data Management and Processing
The Advanced Materials and Manufacturing Technologies (AMMT) program continues to advance a data-driven approach to demonstrate the utility of additive manufacturing for fabricating components for nuclear applications. A key scientific goal is to leverage data to better understand manufacturing outcomes and thereby improve the performance, reliability, and lifespan of nuclear components. Ultimately, this effort supports the development of standards for certification and qualification of additively manufactured components, enabling broader industry adoption. In support of this objective, the AMMT program is building and deploying a data management platform to record, index, analyze, and make available the manufacturing data generated across the AMMT program. In FY 2023, the team conceptualized the architecture of the platform and, in FY 2024, deployed the first functional version at the Oak Ridge National Laboratory (ORNL) Manufacturing Demonstration Facility (MDF). In FY 2025, the platform was officially opened to all AMMT members. To enable this expansion, core modifications and enhancements were developed, including improvements to the user interface and workflows for data entry and retrieval. Most notably, robust security and access control mechanisms were implemented to protect data and manage information sharing. This effort featured a logging system, protected views, and controlled access mechanisms. This report documents these enhancements and the transition of the platform into program-wide use.
36 MATERIALS SCIENCE↗
Data Quality Assessment of Optiwatt Vehicle Telematics Data
In October 2024, the Idaho National Laboratory (INL) received data from Optiwatt (Compass Global, Inc.) describing the driving and charging behavior of electric vehicle (EV) drivers. The data shared had been collected from approximately 10,000 vehicles and included vehicle specifications, driving information like odometer readings at the beginning and end of origin-destination pairs (i.e., trips with identification of home for trip start and end for Tesla vehicles), and charging information such as charging energy consumed per charge session and if the charge occurred at home. The vehicle data were provided from 9 EV makes and 18 EV models, with production years ranging from 2012–2024, but more than 9,500 of the vehicles were Tesla EVs. The data includes more than six million trips and more than three million charging events that occurred between June 2023 to Aug 2024 and collected from California and the Eastern United States. The purpose of this report is to review the quality of the data received from Optiwatt and the feedback INL received from Optiwatt after data concerns were shared with them.
33 - ADVANCED PROPULSION SYSTEMS↗
Data for "1H NMR quantification of aromatic monomers from reductive catalytic fractionation"
Data here corresponds to the data presented in the academic publication titled "1H NMR quantification of aromatic monomers from reductive catalytic fractionation" set to be published in ACS Sustainable Chemistry and Engineering. Data was gathered at the National Renewable Energy Laboratory From 2022-2025.
09 BIOMASS FUELS↗
SITCOMTN-161: PSF assessment in the field of Abell 360 and shapeHSM shear profile using LSSTComCam data
The Rubin LSSTComCam on-sky campaign performed at the end of 2024 provided observations of the Abell 360 galaxy cluster; these data allow a preliminary study of cluster weak lensing analysis using Rubin Data Preview 1 (DP1) data. Among all the steps required for such analyses, accurate modeling of the PSF is essential. This work uses several diagnostics, mostly based on the residuals between the second moments of stars and the PSF model, to characterize the accuracy of the PSF modeling in the A360 field. We find the level of the residuals to be sufficiently low not to hinder the measurement of the tangential shear profile around A360. With a simple source selection process, we demonstrate that outputs of the LSST Science Pipelines can be used to detect the tangential shear profile in Abell 360 at the 3.6σ level, and our analysis indicates that contamination from PSF modeling systematics is negligible.
Dell'Antonio, Ian [Brown University]↗
SITCOMTN-165: Surface brightness profiles around massive galaxies in LSSTComCam data
Low surface brightness photometry of massive galaxies can probe the extended stellar halos and intracluster light but is often limited by background subtraction and masking systematics. Here we present radial surface brightness profiles for the brightest cluster galaxy of Abell 360 and 2dFGRS TGS323Z113 using Rubin Observatory Commissioning Camera (LSSTComCam) Data Preview 1 in the g, r, and z bands. Our results show that the LSSTComCam data can go down to ~28 mag arcsec^-2 in the r band. Comparison to DECaLS measurements processed with Legacyhalos shows good agreement of surface brightness profiles at small radii while the LSSTComCam data extend deeper at larger radii.
79 ASTRONOMY AND ASTROPHYSICS↗
Preprocessing of multispectral data and simulation of ERTS data channels to make computer terrain maps of a Yellowstone National Park test site
The possibility of improving the accuracy of terrain classification by preprocessing spectral data was investigated. Terrain maps were made using the following techniques: 1) preprocessing by scan angle function transformation, using the computer-selected best set of three channels; and 2) preprocessing by ratio transformation, using the specified ERTS data channels, simulated by fitting the spectral response of each of the 12 data channels to the ERTS channels by a set of weighting coefficients. By using a simple technique during printout, the maps were produced in color. The normalized scan angle function transformation resulted in the most accurate classification. The best ratio transformation for the Yellowstone Park data was the ratio of each channel to the sum of all channels. A supervised training program involving maximum likelihood decision for selecting the best spectrometer channels and similar techniques for digitizing the data of the analog magnetic tapes were used. Cloud shadows were recognized in addition to eight classes of terrain. Preprocessing of data resulted in more accurate maps, required fewer training areas (hence less preparation and computer time), and enabled much of the area formerly classified as shadow to be reclassified according to actual terrain type.
Smedes, H. W.↗
A real time programmable data compression system for video data.
Description of the implementation of a data compression system for the real-time operational transmission (through microwave links) of ATS satellite pictures between the command and data acquisition station and a central location for computer processing. The system features the use of general-purpose minicomputers for encoding and decoding; this makes it possible to vary the employed data compression technique and to make simultaneous statistical calculations on the data. Data compression and expansion is accomplished in a manner that does not lower data quality.
Kutz, R. L.↗
Retransmission of water resources data using the ERTS-1 data collection system
The Water Survey of Canada operates a network of approximately 2400 gauging stations at which water level data are collected. Nine DCPs were installed in isolated areas of northern and western Canada. It was felt that DCPs in these locations would be exposed to climatic conditions severe enough to provide a check on system reliability. Water level data are transmitted from all sites and, also, some of the DCPs are used to transmit ice break-up, water velocity, precipitation, air temperature, water stage recorder clock operation or DCP battery voltage. Consideration is being given to transmitting other parameters that would be of value in flood or flow forecasting. Results of the project have been excellent. There has been no data loss that can be attributed to failure of a DCP although data were lost because of sensor malfunctions. The quality of data collected compares favourably with that of the hard record obtained at the remote sites. Costs of using the ERTS Data Collection System are reasonable.
Halliday, R. A.↗
Orbiter data reduction complex data processing requirements for the OFT mission evaluation team (level C)
This document addresses requirements for post-test data reduction in support of the Orbital Flight Tests (OFT) mission evaluation team, specifically those which are planned to be implemented in the ODRC (Orbiter Data Reduction Complex). Only those requirements which have been previously baselined by the Data Systems and Analysis Directorate configuration control board are included. This document serves as the control document between Institutional Data Systems Division and the Integration Division for OFT mission evaluation data processing requirements, and shall be the basis for detailed design of ODRC data processing systems.
Source record↗
Pioneer Venus Unified Abstract Data Library and Quick Look Data Delivery System
Development of the Pioneer Venus (PV) Unified Abstract Data System (UADS) and Quick Look Data System (QLDS) was prompted by the need to provide PV investigators rapid and easy access to PV mission data. The UADS is intended to maximize the scientific benefits of the mission by facilitating the exchange of reduced scientific data. QLDS provides a method by which sampled daily mission data is rapidly transmitted to principal investigators providing them a quick look at that orbit's data.
Ferandin, J. A.↗
Application of digital terrain data to quantify and reduce the topographic effect on Landsat data
Landsat multispectral scanner (MSS) data are integrated with 30 m US Geological Survey digital terrain data to quantify and reduce topographic effects on imagery of a forested mountain ridge in central Pennsylvania. Spectral band ratioing slightly decreased variation of Landsat data for 50% of the transects and could not eliminate direct-light topographic effects. A Lambertian model increased topographic effect, and a modified Lambertian model decreased variation produced by the original model, but gave higher variances than found in the raw Landsat data. A non-Lambertian model, however, decreased variation by 86%, reducing topographic effects. The study demonstrated that high quality digital terrain data can be used to develop and test improved radiative transfer models, and enhance the utility of MSS data.
Justice, C. O.↗