Search NASA⌕ Search

SEARCH · Search NASA

Results for “data sources”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Machine Learning Modeling Pipeline for Extracting Nuclear Proliferation Events of Interest from Open Data Sources (U)

In FY2020, the Savannah River National Laboratory (SRNL) and the Sanghani Center for Artificial Intelligence and Data Analytics at Virginia Polytechnic Institute and State University entered a collaboration funded by Department of Energy’s (DOE) Office of Defense Nuclear Nonproliferation Research and Development. The project’s mission was to take the first steps toward developing a demonstration prototype system that uses multiple machine learning and data analytics methods on largescale open data sources to identify new, developing, and/or undeclared nuclear programs. Given the SRNL team’s on-site perspective of events culminating in the DOE’s decision to pursue the Savannah River Plutonium Processing Facility (SRPPF), the team targeted the identification of events and indicators in retrospective datasets that pointed to the activity of “fissile core fabrication at the Savannah River Site” prior to the official announcement in May of 2018. A preliminary modeling pipeline was developed in FY20 that showed the datasets contained adequate signal for continuation of efforts. In FY21, a modular demonstration prototype modeling pipeline has continued in development for two text-based data sources: a broad internet archive (Webhose Ltd.) and a decahose Twitter database (i.e., a global sampling of one in every ten Tweets). The techniques that have been developed rely on graph theory and anomaly detection to identify contextual shifts in key words and phrases at various points in time such that indicators of events of interest could be identified and subsequently, events could be extracted from the corpuses. The foundational concept behind the approaches is that contextual shifts in key words and phrases can act as indicators of events of interest. Both datasets have proven successful in extracting events of interest related to pit production at the Savannah River Site prior to the official announcement. In addition, the pipelines have generated a wide range of events broadly summarized as: the awarding of DOE contracts at major sites, DOE investments in various programs, accidents at DOE national laboratories, speculations about the fate of pit production in the DOE complex, domestic and international shipments and receipts of nuclear materials at DOE sites, termination of non-proliferation agreements with Russia, termination of MOX, new weapons development approvals/testing, nuclear posture reviews, major DOE cleanup/production milestones, political opinions, and nuclear watch groups’ opinions, among many others.

97 MATHEMATICS AND COMPUTING↗

ASCR Workshop on In Situ Data Management: Enabling Scientific Discovery from Diverse Data Sources

In January 2019, the U.S. Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions for in situ data management (ISDM). The workshop defined ISDM as the practices, capabilities, and procedures to control the organization of data and enable the coordination and communication among heterogeneous tasks, executing simultaneously in a high-performance computing system, cooperating toward a common objective. The workshop revealed two primary, interdependent motivations for processing and managing data in situ. The first motivation is that the in situ methodology enables scientific discovery from a broad range of data sources over a wide scale of computing platforms: leadership-class systems, clusters, clouds, workstations, and embedded devices at the edge. The successful development of ISDM capabilities will benefit real-time decision-making, design optimization, and data-driven scientific discovery. The second motivation is the need to decrease data volumes. ISDM can make critical contributions to managing large data volumes from computations and experiments to minimize data movement, save storage space, and boost resource efficiency, often while simultaneously increasing scientific precision. A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery across the Office of Science mission space. Applications wanting to use the in situ capabilities include those where data analysis feeds back to the simulation, decisions are made autonomously, big data or machine learning is among the tasks to be coordinated, and computations need to be completed in real time. The workshop identified six priority research directions that highlight the components and capabilities needed for ISDM to be successful for the wide variety of applications discussed: making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗

EXTRAGALACTIC MILLIMETER-WAVE SOURCES IN SOUTH POLE TELESCOPE SURVEY DATA: SOURCE COUNTS, CATALOG, AND STATISTICS FOR AN 87 SQUARE-DEGREE FIELD

We report the results of an 87 deg{sup 2} point-source survey centered at R.A. 5{sup h}30{sup m}, decl. -55{sup 0} taken with the South Pole Telescope at 1.4 and 2.0 mm wavelengths with arcminute resolution and milli-Jansky depth. Based on the ratio of flux in the two bands, we separate the detected sources into two populations, one consistent with synchrotron emission from active galactic nuclei and the other consistent with thermal emission from dust. We present source counts for each population from 11 to 640 mJy at 1.4 mm and from 4.4 to 800 mJy at 2.0 mm. The 2.0 mm counts are dominated by synchrotron-dominated sources across our reported flux range; the 1.4 mm counts are dominated by synchrotron-dominated sources above {approx}15 mJy and by dust-dominated sources below that flux level. We detect 141 synchrotron-dominated sources and 47 dust-dominated sources at signal-to-noise ratio S/N >4.5 in at least one band. All of the most significantly detected members of the synchrotron-dominated population are associated with sources in previously published radio catalogs. Some of the dust-dominated sources are associated with nearby (z << 1) galaxies whose dust emission is also detected by the Infrared Astronomy Satellite. However, most of the bright, dust-dominated sources have no counterparts in any existing catalogs. We argue that these sources represent the rarest and brightest members of the population commonly referred to as submillimeter galaxies (SMGs). Because these sources are selected at longer wavelengths than in typical SMG surveys, they are expected to have a higher mean redshift distribution and may provide a new window on galaxy formation in the early universe.

79 ASTRONOMY AND ASTROPHYSICS↗

Data source authentication of synchrophasor measurement devices based on 1D-CNN and GRU

Synchrophasor measurement devices (SMDs) have been widely deployed to support real-time monitoring and control of power systems. In the meantime, data spoofing has emerged in recent years. Therefore, it is of great importance to study data authentication algorithms for detecting and defending the data spoofing effectively. Here, a one-dimensional convolutional neural network (1D-CNN) is utilized to extract temporal signatures hidden in frequency, voltage angle and amplitude data; then the gated recurrent unit (GRU) employs these temporal signatures for data source authentication. In case studies, the performances of different algorithms are tested in large-scale power systems with numerous SMDs for the first time, and comparisons among different algorithms show that the proposed algorithm can achieve a higher accuracy of data source authentication with a shorter time window.

47 OTHER INSTRUMENTATION↗

Landscape analysis of environmental data sources for linkage with SEER cancer patients database

Abstract One of the challenges associated with understanding environmental impacts on cancer risk and outcomes is estimating potential exposures of individuals diagnosed with cancer to adverse environmental conditions over the life course. Historically, this has been partly due to the lack of reliable measures of cancer patients’ potential environmental exposures before a cancer diagnosis. The emerging sources of cancer-related spatiotemporal environmental data and residential history information, coupled with novel technologies for data extraction and linkage, present an opportunity to integrate these data into the existing cancer surveillance data infrastructure, thereby facilitating more comprehensive assessment of cancer risk and outcomes. In this paper, we performed a landscape analysis of the available environmental data sources that could be linked to historical residential address information of cancer patients’ records collected by the National Cancer Institute’s Surveillance, Epidemiology, and End Results Program. The objective is to enable researchers to use these data to assess potential exposures at the time of cancer initiation through the time of diagnosis and even after diagnosis. The paper addresses the challenges associated with data collection and completeness at various spatial and temporal scales, as well as opportunities and directions for future research.

60 APPLIED LIFE SCIENCES↗

Risk-Aware Measurement Synchronization and Recovery for DSSE With Heterogeneous Data Sources

Power distribution systems are increasingly integrating heterogeneous sensors with varying data reporting rates and types, which pose challenges to achieving observability at the desired temporal resolution of distribution system state estimation (DSSE). Multisensor failures caused by extreme events exacerbate these issues, introducing substantial uncertainties into DSSE. This article proposes a novel solution to these challenges by ensuring high-resolution system observability despite heterogeneous data sources and multisensor failures. First, a deep learning architecture combining long short-term memory (LSTM) and graph convolutional network (GCN) is employed to synchronize meters with different reporting rates, aiming to achieve system observability. A random-walk-model-based approach is introduced to generate pseudo-measurements while properly characterizing their uncertainties under multisensor failures. Finally, a disaster-risk-informed observability metric (RiOM) is defined to quantify the uncertainty associated with state estimation results. The proposed framework offers deeper insights into the system observability on the fly compared with conventional analysis. The effectiveness of the framework is demonstrated on an IEEE standard test case and a large-scale real-world distribution feeder in mid-Minnesota in the U.S.

97 MATHEMATICS AND COMPUTING↗

The Freight Analysis Framework Verson 4 (FAF4) - Building the FAF4 Regional Database: Data Sources and Estimation Methodologies

The Freight Analysis Framework (FAF) integrates data from a variety of sources to create a comprehensive national picture of freight movements among states and major metropolitan areas by all modes of transportation. It provides a national picture of current freight flows to, from, and within the United States, assigns the flows to the transportation network, and projects freight flow patterns into the future. The FAF4 is the fourth database of its kind, FAF1 provided estimates for truck, rail, and water tonnage for calendar year 1998, FAF2 provided a more complete picture based on the 2002 Commodity Flow Survey (CFS) and FAF3 made further improvements building on the 2007 CFS. Since the first FAF effort, a number of changes in both data sources and products have taken place. The FAF4 flow matrix described in this report is used as the base-year data to forecast future freight activities, projecting shipment weights and values from year 2020 to 2045 in five-year intervals. It also provides the basis for annual estimates to the FAF4 flow matrix, aiming to provide users with the timeliest data. Furthermore, FAF4 truck freight is routed on the national highway network to produce the FAF4 network database and flow assignments for truck. This report details the data sources and methodologies applied to develop the base year 2012 FAF4 database. An overview of the FAF4 components is briefly discussed in Section 2. Effects on FAF4 from the changes in the 2012 CFS are highlighted in Section 3. Section 4 provides a general discussion on the process used in filling data gaps within the domestic CFS matrix, specifically on the estimation of CFS suppressed/unpublished cells. Over a dozen CFS OOS components of FAF4 are then addressed in Section 5 through Section 11 of this report. This includes discussions of farm-based agricultural shipments in Section 5, shipments from fishery and logging sectors in Section 6. Shipments of municipal solid wastes and debris from construction and demolition activities are covered in Section 7. Movements involving OOS industry sectors on Retail, Services, and Household/Business Moves are addressed in Section 8. Flows of OOS commodity on crude petroleum and natural gas are presented in Sections 9 and 10, respectively. Discussions regarding shipments of foreign trade, including trade with Canada/Mexico, international airfreight, and waterborne foreign trade, are then discussed in Section 11. Several appendices are also provided at the end of this report to offer additional information.

97 MATHEMATICS AND COMPUTING↗

A Review of Global Precipitation Data Sets: Data Sources, Estimation, and Intercomparisons

Abstract In this paper, we present a comprehensive review of the data sources and estimation methods of 30 currently available global precipitation data sets, including gauge‐based, satellite‐related, and reanalysis data sets. We analyzed the discrepancies between the data sets from daily to annual timescales and found large differences in both the magnitude and the variability of precipitation estimates. The magnitude of annual precipitation estimates over global land deviated by as much as 300 mm/yr among the products. Reanalysis data sets had a larger degree of variability than the other types of data sets. The degree of variability in precipitation estimates also varied by region. Large differences in annual and seasonal estimates were found in tropical oceans, complex mountain areas, northern Africa, and some high‐latitude regions. Overall, the variability associated with extreme precipitation estimates was slightly greater at lower latitudes than at higher latitudes. The reliability of precipitation data sets is mainly limited by the number and spatial coverage of surface stations, the satellite algorithms, and the data assimilation models. The inconsistencies described limit the capability of the products for climate monitoring, attribution, and model validation.

54 ENVIRONMENTAL SCIENCES↗

The Open-source Data Inventory for Anthropogenic CO 2 , version 2016 (ODIAC2016): a global monthly fossil fuel CO 2 gridded emissions data product for tracer transport simulations and surface flux inversions

The Open-source Data Inventory for Anthropogenic CO 2 (ODIAC) is a global high-spatial-resolution gridded emissions data product that distributes carbon dioxide (CO 2 ) emissions from fossil fuel combustion. The emissions spatial distributions are estimated at a 1 × 1 km spatial resolution over land using power plant profiles (emissions intensity and geographical location) and satellite-observed nighttime lights. This work describes the year 2016 version of the ODIAC emissions data product (ODIAC2016) and presents analyses that help guide data users, especially for atmospheric CO 2 tracer transport simulations and flux inversion analysis. Since the original publication in 2011, we have made modifications to our emissions modeling framework in order to deliver a comprehensive global gridded emissions data product. Major changes from the 2011 publication are (1) the use of emissions estimates made by the Carbon Dioxide Information Analysis Center (CDIAC) at the Oak Ridge National Laboratory (ORNL) by fuel type (solid, liquid, gas, cement manufacturing, gas flaring, and international aviation and marine bunkers); (2) the use of multiple spatial emissions proxies by fuel type such as (a) nighttime light data specific to gas flaring and (b) ship/aircraft fleet tracks; and (3) the inclusion of emissions temporal variations. Using global fuel consumption data, we extrapolated the CDIAC emissions estimates for the recent years and produced the ODIAC2016 emissions data product that covers 2000–2015. Our emissions data can be viewed as an extended version of CDIAC gridded emissions data product, which should allow data users to impose global fossil fuel emissions in a more comprehensive manner than the original CDIAC product. Our new emissions modeling framework allows us to produce future versions of the ODIAC emissions data product with a timely update. Such capability has become more significant given the CDIAC/ORNL's shutdown. The ODIAC data product could play an important role in supporting carbon cycle science, especially modeling studies with space-based CO 2 data collected in near real time by ongoing carbon observing missions such as the Japanese Greenhouse gases Observing SATellite (GOSAT), NASA's Orbiting Carbon Observatory-2 (OCO-2), and upcoming future missions. The ODIAC emissions data product including the latest version of the ODIAC emissions data (ODIAC2017, 2000–2016) is distributed from http://db.cger.nies.go.jp/dataset/ODIAC/ with a DOI (https://doi.org/10.17595/20170411.001).

54 ENVIRONMENTAL SCIENCES↗

Residential Electricity Prices: A Review of Data Sources and Estimation Methods

This paper presents an empirical study of residential electricity prices based on a compilation of tariff data collected for Lawrence Berkeley Lab's Tariff Analysis Project (TAP). We also review other data sources commonly used to estimate residential sector electricity prices, including utility data published by the Energy Information Administration (EIA), the Typical Bills reports published by the Edison Electric Institute, and household-level electricity consumption and expenditure data included with the EIA's Residential Energy Consumption Surveys. We define three different types of price that can be estimated from the data, construct estimation methods, compare the results and evaluate the relative strengths and weaknesses of each approach. We examine several sources of variation including seasonal, regional, industry structure, and the magnitude of baseline household consumption. The tariff data are also used to explore the impact of moving households from standard tariffs to time-of-use pricing, and to summarize how this affects predictions of electricity bill savings for both demand reduction and demand shifting. (auth)

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Asteroid models from photometry and complementary data sources

I discuss inversion methods for asteroid shape and spin reconstruction with photometry (lightcurves) and complementary data sources such as adaptive optics or other images, occultation timings, interferometry, and range-Doppler radar data. These are essentially different sampling modes (generalized projections) of plane-of-sky images. An important concept in this approach is the optimal weighting of the various data modes. The maximum compatibility estimate, a multi-modal generalization of the maximum likelihood estimate, can be used for this purpose. I discuss the fundamental properties of lightcurve inversion by examining the two-dimensional case that, though not usable in our three-dimensional world, is simple to analyze, and it shares essentially the same uniqueness and stability properties as the 3-D case. After this, I review the main aspects of 3-D shape representations, lightcurve inversion, and the inclusion of complementary data.

79 ASTRONOMY AND ASTROPHYSICS↗

Surface albedo and reflectance: Review of definitions, angular and spectral effects, and intercomparison of major data sources in support of advanced solar irradiance modeling over the Americas

This study defines albedo and reflectance in the context of solar applications. It examines the main sources of surface albedo data that can be used to help solar irradiance modeling efforts, such as undertaken by the National Renewable Energy Laboratory (NREL) over the Americas, for instance, per the National Solar Radiation Database (NSRDB). The MODIS-based albedo database, as used as input to derive historical time series of solar irradiance in the current version of the NSRDB, is compared to other sources of global gridded data over the Americas in general and some specific areas in particular. The particular role of snow cover is analyzed. It is found that, in the current MODIS-based NSRDB implementation, snow-impacted areas have been assigned a too high albedo in general because of the lack of consideration for the antagonistic effect of surface roughness, among other factors. Other albedo databases, such as CMSAF, MINES-ParisTech or the ERA5 reanalysis, do not display such high albedos at latitudes above ~40 degrees N. The gap-free and snow-free MODIS MCD43GF product is found affected by a long-term downward trend over some areas and out-of-phase seasonal effects over other areas, in comparison with MERRA-2 reanalysis data in particular. Comparisons with actual albedo measurements at the arid Desert Rock site in Nevada also show that albedo data derived from land surface models are affected by systematic bias. Five albedo databases are intercompared for the area around Ivanpah Playa, California, which has great importance in both remote sensing and solar applications. Various aspects of the proper usage of albedo data in solar radiation modeling are discussed, regarding calculations of both the backscattering process and reflections on a tilted surface: spatial resolution, temporal resolution, angular effects, and spectral effects. An important distinction is made between the use of albedo data at ~4 km resolution for backscattering calculations and at ~10 m resolution for transposition calculations to tilted planes, which might additionally require spectral information for monofacial or bifacial PV applications. It is shown that the tilted calculations usually performed in solar resource assessments are far from realistic in the case of most large-scale PV applications, for which a more suitable methodological approach is described. For such needs, the limitations of the current sources of high-resolution reflectance data are discussed. The impact of surface albedo estimation errors on the estimation of backscattered irradiance over snowy areas is evaluated. It is found non-negligible, and even large under cloudy conditions.

14 SOLAR ENERGY↗

Tsdat: An Open-Source Data Standardization Framework for Marine Energy and Beyond: Preprint

Many organizations are tasked with the collection and processing of large quantities of data from various measurement devices. Data reported from these sources are often not interoperable with datasets and software used by analysts and other organizations in the same field, introducing barriers for collaboration on large-scale projects. This poses a particular problem for cross-device comparisons and machine learning applications. To address these challenges, the open source Time-Series Data Pipelines (Tsdat) software was developed by a joint collaboration between Pacific Northwest National Laboratory, the National Renewable Energy Laboratory, and Sandia National Laboratories to facilitate collaboration and accelerate advancements in the Marine Energy domain through the development of an open-source ecosystem of tools. This paper will describe the Tsdat software and the data standards within which the framework operates. A beta version of the framework has been released and is currently being used by several projects in marine energy, wind energy, and building energy systems.

big data↗

Machine Learning Using Open Data Sources for Detection of Nuclear Proliferation Activities (U)

In FY2020, Savannah River National Laboratory (SRNL) in collaboration with the Sanghani Center for Artificial Intelligence and Data Analytics (SCAIDA) at Virginia Polytechnic Institute and State University (VT) and funded by the Department of Energy’s (DOE) Defense Nuclear Nonproliferation Research and Development, began developing a demonstration prototype system that uses multiple machine learning and data analytic methods on large-scale open data sources to identify new, developing, and/or undeclared nuclear programs. Using the announcement in May 2018 of the proposed Savannah River Plutonium Processing Facility (SRPPF) as a test subject, the goal of this 2-year project is to forecast the SRPPF using only data prior to May 2018. The project work is split into a preliminary prototype development for the first year with an initial evaluation of viability followed by the second year of development to create an integrated prototype system and more extensive performance evaluation. This report documents the results of the preliminary-phase tasks.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Priority research directions for in situ data management: Enabling scientific discovery from diverse data sources

In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms.

97 MATHEMATICS AND COMPUTING↗

Estimating soybean yields from high-temporal-resolution multi-source data using deep learning

Accurate and timely crop yield prediction is crucial for ensuring food security and maintaining stable agricultural markets. In recent years, there has been a surge in interest in leveraging high-temporal-resolution, multi-source data for effective crop growth monitoring and yield estimation. A notable challenge arises from the difficulty in capturing the intricate interactions between variables across different time steps within these high-temporal-resolution time series datasets. This complexity hinders the reliable extraction of yield information from voluminous and often noisy datasets, especially during periods of extreme weather events. Here, in this study, we propose an Attention and Graph Isomorphism Network-enhanced Bi-directional Long Short-Term Memory network (AGB-LSTM) for estimating county-level soybean yield in the United States. This model integrates a diverse set of remote sensing data, including Near-Infrared Reflectance of Vegetation (NIRv), Sun-Induced chlorophyll Fluorescence (SIF), and Gross Primary Productivity (GPP), along with environmental covariates. The AGB-LSTM effectively leverages information related to crop yield from high-temporal-resolution time series data (5-days), achieving an accuracy of R²= 0.67 and rRMSE = 14.46%. This approach significantly outperforms traditional machine learning methods such as Random Forest (RF) (R²= 0.52, rRMSE = 17.36%) and Bi-LSTM (R²= 0.58, rRMSE = 16.17%). Sensitivity experiments with different time steps and ranges demonstrated that our model could accurately and stably predict yields 1 to 2 months before harvest. Moreover, data with a finer temporal resolution consistently improved prediction performance, resulting in an approximately 20% increase in and an approximately 20% decrease in rRMSE compared to using monthly composites. We also evaluated the robustness of the model under extreme climate events and observed strong performance (R²= 0.50, rRMSE = 21.32%). Finally, yield mapping for major soybean-producing regions in North America in 2023 revealed spatial patterns that closely matched USDA yield reports. Our findings suggest that the AGB-LSTM model is a promising and effective method for estimating yield and has notable potential for global crop yield forecasting.

Deep learning↗