Search NASA⌕ Search

SEARCH · Search NASA

Results for “data aggregation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Monthly Quality-filtered Aggregation of NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR)

This dataset contains gridded monthly Leaf Area Index (LAI) derived from the daily NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR). This data record spans from 1981 to 2024 using data from NOAA polar orbiting satellites: NOAA-7, -9, -11, -14, -16, -17, -18, -19 and S-NPP. The data are projected on a 0.05 degree x 0.05 degree global grid, as in the original CDR. The original CDR is one of the Land Surface CDR products produced by the NASA Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), which is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program. This dataset is in the netCDF-4 file format following ACDD and CF Conventions. This dataset has applied quality assurance information to only include "OK" data from the original CDR in the monthly aggregation.

Vermote, Eric [NASA Goddard Space Flight Center (G↗

Accelerating shared file checkpoint with local burst buffers

A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.

Gooding, Thomas↗

Trace elements in the Allende meteorite. IV - Amoeboid olivine aggregates

INAA data for Ca, Sc, Hf, La, Ce, Sm, Eu, Tb, Yb, Lu, Os, Ir, Ru, Na, Cl, Br, Fe, Mn, Cr, Co, Au, As, and Sb are presented for ten amoeboid aggregates from the Allende meteorite. Only one lacks olivine. Seven of the remainder, as a group, have cosmic proportions of refractory lithophile and siderophile elements and appear to have formed when coarse-grained Allende inclusion material underwent partial reaction with a low-temperature nebular gas and mixture with FeO-rich olivine. The other two have highly fractionated abundances of refractory elements relative to one another compared to Cl chondrites, including Group II REE patterns, and probably formed by the mixing of fine-grained Allende inclusion material with FeO-rich olivine. Non-refractory siderophile components are also different in composition in each type of amoeboid olivine aggregate.

Grossman, L.↗

Network Intrusion Detection and Visualization using Aggregations in a Cyber Security Data Warehouse

The challenge of achieving situational understanding is a limiting factor in effective, timely, and adaptive cyber-security analysis. Anomaly detection fills a critical role in network assessment and trend analysis, both of which underlie the establishment of comprehensive situational understanding. To that end, we propose a cyber security data warehouse implemented as a hierarchical graph of aggregations that captures anomalies at multiple scales. Each node of our pro-posed graph is a summarization table of cyber event aggregations, and the edges are aggregation operators. The cyber security data warehouse enables domain experts to quickly traverse a multi-scale aggregation space systematically. We describe the architecture of a test bed system and a summary of results on the IEEE VAST 2012 Cyber Forensics data.

anomaly detection↗

Commercial and Multifamily Building Tenant Energy Usage Aggregation and Privacy

In a number of cities and states, building owners are required to disclose and/or benchmark their building energy use. This requires the building owner to possess monthly whole-building energy usage information, which can be challenging for buildings in which individual tenants have their own utility meters and accounts with the utility. Some utilities and utility regulators have turned to aggregation of customer data as a way to give building owners the whole-building energy usage data while protecting customer privacy. However, no utilities or regulators appear to have conducted a concerted statistical, cybersecurity, and privacy analysis to justify the level of aggregation selected. Therefore, the Tennant Data Aggregation Task was established to help utilities address these issues and provide recommendations as well as a theoretical justification of the aggregation threshold. This study is focused on the use case of submitting data for ENERGY STAR Portfolio Manager (ESPM), but it also looks at other potential use cases for monthly energy consumption data.

data aggergation↗

Global crop forecasting

The needs for and remote sensing means of global crop forecasting are discussed, and key results of the Large Area Crop Inventory Experiment (LACIE) are presented. Current crop production estimates provided by foreign countries are shown often to be inadequate, and the basic elements of crop production forecasts are reviewed. The LACIE project is introduced as a proof-of-concept experiment designed to assimilate remote sensing technology, monitor global wheat production, evaluate key technical problems, modify the technique accordingly and demonstrate the feasibility of a global agricultural monitoring system. The global meteorological data, sampling and aggregation techniques, Landsat data analysis procedures and yield forecast procedures used in the experiment are outlined. Accuracy assessment procedures employed to evaluate LACIE technology performance are presented, and improvements in system efficiency and capacity during the three years of operation are pointed out. Results of LACIE estimates of Soviet, U.S. and Canadian wheat production are presented which demonstrate the feasibility and accuracy of the remote-sensing approach for global food and fiber monitoring.

Macdonald, R. B.↗

PV System Availability from Commercial and Utility-Scale Systems [Slides]

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with worse inverter availability results for larger PV systems. Potential causes of this effect are under investigation.

14 SOLAR ENERGY↗

PV Inverter Availability from the U.S. PV Fleet

In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with better inverter availability results for smaller PV systems. Potential causes of this effect may include the selection of inverter itself: smaller inverters 6kW-250kW showed better average availability than inverters 300kW-5MW. The elimination of string combiner boxes and lower energy impact when one particular inverter goes off-line are potential benefits of a string inverter-based PV system architecture. DNV also analyzed availability data from over 1100 operating systems and found similar trends. DNV's P50 industry guidance on expected availability has been updated to reflect the data and the following observations: utility scale systems have lower availability than DG systems, availability is lower in first year compared to subsequent years, and that actual availability is lower than expected.

fleet↗

Carbon Storage Site Mapping Inquiry Tool (MapIT)

To date, 48 projects, consisting of 139 wells, are currently under review with the Environmental Protection Agency’s (EPA) Underground Injection Control (UIC) Program for Class VI – wells used for geologic sequestration of carbon dioxide. The number of applications submitted is expected to increase in coming years with the increase of the 45Q tax credit available to projects that initiate construction prior to 2033. The amount of data collected to submit a Class VI permit is vast, and often disparate, coming from state, federal, and commercial entities, as well as field-specific data collected within an area of interest. When preparing for site selection and permitting, the initial aggregation of relevant public data can be time intensive. The Carbon Storage Site Mapping Inquiry tool (MapIT) was created to support and accelerate the discovery and accessibility of open-source data and information available across the USA. Data was aggregated and organized based on data types described within the EPA UIC Class VI permit documentation. The online tool enables users to explore hundreds of geospatial data layers and connect to additional external resources, leveraging API and REST services where possible to ensure updates to data in real time. MapIT enables users to explore state and federal data related to geologic, geophysical, structural, hydrologic, and contextual information. In addition to displaying spatial data and linking to external resources, MapIT leverages custom widgets to ensure that internal data and external data are discoverable and accessible. The widgets connect users to resources such as the USGS publications and the USGS Earthquake Catalog based on a user-defined location. This talk will describe data aggregation workflows, data types, data preparation, and tool development for MapIT. The Carbon Storage Site Mapping Inquiry Tool and underlying database are valuable, intuitive resources that empower government, academic, commercial and industry stakeholders to explore, analyze, and acquire carbon storage related data.

Morkner, Paige↗

A Generalized approach to the operationalization of Software Quality Models

Comprehensive measures of quality are a research imperative, yet the development of software quality models is a wicked problem. Definitive solutions do not exist and quality is subjective at its most abstract. Definitional measures of quality are contingent on a domain, and even within a domain, the choice of representative characteristics to decompose quality is subjective. Thus, the operationalization of quality models brings even more challenges. A promising approach to quality modeling is the use of hierarchies to represent characteristics, where lower levels of the hierarchy represent concepts closer to real-world observations. Building upon prior hierarchical modeling approaches, we developed the Platform for Investigative software Quality Understanding and Evaluation (PIQUE). PIQUE surmounts several quality modeling challenges because it allows modelers to instantiate abstract hierarchical models in any domain by leveraging organizational tools tailored to their specific contexts. Here, we introduce PIQUE; exemplify its utility with two practical use cases; address challenges associated with parameterizing a PIQUE model; and describe algorithmic techniques that tackle normalization, aggregation, and interpolation of measurements.

Data aggregation↗

Storages Are Not Forever

Not unlike the concern over diminishing fossil fuel, information technology is bringing its own share of future worries. Here, we chose to look closely into one concern in this paper, namely the limited amount of data storage. By a simple extrapolatory analysis, it is shown that we are on the way to exhaust our storage capacity in less than two centuries with current technology and no recycling. This can be taken as a note of caution to expand research initiative in several directions: firstly, bringing forth innovative data analysis techniques to represent, learn, and aggregate useful knowledge while filtering out noise from data; secondly, tap onto the interplay between storage and computing to minimize storage allocation; thirdly, explore ingenious solutions to expand storage capacity. Throughout this paper, we delve deeper into the state-of-the-art research and also put forth novel propositions in all of the abovementioned directions, including space- and time-efficient data representation, intelligent data aggregation, in-memory computing, extra-terrestrial storage, and data curation. The main aim of this paper is to raise awareness on the storage limitation we are about to face if current technology is adopted and the storage utilization growth rate persists. In the manuscript, we propose some storage solutions and a better utilization of storage capacity through a global DIKW hierarchy.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Analyzing and visualizing next generation climate data (Final Report)

The project "Analyzing and visualizing next generation climate data" adds block-structured (mosaic) grid support, parallel processing, and 2D/3D curvilinear interpolation to the open-source UV-CDAT climate data analysis tool. Block structured grid support complies to the Gridspec extension submitted to the Climate and Forecast metadata conventions. It contains two parts: aggregation of data spread over multiple mosaic tiles (M-SPEC) and aggregation of temporal data stored in different files (F-SPEC). Together, M-SPEC and F-SPEC allow users to interact with data stored in multiple files as if the data were in a single file. For computational expensive tasks, a flexible, multi-dimensional, multi-type distributed array class allows users to process data in parallel using remote memory access. Both nodal and cell based interpolation is supported; users can choose between different interpolation libraries including ESMF and LibCF depending on the their particular needs.

54 ENVIRONMENTAL SCIENCES↗

SU-E-T-186: Cloud-Based Quality Assurance Application for Linear Accelerator Commissioning

Purpose: To identify anomalies and safety issues during data collection and modeling for treatment planning systems Methods: A cloud-based quality assurance system (AQUIRE - Automated QUalIty REassurance) has been developed to allow the uploading and analysis of beam data aquired during the treatment planning system commissioning process. In addition to comparing and aggregating measured data, tools have also been developed to extract dose from the treatment planning system for end-to-end testing. A gamma index is perfomed on the data to give a dose difference and distance-to-agreement for validation that a beam model is generating plans consistent with the beam data collection. Results: Over 20 linear accelerators have been commissioning using this platform, and a variety of errors and potential saftey issues have been caught through the validation process. For example, the gamma index of 2% dose, 2mm DTA is quite sufficient to see curves not corrected for effective point of measurement. Also, data imported into the database is analyzed against an aggregate of similar linear accelerators to show data points that are outliers. The resulting curves in the database exhibit a very small standard deviation and imply that a preconfigured beam model based on aggregated linear accelerators will be sufficient in most cases. Conclusion: With the use of this new platform for beam data commissioning, errors in beam data collection and treatment planning system modeling are greatly reduced. With the reduction in errors during acquisition, the resulting beam models are quite similar, suggesting that a common beam model may be possible in the future. Development is ongoing to create routine quality assurance tools to compare back to the beam data acquired during commissioning. I am a medical physicist for Alzyen Medical Physics, and perform commissioning services.

60 APPLIED LIFE SCIENCES↗

Medium Duty ARRA Data Reporting and Analysis (Presentation)

This project compiles medium-duty (MD) aggregated deployment data and provides the compiled detailed analyses to industry. The U.S. Department of Energy's (DOE's) American Recovery and Reinvestment Act (ARRA) deployment and demonstration projects are helping to commercialize technologies for all-electric vehicles, electrified accessories, and electric charging infrastructure. Over 3.2 million miles of in-service all-electric MD truck data from 560 different vehicles have been collected since 2011, and usage data from over 1,000 truck electrification sites have been collected since 2013. Through the DOE's Vehicle Technologies Office, NREL is working to analyze real-time data from these deployment and demonstration projects to quantify the benefits: results and summary statistics are made available through the NREL website as quarterly and annual reports; 23 aggregated reports have been published on the performance and operation of these vehicles; and detailed data are being extracted to help further understand battery use and performance.

2014 AMR↗

NREL's Clean Energy Policy Analyses Project: 2009 U.S. State Clean Energy Data Book, October 2010

This data book provides a summary of the status of state-level energy efficiency and renewable energy (taken together as clean energy) developments and supporting policy implementation. It is intended as a reference book for those interested in the progress of the states and regions toward a clean energy economy. Although some national-scale data are given in the initial section, the data aremostly aggregated by states and region, and no data on federal- or utility-level policies are presented here.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

TU-C-BRE-03: Aggregation of Linac Measurement Data

Purpose: Accurate data of linear accelerator radiation characteristics is important for treatment planning system commissioning as well as regular quality assurance of the machine. The RPC has performed site visits of numerous machines . Data gathered from Varian machines from the past 15 years are presented. The data collected can be used as a secondary check or when commissioning a new machine to verify that values are reasonable. Methods: Data from the past 15 years of RPC site visits was compiled and analyzed. Data was composed from measurements from approximately 400 Varian machines. Each dataset consists of several point measurements at various locations in a water phantom to measure percentage depth dose, output factors, including small MLC fields, off-axis factors, and wedge factors if applicable. Common statistical values are presented for each machine type. Where applicable, data was compared to other reference data given by the vendor or a select number of previous researchers. Results: Data is separated by energy and parameter and then analyzed by machine class. Data distributions of the parameter data were normal except occasionally at the tails. Distributions of the data for each class and parameter are tabulated to give not simply a singular reference value, but metrics about the distribution: 5th and 95th percentile values and the standard deviation as well as the median. Conclusion: The RPC has collected numerous data on Varian linacs and presented the finding of the past 15 years. The data can be used as a reference data set for physicists to compare against. A linac that deviates from the values does not necessarily indicate there is a problem as long as the treatment planning system correlates to the machine. Comparison of linac and treatment planning system data to external reference data can prevent serious treatment errors.

07 ISOTOPE AND RADIATION SOURCES↗

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Regional rainfall climatologies derived from Special Sensor Microwave Imager (SSM/I) data

Climatologies of convective precipitation were derived from passive microwave observations from the Special Sensor Microwave Imager using a scattering-based algorithm of Adler et al. Data were aggregated over periods of 3-5 months using data from 4 to 5 years. Data were also stratified by satellite overpass times (primarily 06 00 and 18 00 local time). Four regions (Mexico, Amazonia, western Africa, and the western equatorial Pacific Ocean (TOGA COARE area) were chosen for their meteorological interest and relative paucity of conventional observations. The strong diurnal variation over Mexico and the southern United States was the most striking aspect of the climatologies. Pronounced morning maxima occured offshore, often in concativities in the coastline, the result of the increased convergence caused by the coastline shape. The major feature of the evening rain field was a linear-shaped maximum along the western slope of the Sierra Madre Occidental. Topography exerted a strong control on the rainfall in other areas, particularly near the Nicaragua/Honduras border and in Guatemala, where maxima in excess of 700 mm/month were located adjacent to local maxima in terrain. The correlation between the estimates and monthly gage data over the southern United States was low (0.45), due mainly to poor temporal sampling in any month and an inadequate sampling of the diurnal cycle. Over the Amazon Basin the differences in morning versus evening rainfall were complex, with an alternating series of morning/evening maxima aligned southwest to northeast from the Andes to the northeast Brazilian coast. A real extent of rainfall in Amazonia was slightly higher in the evening, but a maximum in morning precipitation was found on the Amazon River just east of Manaus. Precipitation over the water in the intertropical convergence zone (ITCZ) north of Brazil was more pronounced in the morning, and a pronounced land-/sea-breeze circulation was found along the northeast coast of Brazil. Inter-comparison of four years revealed 1992 to be the driest over Amazonia, with about a 23% decrease in mean rate compared to the 4-year mean estimated rain rate.

Negri, Andrew J.↗