Using Flux By Cloud Type Data to Study Convective Aggregation in the Tropics
Explore the source record for details and available documents.
Convective aggregation↗
SEARCH · Search NASA
Results for “Data aggregation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Information Management Platform for Data Analytics and Aggregation (IMPALA) System Design Document
The System Design document tracks the design activities that are performed to guide the integration, installation, verification, and acceptance testing of the IMPALA Platform. The inputs to the design document are derived from the activities recorded in Tasks 1 through 6 of the Statement of Work (SOW), with the proposed technical solution being the completion of Phase 1-A. With the documentation of the architecture of the IMPALA Platform and the installation steps taken, the SDD will be a living document, capturing the details about capability enhancements and system improvements to the IMPALA Platform to provide users in development of accurate and precise analytical models. The IMPALA Platform infrastructure team, data architecture team, system integration team, security management team, project manager, NASA data scientists and users are the intended audience of this document. The IMPALA Platform is an assembly of commercial-off-the-shelf (COTS) products installed on an Apache-Hadoop platform. User interface details for the COTS products will be sourced from the COTS tools vendor documentation. The SDD is a focused explanation of the inputs, design steps, and projected outcomes of every design activity for the IMPALA Platform through installation and validation.
Carnell, Andrew↗
Observed Cloud Type‐Sorted Cloud Property and Radiative Flux Changes With the Degree of Convective Aggregation From CERES Data
Explore the source record for details and available documents.
Kuan‐Man Xu↗
BAMCensus (The Behavior and Advanced Mobility Census Dataset Aggregator) [SWR-25-120]
This software is a high-performance tool developed in Rust for downloading and processing large-scale geospatial datasets, specifically focusing on US Census data. It is designed to address scaling limitations found in existing tools, such as R's [tidycensus](https://walker-data.com/tidycensus/), by providing performant streaming dataset JOIN operations between various US Census datasets (like ACS and LEHD) and their corresponding geometries stored on the TIGER/Lines web server. The tool automates the process of joining these data sources, returning aggregated data to the user based on a specified census GEOID type. The tool automates the process of joining these data sources, returning aggregated data to the user based on a specified census GEOID type. Its primary motivation stems from the need for a high-performance solution to combine spatial datasets with graph traversals within the context of mobility analysis tooling being developed at NREL's Behavior and Advanced Mobility (BAM) group.
Fitzgerald, Robert [National Renewable Energy Labo↗
Identifying Trends in Landing Profiles After Long Duration Spaceflight
BACKGROUND As NASA’s mission programs begin to shift from Low Earth Orbit (LEO) to planetary mission profiles, the importance of understanding the phenomena a human experience when returning from spaceflight is of increased importance. The purpose of this study was to assess trends across individuals in the symptoms experienced during the landing process of long duration ISS missions. METHODS Data for this study is an aggregation of mission level data from forms used by crew surgeons at R+0, electronic medical records, notes from weekly Space Medicine Operation Team meetings, records in the Shuttle Mission Data Archive, and data collected by research teams in the Human Research Program (HRP). Aggregated data was used in descriptive analysis of the trends in medical events and countermeasure use. The data was also used to identify high value targets for emphasis in on going surveillance efforts. RESULTS Completeness of reporting varied greatly over time and across missions dependent on information sources available for data extraction. Landing forms provided the most complete data but still missed or left unconfirmed, several, common symptoms, and countermeasures. Data aggregation did show to cover gaps within individual data sources. Commonly reported symptoms included nystagmus, vertigo, and nausea. Common protocols for fluid loading and meclizine are reflected within the data set. DISCUSSION The data set generated in this project provides a foundation that show how symptoms of astronauts as they return to Earth varies across the long duration ISS missions. While there is an abundance of information provided within the data set, there are still inconsistencies in reporting especially in the absence of the landing day form. Further data collection practices should be standardized to ease the burden on crew surgeons during R+0 exams and better collect high value variables.
Sam Jacobs↗
Aggregated carbon dioxide flux and hydrometeorology data from an Amazonian palm swamp peatland in Peru: 2018, 2019, and 2022
This dataset contains eddy covariance carbon dioxide flux and hydrometeorological measurements made in an Amazonian palm swamp peatland near Iquitos, Peru. These data have been aggregated from half-hourly observations that are available from AmeriFlux (https://ameriflux.lbl.gov/; site PE-QFR). These data files are CSV (comma separated values) format and can be imported using Matlab, R, or Excel. Three full years of data are reported (2018, 2019 and 2022) during which time there were large differences in annual net ecosystem carbon dioxide exchange. The gap was caused by instrument malfunction and extended delays in repairs because of the Covid-19 pandemic. This research was conducted to better understand the carbon cycle of tropical peatlands, and was supported by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research, Terrestrial Ecosystem Science Program, under Award Number DE-SC0020167.
54 ENVIRONMENTAL SCIENCES↗
Real-Time Lifetime Prediction of Semiconductor Devices Using Hardware-in-the-Loop
This paper presents a unique approach to enable real-time lifespan prediction of semiconductor power modules using a Hardware-in-the-Loop (HIL) system. By integrating the module's overall loss characteristics-specifically switching and conduction losses-with a thermoelectric model of the thermal management system, this research demonstrates that the model can dynamically estimates the junction temperature profile of the semiconductor devices in response to a changing torque demand profile for the motor drive system. This capability enables continuous monitoring of the module's operational time and cumulative stress induced on the devices to compute accumulated remaining lifetime or time-to-failure (TTF). This study provides an architectural framework for the HIL system with high-fidelity component models of multiple physical domains, allowing simulation of dynamic behaviors of a closely-coupled motor drive system. The advanced real-time computation and measurement functionalities of the HIL system allow for both dynamic lifetime calculations based on simulated data and aggregate lifetime predictions utilizing historical data. Moreover, this paper details an algorithm that not only computes cumulative damage but also synthesizes these data into a comprehensive aggregated lifetime metric. This methodology can enhance the maintenance scheduling strategies and operational reliability of semiconductor devices in critical applications, ultimately extending their service life while optimizing performance.
hardware-in-the-loop (HIL)↗
INGENIOUS Thermal Conductivity Measurement Source Categorization
Thermal conductivity (TC) data taken for different wells at a specified drill depth. This is an abridged version of the complete SMU heat flow database, downloaded from the SMU node of the NGDS at the beginning of INGENIOUS (approximately April 2021), and filtered to the INGENIOUS study area. This National Geothermal Data System (NGDS) project aggregates geothermal data collected and curated by the SMU Geothermal Laboratory and its partner organizations. All columns in this database are the same as the SMU database, except for 2 additions associated with this project. Repeated columns are for data correlation purposes. Column descriptions and data types are the same as previous iterations of the SMU database. The new values that are the addition are two new columns developed as part of the INGENIOUS project: INGENIOUS TC Value | INGENIOUS notes INGENIOUS notes are individual notes that were written for specific data points during the analysis process. There are not always notes associated with each input value. INGENIOUS TC Value includes 4 values: 1. Assumed Measured These are values that are assumed to be measured thermal conductivity values, either within a specific well or within the same study region. Many of these have either a published reference, a reported standard deviation, or a unique thermal conductivity value. 2. Data release - assumed measured These are values in the SMU database that are from proprietary data that were added to the SMU database and are labeled as data release for their reference. These values were searched for in person at the SMU Geothermal Laboratory as well as virtual examination of data available on the NGDS. For many of these, there are reported thermal conductivity values associated with the heat flow data in the database, but no specific table or reference to measurements in the original data release files. 3. Known measured These are values that have a reported measurement, either as an original file in the SMU data files on the NGDS or a reported table in a publication. In the rare circumstances, Maria Richards or David Blackwell confirmed measurement. Confirmation of measurement would be written in the INGENIOUS notes column. 4. Unmeasured Unmeasured values are those that are known to be unmeasured, either estimated from another report or no information given. In the SMU database, there are wells that have a heat flow but no thermal conductivity. These are categorized as unmeasured. There are also heat flow values that are stated to have estimated or generalized average thermal conductivity values for the region and rock type. Because these are known to be unmeasured, they are categorized as such. 5. Blank Blank values are either A quality or X quality. These quality values are stated in the INGENIOUS notes. These values were not going to change associated with the heat flow analysis, so these were not examined.
15 GEOTHERMAL ENERGY↗
Monthly Quality-filtered Aggregation of NOAA Climate Data Record (CDR) of AVHRR Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR), Version 5
This dataset contains gridded monthly Leaf Area Index (LAI) derived from the daily NOAA Climate Data Record (CDR) of AVHRR Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR), Version 5. This data record spans from 1981 to 2018 using data from eight NOAA polar orbiting satellites: NOAA-7, -9, -11, -14, -16, -17, -18 and -19. The data are projected on a 0.05 degree x 0.05 degree global grid, as in the original CDR. The original CDR is one of the Land Surface CDR Version 5 products produced by the NASA Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), which is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program. This dataset is in the netCDF-4 file format following ACDD and CF Conventions. This dataset has applied quality assurance information to only include "OK" data from the original CDR in the monthly aggregation.
Vermote, Eric [NASA Goddard Space Flight Center (G↗
Monthly Quality-filtered Aggregation of NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR)
This dataset contains gridded monthly Leaf Area Index (LAI) derived from the daily NOAA Climate Data Record (CDR) of AVHRR (Version 5) and VIIRS (Version 1) Leaf Area Index (LAI) and Fraction of Absorbed Photosynthetically Active Radiation (FAPAR). This data record spans from 1981 to 2024 using data from NOAA polar orbiting satellites: NOAA-7, -9, -11, -14, -16, -17, -18, -19 and S-NPP. The data are projected on a 0.05 degree x 0.05 degree global grid, as in the original CDR. The original CDR is one of the Land Surface CDR products produced by the NASA Goddard Space Flight Center (GSFC) and the University of Maryland (UMD), which is accompanied by algorithm documentation, data flow diagram and source code for the NOAA CDR Program. This dataset is in the netCDF-4 file format following ACDD and CF Conventions. This dataset has applied quality assurance information to only include "OK" data from the original CDR in the monthly aggregation.
Vermote, Eric [NASA Goddard Space Flight Center (G↗
Accelerating shared file checkpoint with local burst buffers
A data management system and method for accelerating shared file checkpointing. Written application data is aggregated in an application data file created in a local burst buffer memory at a compute node, and an associated data mapping built index to maintain information related to the offsets into a shared file at which segments of the application data is to be stored in a parallel file system, and where in the buffer those segments are located. The node asynchronously transfers a data file containing the application data and the associated data mapping index to a file server for shared file storage. The data management system and method further accelerates shared file checkpointing in which a shared file, together with a map file that specifies how the shared file is to be distributed, is asynchronously transferred to local burst buffer memories at the nodes to accelerate reading of the shared file.
Gooding, Thomas↗
Trace elements in the Allende meteorite. IV - Amoeboid olivine aggregates
INAA data for Ca, Sc, Hf, La, Ce, Sm, Eu, Tb, Yb, Lu, Os, Ir, Ru, Na, Cl, Br, Fe, Mn, Cr, Co, Au, As, and Sb are presented for ten amoeboid aggregates from the Allende meteorite. Only one lacks olivine. Seven of the remainder, as a group, have cosmic proportions of refractory lithophile and siderophile elements and appear to have formed when coarse-grained Allende inclusion material underwent partial reaction with a low-temperature nebular gas and mixture with FeO-rich olivine. The other two have highly fractionated abundances of refractory elements relative to one another compared to Cl chondrites, including Group II REE patterns, and probably formed by the mixing of fine-grained Allende inclusion material with FeO-rich olivine. Non-refractory siderophile components are also different in composition in each type of amoeboid olivine aggregate.
Grossman, L.↗
Global crop forecasting
The needs for and remote sensing means of global crop forecasting are discussed, and key results of the Large Area Crop Inventory Experiment (LACIE) are presented. Current crop production estimates provided by foreign countries are shown often to be inadequate, and the basic elements of crop production forecasts are reviewed. The LACIE project is introduced as a proof-of-concept experiment designed to assimilate remote sensing technology, monitor global wheat production, evaluate key technical problems, modify the technique accordingly and demonstrate the feasibility of a global agricultural monitoring system. The global meteorological data, sampling and aggregation techniques, Landsat data analysis procedures and yield forecast procedures used in the experiment are outlined. Accuracy assessment procedures employed to evaluate LACIE technology performance are presented, and improvements in system efficiency and capacity during the three years of operation are pointed out. Results of LACIE estimates of Soviet, U.S. and Canadian wheat production are presented which demonstrate the feasibility and accuracy of the remote-sensing approach for global food and fiber monitoring.
Macdonald, R. B.↗
PV System Availability from Commercial and Utility-Scale Systems [Slides]
In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with worse inverter availability results for larger PV systems. Potential causes of this effect are under investigation.
14 SOLAR ENERGY↗
PV Inverter Availability from the U.S. PV Fleet
In the PV Fleet Performance Data Initiative, we partner with photovoltaic (PV) fleet owners to collect time-series PV production data, and publish aggregated, anonymized results. An assessment of system availability is conducted on 1128 systems which passed our data quality checks, and include cumulative energy meter data. Overall inverter availability is low in the first 6 months of system performance before reaching steady-state by the end of the first year. System-level aggregated data shows a median (P50) system availability of 0.99, and a lower P90 value of 0.95. A dependence on system size is also identified, with better inverter availability results for smaller PV systems. Potential causes of this effect may include the selection of inverter itself: smaller inverters 6kW-250kW showed better average availability than inverters 300kW-5MW. The elimination of string combiner boxes and lower energy impact when one particular inverter goes off-line are potential benefits of a string inverter-based PV system architecture. DNV also analyzed availability data from over 1100 operating systems and found similar trends. DNV's P50 industry guidance on expected availability has been updated to reflect the data and the following observations: utility scale systems have lower availability than DG systems, availability is lower in first year compared to subsequent years, and that actual availability is lower than expected.
fleet↗
Carbon Storage Site Mapping Inquiry Tool (MapIT)
To date, 48 projects, consisting of 139 wells, are currently under review with the Environmental Protection Agency’s (EPA) Underground Injection Control (UIC) Program for Class VI – wells used for geologic sequestration of carbon dioxide. The number of applications submitted is expected to increase in coming years with the increase of the 45Q tax credit available to projects that initiate construction prior to 2033. The amount of data collected to submit a Class VI permit is vast, and often disparate, coming from state, federal, and commercial entities, as well as field-specific data collected within an area of interest. When preparing for site selection and permitting, the initial aggregation of relevant public data can be time intensive. The Carbon Storage Site Mapping Inquiry tool (MapIT) was created to support and accelerate the discovery and accessibility of open-source data and information available across the USA. Data was aggregated and organized based on data types described within the EPA UIC Class VI permit documentation. The online tool enables users to explore hundreds of geospatial data layers and connect to additional external resources, leveraging API and REST services where possible to ensure updates to data in real time. MapIT enables users to explore state and federal data related to geologic, geophysical, structural, hydrologic, and contextual information. In addition to displaying spatial data and linking to external resources, MapIT leverages custom widgets to ensure that internal data and external data are discoverable and accessible. The widgets connect users to resources such as the USGS publications and the USGS Earthquake Catalog based on a user-defined location. This talk will describe data aggregation workflows, data types, data preparation, and tool development for MapIT. The Carbon Storage Site Mapping Inquiry Tool and underlying database are valuable, intuitive resources that empower government, academic, commercial and industry stakeholders to explore, analyze, and acquire carbon storage related data.
Morkner, Paige↗
A Generalized approach to the operationalization of Software Quality Models
Comprehensive measures of quality are a research imperative, yet the development of software quality models is a wicked problem. Definitive solutions do not exist and quality is subjective at its most abstract. Definitional measures of quality are contingent on a domain, and even within a domain, the choice of representative characteristics to decompose quality is subjective. Thus, the operationalization of quality models brings even more challenges. A promising approach to quality modeling is the use of hierarchies to represent characteristics, where lower levels of the hierarchy represent concepts closer to real-world observations. Building upon prior hierarchical modeling approaches, we developed the Platform for Investigative software Quality Understanding and Evaluation (PIQUE). PIQUE surmounts several quality modeling challenges because it allows modelers to instantiate abstract hierarchical models in any domain by leveraging organizational tools tailored to their specific contexts. Here, we introduce PIQUE; exemplify its utility with two practical use cases; address challenges associated with parameterizing a PIQUE model; and describe algorithmic techniques that tackle normalization, aggregation, and interpolation of measurements.
Data aggregation↗
Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure
This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.
24 POWER TRANSMISSION AND DISTRIBUTION↗