SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
At the NSF-DOE Vera C. Rubin Observatory we write data from the LSSTCam and LATISS instruments using FITS format with one file written per detector. Here we discuss the layout of those FITS files and describe the FITS headers.
Contains all of the requisite files for reproducing the figures/resultis in the Phys. Rev. A. submission cited below.
This data package contains data from terrestrial laser scanning (TLS) at the Pasoh Forest Reserve, Malaysia. The Pasoh Forest Reserve is a facility of the Forest Research Institute Malaysia, and contains evergreen lowland dipterocarp forest. The Next-Generation Ecosystem Experiments Tropics (NGEE-Tropics) study areas at Pasoh were established to study how different species respond to climatic variation and soil water availability. Two study areas were chosen representing different topography and species. The TLS data archived here were collected to provide detailed, three-dimensional information about forest structure. Specifically, data were collected to allow tree-level characterization of woody structure and leaf area for 12 focal trees with FloraPulse and sap flux sensors, facilitating estimation of woody biomass and leaf area to allow upscaling of water content and transpiration data to the tree-level. Scan positions were not selected to provide consistent data for non-focal trees with the study areas. This data package contains the following data: - High-level files document further details of the campaign and data package: 1_CampaignSummary.csv provides details about the campaign and study site, 2_ScanAreasDetail.csv provides details about each separate scan area (groups of scans post-processed into a single point cloud), 3_TerrestrialLidarSensor.csv provides further technical details about the Riegl VZ-400i TLS sensor, TLS_CSV_dd.csv is a CSV Data Dictionary providing information about the fields in CSV files following the ESS-DIVE CSV File Formatting Guidelines Reporting Format, TLS_flmd.csv is a File Level Metadata file providing information about each file in the data package following the ESS-DIVE File Level Metadata Reporting Format, and README.txt is a text file describing the overall project and file structure. - Level 0 data are the raw data (.PROJ folders) as recorded by the Riegl VZ-400i TLS instrument before scan co-registration and post-processing with the Riegl's proprietary RiSCAN PRO software, which requires a license. - Level 1 data contain post-processed, co-registered data from each scan area. The "PointClouds" folder for each scan area contains a .las file with 1 cm resolution point cloud data exported from RiSCAN PRO. These are the main files likely to be of interest to most users and can be further processed with any software capable of manipulating .las files (e.g. Python, R CloudCompare). The "Project Information" folder contains log files from post-processing in RiSCAN PRO that may be of interest to users who want to see detailed records of post-processing, including all PDF reports generated by RiSCAN PRO. The "ScanPositions" folder contains information about the final position of all TLS scans, after post-processing, in multiple formats. The file ScanPositions_*.csv provides final geo-referenced scan positions, and the file SOP_backup_*.csv can be used in RiSCAN PRO to restore the co-registered scan positions if users wish to re-process raw data (Level 0 .PROJ folders) with RiSCAN PRO software (e.g., subsample to a different resolution, exclude a certain scan position, or apply different filters on reflectance or deviation values) without redoing time-consuming co-registration steps.
This data package contains data from terrestrial laser scanning (TLS) at five urban park sites in Knoxville, Tennessee, USA. All parks include open-grown and/or closed-canopy trees and mixed nearby land use. These study sites were established as part of the Urban Biogeochemistry Pilot Project, which has an overall goal of better understanding how hydrobiogeochemical cycling is altered within the human environment. These five sites represent a gradient of urbanization, and were instrumented to understand hydrological and biogeochemical cycling (e.g., soil moisture, soil physical properties and biogeochemistry, tree transpiration, species type). The TLS data archived here were collected to provide detailed, three-dimensional information about forest structure. Specifically, data were collected to allow tree- and stand-level characterization of woody structure and leaf area. TLS scans were placed to capture the area around trees with sap flow sensors, and as much of a 50 m radius area around the meteorological station as possible given site property limits. Derived products will allow upscaling of water content and transpiration data. This data package contains the following data: - High-level files document further details of the campaign and data package: 1_CampaignSummary.csv provides details about the campaign and study site, 2_ScanAreasDetail.csv provides details about each separate scan area (groups of scans post-processed into a single point cloud), 3_TerrestrialLidarSensor.csv provides further technical details about the Riegl VZ-400i TLS sensor, TLS_CSV_dd.csv is a CSV Data Dictionary providing information about the fields in CSV files following the ESS-DIVE CSV File Formatting Guidelines Reporting Format, TLS_flmd.csv is a File Level Metadata file providing information about each file in the data package following the ESS-DIVE File Level Metadata Reporting Format, and README.txt is a text file describing the overall project and file structure. - Level 0 data are the raw data (.PROJ folders) as recorded by the Riegl VZ-400i TLS instrument before scan co-registration and post-processing with the Riegl's proprietary RiSCAN PRO software, which requires a license. - Level 1 data contain post-processed, co-registered data from each scan area. The "PointClouds" folder for each scan area contains a .las file with 1 cm resolution point cloud data exported from RiSCAN PRO. These are the main files likely to be of interest to most users and can be further processed with any software capable of manipulating .las files (e.g. Python, R CloudCompare). The "Project Information" folder contains log files from post-processing in RiSCAN PRO that may be of interest to users who want to see detailed records of post-processing, including all PDF reports generated by RiSCAN PRO. The "ScanPositions" folder contains information about the final position of all TLS scans, after post-processing, in multiple formats. The file ScanPositions_*.csv provides final geo-referenced scan positions, and the file SOP_backup_*.csv can be used in RiSCAN PRO to restore the co-registered scan positions if users wish to re-process raw data (Level 0 .PROJ folders) with RiSCAN PRO software (e.g., subsample to a different resolution, exclude a certain scan position, or apply different filters on reflectance or deviation values) without redoing time-consuming co-registration steps.
During the development of high-consequence items, test systems should be capable of differentiating between test failures resulting from narrowly missing requirements versus those indicating potentially catastrophic faults. In many instances, classifying the data corresponds to simply identifying whether measured waveforms have approximately the anticipated shape. Cast in this light, the problem reduces to converting raw data into a form optimal for use with neural network classifiers. This manuscript investigates different means of representing raw data for image classification. Raw data plots and Short Time Fourier Transform (STFT) spectrograms are classified by both custom built, small-scale, Convolution Neural Networks (CNN) and open-source, multi-million parameter, pre-trained deep CNNs. In the case of time varying frequency content, the STFTs provide images with greater detail and can be accurately classified with simpler networks. This requires less memory and runs faster than classifying the raw data using the more sophisticated options—making STFTs optimal for applications with memory constraints. STFTs are not a panacea. In some cases the time-domain signal contains useful information that should not be discarded. Rather than using raw data or STFTs, the images can be constructed from both by using red and green channels of an RGB image to visualize the real and imaginary components of the transform, with the raw data occupying the blue channel.
Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE
A central repository of thermophysical and thermochemical properties of molten salt compositions of relevance to molten salt reactors (MSRs) is vital in supporting the broad community of MSR developers, who are at various stages of developing and deploying their reactor designs. In general, these MSR designs differ significantly from developer to developer (e.g., with respect to the hardness of the neutron spectra, level of fissile loading, target multicomponent temperatures and power levels, and moderating capabilities). Therefore, the fuel and coolant salts being considered vary greatly: they may be chlorides or fluorides, they utilize different actinides at different ratios, and the cations in the melt are selected based on perceived advantages and disadvantages. Considering the general need for thermal properties, and the vastness of the array of potential candidate salt mixtures, the Molten Salt Thermal Properties Database (MSTDB) was initiated in 2018 with the goal of providing thermophysical and thermochemical characterization of key molten salt compounds and mixtures across their temperature and compositional domains. The MSTDB is thus divided into the thermophysical arm (MSTDB-TP) and the thermochemical arm (MSTDB-TC). The MSTDB is an effort funded by the Department of Energy, Office of Nuclear Energy (DOE-NE) Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, and the MSR Campaign. This report provides an overview of the MSTDB-TP v4.0 in terms of the data contained within, the state of the tools used to access the data, the availability of predictive models that leverage the raw data in the database, the preliminary status of developmental efforts that are currently underway, and an account of future goals for MSTDB-TP. The primary goal for the update from MSTDB-TP v.3.1 to v4.0 was the incorporation of surface tension data into the database; this property is important for thermal hydraulics modeling and species transport in other tools that have been developed under the NEAMS program. A breakdown of the surface tension data that have been added into MSTDB-TP v4.0 is provided herein, and the manner in which the quality of the data has been assessed is also documented. For MSTDB-TP v4.0, newly published thermophysical property data—primarily from collaborative experimental efforts under the MSR Campaign—have been incorporated into the database, and the resulting expansion is documented here. Because of the size to which MSTDB-TP has grown, the raw data format has now been recast into JavaScript Object Notation (JSON) format for easier connection with the MSTDB-TP application programming interface (API). Saline; the pre-existing comma-separated value (CSV) format has been deprecated but is still maintained, accessible, and up to date. As a final effort in packaging the MSTDB-TP v4.0 update, the graphical user interface (GUI) for MSTDB has been updated to allow full accessibility to the density and viscosity predictive models, which are based on Redlich-Kister expansions of MSTDB-TP raw data. Some other major aspects of this report, in terms of preliminary and future work, include: (1) documentation of the formalism and preliminary testing of a kinetic theory model that may act as a predictive model for thermal conductivity; (2) documentation of the candidate predictive models that may be considered in the future for surface tension, making use of the surface tension data now in MSTDB-TP v4.0; (3) a preliminary account of a data collection process that will enable the filling of additional gaps within MSTDB-TP, namely with data which have been collected computationally (e.g., through ab initio molecular dynamics).
Raw Mass Spectrometry data generated for the manuscript "Rubisco kinetic acclimation at the holoenzyme level." It characterizes the stoichiometry of Rubisco subunits in different Rubisco samples.
Raw data for Figure 2 is located in: /Experiments/Fig2_delayed_aggregation Raw data for Figures 3 and 4 can be found at: /Experiments/Fig3_4_Core_shell The analysis code is available at: /Experiments/Code Raw data for the Figure 5 simulation is locate in: /Simulation/Raw_data/Fig5_Structural_features_of_cluster
These data are from Bandopadhyay et al., "Soil microbial ecology and microbiome-metabolite linkages improve understanding of ecosystem states along terrestrial-aquatic interfaces". This study aims to understand the soil microbial ecology along terrestrial-aquatic interfaces of a freshwater and estuarine region and how it relates to organic matter. We analyzed soil microbial (16S rRNA gene) and organic matter (Fourier-transform ion cyclotron resonance mass spectrometry, FTICR-MS) composition from upland (forested), transition (stressed forest), and wetland positions at three sites in each of the Lake Erie (freshwater) and Chesapeake Bay (estuarine) regions. This dataset includes 16S rRNA gene amplicon data (only processed file types included here) and organic matter composition from FTICR-MS data (raw and processed files included here) from upland (forested), transition (stressed forest), and wetland positions at three sites in each of the Lake Erie and Chesapeake Bay regions. These sites are part of the COMPASS-FME project (https://compass.pnnl.gov/FME/COMPASSFME). File formats and software needed to access files: 16S rRNA gene amplicon data: These files follow the format reported here https://ess-dive.gitbook.io/amplicon-sequencing-reporting-format#updates-in-v1.0.1. As per this format, there are four file types reported: 1. Taxon tables (also called sequence-by-sample or OTU (operational taxonomic unit)/ESV (exact sequence variant) tables) : available in a .txt file format and accessible using TextEdit or MS Excel. 2. Representative sequences (also called consensus sequences) : available in a .fasta format and accessible using TextEdit. 3. Sequencing metadata : available in a MS Excel workbook file format and CSV file format 4. Bioinformatic metadata : available in a MS Excel workbook file format and CSV file format FTICR-MS data: 1. Raw data converted to a processed file with intensities of the peaks in the given samples : available in a MS Excel CSV file format 2. Processed file used in analyses and visualizations (appended as icr_long_) : available in a MS Excel CSV file format 3. Metadata file for ICR features (appended as icr_meta) : available in a MS Excel CSV file format
This dataset contains raw data from the laser disdrometer at Nantucket. The disdrometer measures droplet size distribution, hydrometer type, and precipitation rate.
This dataset contains raw data from the WFIP3 RHOD site supplementary meteorological sensors: T/RH and barometer; 1-sec average. The supplementary meteorological suite was added to PNNL Surface Flux Station to provide independent measurements of air temperature, relative humidity, and atmospheric pressure.
One of the goals of the Deep Underground Neutrino Experiment (DUNE) is to use the massive underground liquid argon time projection chamber (LArTPC) detectors at its far site for multimessenger astronomy (MMA), in the detection of neutrinos from core-collapse supernovae (SNe). Its current baseline trigger strategy detects activity in the detector that is consistent with supernova (SN) neutrinos and saves the raw data for further offline analysis but provides no prompt pointing information crucial for optical follow-ups by other observatories. This approach is based on the assumption that prompt pointing determination using raw data is computationally prohibitive. In this article, we demonstrate a proof-of-concept based on applying extreme data reduction on the buffered SN data in the DUNE data acquisition (DAQ) system’s front-end computers using a machine learning (ML) workflow. This reduces the data by ~5 orders of magnitude, allowing a full track reconstruction to be carried out quickly on a single server. The total time to perform the ML-based data reduction and the full track reconstruction is less than the time to transfer the SN data back to Fermilab or a high-performance computing (HPC) center. This shows that prompt processing of raw SN data is possible and, in fact, trivial once the data have been reduced to reject radiological backgrounds, paving the way to a high-quality SN pointing trigger that is based on fully reconstructed data instead of trigger primitives (TPs).
This data package contains raw data, data output, and code of photosynthetic temperature response curves from tropical forests. Data are used in "Photosynthetic responses to temperature across the tropics: a meta-analytic approach". The research investigates how photosynthetic optimum temperatures and shapes of photosynthetic temperature response curves varies across tropical forest climates and growth conditions. Growth climate variables include mean annual temperature, max and max temperature, diurnal temperature range, and aridity index. Meta-analysis combines 18 datasets with representation from Africa, Oceana, North American, and South America. Growth conditions analyzed considers deciduousness, successional status, light conditions, and whether plants are grown in situ or ex situ. All files, except for the raw datafile (Tropical_MetaAnalysis_Master_3.8.22_Edit.csv), have been processed using R code, which is provided.
Large-scale scientific collaborations like ATLAS, Belle II, CMS, DUNE, and others involve hundreds of research institutes and thousands of researchers spread across the globe. These experiments generate petabytes of data, with volumes soon expected to reach exabytes. Consequently, there is a growing need for computation, including structured data processing from raw data to consumer-ready derived data, extensive Monte Carlo simulation campaigns, and a wide range of end-user analysis. To manage these computational and storage demands, centralized workflow and data management systems are implemented. However, decisions regarding data placement and payload allocation are often made disjointly and via heuristic means. A significant obstacle in adopting more effective heuristic or AI-driven solutions is the absence of a quick and reliable introspective dynamic model to evaluate and refine alternative approaches. In this study, we aim to develop such an interactive system using real-world data. By examining job execution records from the PanDA workflow management system, we have pinpointed key performance indicators such as queuing time, error rate, and the extent of remote data access. The dataset includes five months of activity. Additionally, we are creating a generative AI model to simulate time series of payloads, which incorporate visible features like category, event count, and submitting group, as well as hidden features like the total computational load—derived from existing PanDA records and computing site capabilities. These hidden features, which are not visible to job allocators, whether heuristic or AI-driven, influence factors such as queuing times and data movement.
These files collectively provide a comprehensive overview of the testing process, data analysis, and validation for the Twin Ocean Power device tested at the O.H. Hinsdale Wave Research Laboratory, supported by TEAMER funding. This resource includes an overview of power results for a series of 7 trials. The files included in this comprehensive overview include a comprehensive log sheet for each trial, a summary of all trials, and processing scripts for the raw data. It includes all raw data in .tsv and MATLAB compatible formats, an average power chart, angular velocity charts for each trial, trial metrics, and power output files. This resource includes images of the Twin Ocean Power Wave Energy Converter device components and movement during testing and video recordings of each trial.
This dataset contains water isotope (deuterium (dD) and oxygen (d18O)) data from porewater, experimental source water, and aquifer sources from the Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) experimental field site in Edgewater, MD. Samples were taken from 2023-2026 and included in file "TEMPEST_Water_Isotope_LANL_2023-2026.csv". Porewater samples were taken from 15cm depth at multiple locations in each experimental plot (i.e., control, freshwater, and saltwater). Water samples were stored in 2 ml amber glass vials with septum caps under refrigeration until analysis. A L2140-i Picarro cavity ringdown spectrometer connected to an A0211 Picarro vaporization module was used to run water samples. Standards (United States Geological Survey (USGS) 47 and 48) were used to check instrument accuracy during each run of samples. Post-processing codes were run to calculate isotopic values from raw data. Reported isotope data is post-processed. Raw data CSV files from the Picarro were processed using a Python script and output as excel files containing data calibrated to Vienna Standard Mean Ocean Water (VSMOW). Standardization is based on USGS47 and USGS48 standards that are measured before and after every 5-10 unknowns. Every measurement is comprised of 10 injections. Processing involved discarding the first 4 injections due to a known memory effect. Raw means of the last 6 injections are taken for each measurement and then corrected from the linear calibration relationship of the USGS standards expected vs measured values to obtain corrected compositions relative to VSMOW. Reported isotope data are corrected mean results.
The magnetometer data was collected in multiple directions across the watershed hillslope at the NGEE Arctic Teller Road site at mile marker 27 (TL_MM27) on the Seward Peninsula, Alaska over multiple years in March 2018 and April 2023. The magnetic data were collected using a Geometrics Inc. G-858 gradiometer and G-857 base station in 2018 and the G-864 gradiometer and G857 base station in 2023. The data was collected (in all instances) by towing the gradiometer behind a snow machine around the watershed with the two sensors in a vertical profile with constant spacing during the continuous survey in that specific year. Magnetic total field measurements were collected by gradiometer and base station, and the data processing was performed in Geometrics MagMap2000 software. The processing steps were limited to removal of data spikes (despiking), reading dropouts, and correction/removal of bad GPS points. All offsets between sensors and GPS are stated within the data files and metadata, alongwith the processed and raw data. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).In this data submission there are two sets of raw magnetic data (.bin and .stn for 2018 and base for 2023; raw rover mag for 2023 is in .csv) inside two .zip files that identify the year the mag data was collected. The data are proprietary format to Geometrics and can be opened and processed with MagMap2000 which can be downloaded for free at Geometrics website. There are also two processed data files *.csv for each year and two metadata files *.csv.