Search NASA⌕ Search

SEARCH · Search NASA

Results for “data formats”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

ESS-DIVE Reporting Format for Hydrologic Monitoring Data and Metadata

The ESS-DIVE Reporting Format (RF) for Hydrologic Monitoring Data and Metadata was designed for water level, temperature, electrical conductivity, specific conductance, dissolved oxygen, and pH measured in situ using sensors deployed in water bodies. It can also be applied to a broader suite of variables beyond its original scope. The RF is intended to provide clear and easy guidance to data generators aiming to upload consistent findable, accessible, interoperable, and reusable (FAIR) datasets. The RF includes instructions (pdf), a term guide (csv), templates (csv and xlsx), and a list of recommended vocabulary (csv) to equip the data generator with all of the tools needed to successfully apply the RF. There is also a crosswalk to some key data resources and standards (csv). Users should cite this dataset when using the RF but are encouraged to access the RF files on GitHub (https://github.com/ess-dive-community/essdive-hydrologic-monitoring) where the RF documentation is maintained and updated on the ESS-DIVE Community Space GitHub. Questions and suggestions for improvement can be submitted as GitHub issues.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Results from a multi-laboratory ocean metaproteomic intercomparison: effects of LC-MS acquisition and data analysis procedures

Metaproteomics is an increasingly popular methodology that provides information regarding the metabolic functions of specific microbial taxa and has potential for contributing to ocean ecology and biogeochemical studies. A blinded multi-laboratory intercomparison was conducted to assess comparability and reproducibility of taxonomic and functional results and their sensitivity to methodological variables. Euphotic zone samples from the Bermuda Atlantic Time-series Study (BATS) in the North Atlantic Ocean collected by in situ pumps and the autonomous underwater vehicle (AUV) Clio were distributed with a paired metagenome, and one-dimensional (1D) liquid chromatographic data-dependent acquisition mass spectrometry analysis was stipulated. Analysis of mass spectra from seven laboratories through a common bioinformatic pipeline identified a shared set of 1056 proteins from 1395 shared peptide constituents. Quantitative analyses showed good reproducibility: pairwise regressions of spectral counts between laboratories yielded R 2 values averaged 0.62±0.11, and a Sørensen similarity analysis of the top 1000 proteins revealed 70 %–80 % similarity between laboratory groups. Taxonomic and functional assignments showed good coherence between technical replicates and different laboratories. A bioinformatic intercomparison study, involving 10 laboratories using eight software packages, successfully identified thousands of peptides within the complex metaproteomic datasets, demonstrating the utility of these software tools for ocean metaproteomic research. Lessons learned and potential improvements in methods were described. Future efforts could examine reproducibility in deeper metaproteomes, examine accuracy in targeted absolute quantitation analyses, and develop standards for data output formats to improve data interoperability. Together, these results demonstrate the reproducibility of metaproteomic analyses and their suitability for microbial oceanography research, including integration into global-scale ocean surveys and ocean biogeochemical models.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

CMS Storage Performance with RNTuple

CMS is transitioning to use ROOT’s new RNTuple data storage format for the files CMS will write in the HL-LHC era. Based on initial tests, CMS expects faster I/O and smaller files compared to the present TTree storage format. This contribution will show a comprehensive performance comparison between RNTuple and TTree I/O using CMS AOD and MiniAOD data formats as test cases for both simulation and collision data corresponding to similar data taking conditions of LHC Run 3. Quantities such as the resulting file size, the memory usage of the I/O components, and the rate of events being read from a file or written to a file will be measured. CMS’ data processing relies heavily on reading files over the local or wide area networks. The file read patterns are important because the latencies have been seen to influence the total production job times. Therefore a study on the file read patterns will be conducted by recording traces of the offset, size, and timestamp of each read request for both RNTuple and TTree. The behavior of network reads will be mimicked by reading local files where artificial latency will be added to the read requests. The effect of different latency values on the job times will be studied.

Jones, Christopher D. [Fermilab]↗
DOE OSTI2025

BaZrO3-Ni-defect-formation

These data include migration energies for interstitial Ni defects in doped and undoped BaZrO3, as well as formation energies for Ni-related defects and defect complexes.

08 HYDROGEN↗
DOE OSTI2024

cjohnson-LANL/GRL_Kilauea

The python routines are outlined in detail to perform the methods and results in the manuscript under review in the journal Geophysical Research Letters titled “Seismic features predict ground motions during repeating caldera collapse sequence” with LA-UR-23-33345. All routines are written in open source python and were applied to publicly available data sets. The codes formats the data into the appropriate structure required to train a boosted tree regression model. Other codes produce figure results.

Johnson, Christopher W↗
DOE OSTI2024

Prediction of plant complex traits via integration of multi-omics data

The formation of complex traits is the consequence of genotype and activities at multiple molecular levels. However, connecting genotypes and these activities to complex traits remains challenging. Here, we investigate whether integrating genomic, transcriptomic, and methylomic data can improve prediction for six Arabidopsis traits. We find that transcriptome- and methylome-based models have performances comparable to those of genome-based models. However, models built for flowering time using different omics data identify different benchmark genes. Nine additional genes identified as important for flowering time from our models are experimentally validated as regulating flowering. Gene contributions to flowering time prediction are accession-dependent and distinct genes contribute to trait prediction in different genotypes. Models integrating multi-omics data perform best and reveal known and additional gene interactions, extending knowledge about existing regulatory networks underlying flowering time determination. These results demonstrate the feasibility of revealing molecular mechanisms underlying complex traits through multi-omics data integration.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

mzapy : An Open-Source Python Library Enabling Efficient Extraction and Processing of Ion Mobility Spectrometry-Mass Spectrometry Data in the MZA File Format

We have recently reported MZA, a new and simple mass spectrometry data structure based on the broadly supported HDF5 format and created to facilitate software development. While this format is inherently supportive of application development, the availability of a core library with standard mass spectrometry utilities greatly facilitates fast software development. Here, we present a Python library, mzapy, for efficient extraction and processing of mass spectrometry data in the MZA format. In addition to raw data extraction, mzapy contains supporting utilities enabling tasks including calibration, signal processing, peak finding, and generating plots. Being implemented in pure Python with minimal and largely standardized dependencies makes mzapy uniquely suited to application development in the multi-omics domain. The free and open source mzapy is built with extensibility in mind, and future development will support cloud computing and artificial intelligence/machine learning applications. The software source code is freely available at https://github.com/PNNL-m-q/mzapy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Photoelectrochemically Self Improving Si/GaN Photocathode: Figure 4a Raw Data

Calculated formation energy of (0001 ̅) c-plane GaN surfaces. The density-functional theory (DFT) calculations were performed with the generalized gradient approximation, using the plane-wave pseudopotential quantum-ESPRESSO package. Ultrasoft pseudopotentials were used to describe the interaction between valence electrons and ionic cores, where Ga 3d electrons were explicitly treated as valence electrons. The wave functions and electronic density were expanded in a plane-wave basis set truncated at a cutoff energy of 30 Ry and 240 Ry, respectively. The surface was modeled using a repeated slab geometry using a (2×2) unit cell with a thickness of seven GaN bilayers, where the Ga-terminated bottom side was passivated by pseudo-hydrogen with charge of 1.25e. In addition, the (1010) surface was modeled using 8 GaN atomic layers with a lateral dimension of 9.57 Å x 10.38 Å. A vacuum width of at least 15.0 Å was introduced between consecutive slabs, and a k-point mesh of 5x5x1 and 2x2x1 was used to sample the surface Brillouin zone for the polar and non-polar surfaces, respectively. Calculated formation energy of (0001 ̅) c-plane GaN surfaces with different oxygen configuration within the anion and cation-rich limits: ideal GaN surfaces of c-plane and m-plane; configurations with 100% replacement of nitrogen by oxygen on the surface c-plane and m-plane; 100% replacement of nitrogen by oxygen in the subsurface bilayer c-plane and m-plane; and configurations with 50% replacement of nitrogen of the surface and subsurface bilayers by oxygens c-plane and m-plane. In both cases, the ideal surface is chosen as the reference.

photocathode↗
DOE OSTI2021

Photoelectrochemically Self Improving Si/GaN Photocathode: Figure 4b Raw Data

Calculated formation energy of (1010 ̅) m-plane GaN surfaces. The density-functional theory (DFT) calculations were performed with the generalized gradient approximation, using the plane-wave pseudopotential quantum-ESPRESSO package. Ultrasoft pseudopotentials were used to describe the interaction between valence electrons and ionic cores, where Ga 3d electrons were explicitly treated as valence electrons. The wave functions and electronic density were expanded in a plane-wave basis set truncated at a cutoff energy of 30 Ry and 240 Ry, respectively. The surface was modeled using a repeated slab geometry using a (2×2) unit cell with a thickness of seven GaN bilayers, where the Ga-terminated bottom side was passivated by pseudo-hydrogen with charge of 1.25e. In addition, the (1010) surface was modeled using 8 GaN atomic layers with a lateral dimension of 9.57 Å x 10.38 Å. A vacuum width of at least 15.0 Å was introduced between consecutive slabs, and a k-point mesh of 5x5x1 and 2x2x1 was used to sample the surface Brillouin zone for the polar and non-polar surfaces, respectively. Calculated formation energy of (1010 ̅) m-plane GaN surfaces with different oxygen configuration within the anion and cation-rich limits: ideal GaN surfaces of c-plane and m-plane; configurations with 100% replacement of nitrogen by oxygen on the surface c-plane and m-plane; 100% replacement of nitrogen by oxygen in the subsurface bilayer c-plane and m-plane; and configurations with 50% replacement of nitrogen of the surface and subsurface bilayers by oxygens c-plane and m-plane. In both cases, the ideal surface is chosen as the reference.

photocathode↗
DOE OSTI2024

HarDWR - Harmonized Water Rights Records

For a detailed description of the database of which this record is only one part, please see the HarDWR meta-record. Here we present a new dataset of western U.S. water rights records. This dataset provides consistent unique identifiers for each spatial unit of water management across the domain, unique identifiers for each water right record, and a consistent categorization scheme that puts each water right record into one of 7 broad use categories. These data were instrumental in conducting a study of the multi-sector dynamics of intersectoral water allocation changes through water markets (Grogan et al., in review). Specifically, the data were formatted for use as input to a process-based hydrologic model, WBM, with a water rights module (Grogan et al., in review). While this specific study motivated the development of the database presented here, U.S. west water management is a rich area of study (e.g., Anderson and Woosly, 2005; Tidwell, 2014; Null and Prudencio, 2016; Carney et al, 2021) so releasing this database publicly with documentation and usage notes will enable other researchers to do further work on water management in the U.S. west. The raw downloaded data for each state is described in Lisk et al. (in review), as well as here. The dataset is a series of various files organized by state sub-directories. The first two characters of each file name is the abbreviation for the state the in which the file contains data for. After the abbreviation is the text which describes the contents of the file. Here is each file type described in detail: XXFullHarmonizedRights.csv: A file of the combined groundwater and surface water records for each state. Essentially, this file is the merging of XXGroundwaterHarmonizedRights.csv and XXSurfaceWaterHarmonizedRights.csv by state. The column headers for each of this type of file are: state - The name of the state the data comes from. FIPS - The two-digit numeric state ID code. waterRightID - The unique identifying ID of the water right, the same identifier as its state uses. priorityDate - The priority date associated with the right. origWaterUse - The original stated water use(s) from the state. waterUse - The water use category under the unified use categories established here. source - Whether the right is for surface water or groundwater. basinNum - The alpha-numeric identifier of the WMA the record belongs to. CFS - The maximum flow of the allocation in cubic feet per second (ft3s-1). Arizona is unique among the states, as its surface and groundwater resources are managed with two different sets of boundaries. So, for Arizona, the basinNum column is missing and instead there are two columns: surBasinNum - The alpha-numeric identifier of the surface water WMA the record belongs to. grdBasinNum - The alpha-numeric identifier of the groundwater WMA the record belongs to. XXStatePOD.shp: A shapefile which identifies the location of the Points of Diversion for the state's water rights. It should be noted that not all water right records in XXFullHarmonizedRights.csv have coordinates, and therefore may be missing from this file. XXStatePOU.shp: A shapefile which contains the area(s) in which each water right is claimed to be used. Currently, only Idaho and Washington provided valid data to include within this file. XXGroundwaterHarmonizedRights.csv: A file which contains only harmonized groundwater rights collected from each state. See XXFullHarmonizedRights.csv for more details on how the data is formatted. XXSurfaceWaterHarmonizedRights.csv: A file which contains only harmonized surface water rights collected from each state. See XXFullHarmonizedRights.csv for more details on how the data is formatted. Additionally, one file, stateWMALabels.csv, is not stored within a sub-directory. While we have referred to the spatial boundaries that each state uses to manage its water resources as WMAs, this term is not shared across all states. This file lists the proper name for each boundary set, by state. For those whom may be interested in exploring our code more in depth, we are also making available an internal data file for convenience. The file is in .RData format and contains everything described above as well as some minor additional objects used within the code calculating the cumulative curves. For completeness, here is a detailed description of the various objects which can be found within the .RData file: states: A character vector containing the state names for those states in which data was collected for. More importantly, the index of the state name is also the index in which that state's data can be found in the various following list objects. For example, if California is the third index in this object, the data for California will also be in the third index for each accompanying list. rightsByState_ground: A list of data frames with the cleaned ground water rights collected from each state. This object holds the the data that is exported to created the xxGroundwaterHarmonizedRights.csv files. rightsByState_surface: A list of data frames with the cleaned surface water rights collected from each state. This object holds the the data that is exported to created the xxSurfaceWaterHarmonizedRights.csv files. fullRightsRecs: A list of the combined groundwater and surface water records for each state. This object holds the the data that is exported to created the xxFullHarmonizedRights.csv files. projProj: The spatial projection used for map creation in the beginning of the project. Specifically, the World Geodetic System (WGS84) as a coordinate reference system (CRS) string in PROJ.4 format. wmaStateLabel: The name and/or abbreviation for what each state legally calls their WMAs. h2oUseByState: A list of spatial polygon data frames which contain the area(s) in which each water right is claimed to be used. It should be noted that not all water right records have a listed area(s) of use in this object. Currently, only Idaho and Washington provided valid data to be included in this object. h2oDivByState: A list of spatial points data frames which identifies the location of the Point of Diversion for the state's water rights. It should be noted that not all water right records have a listed Point of Diversion in this object. spatialWMAByState: A list of spatial polygon data frames which contain the spatial WMA boundaries for each state. The only data contained within the table are identifiers for each polygon. It is worth reiterating that Arizona is the only state in which the surface and groundwater WMA boundaries are not the same. wmaIDByState: A list which contains the unique ID values of the WMAs for each state. plottingDim: A character vector used to inform mapping functions for internal map making. Each state is classified as either "tall" or "wide", to maximize space on a typical 8x11 page. The code related to the creation of this dataset can be viewed within HarDWR GitHub Repository/dataHarmonization.

Economics↗
DOE OSTI2023

HarDWR - Raw Water Rights Records

For a detailed description of the database of which this record is only one part, please see the HarDWR meta-record. In order to hold a water right in the western United States, an entity, (e.g., an individual, corporation, municipality, sovereign government, or non-profit) must register a physical document with the state's water regulatory agency. State water agencies each maintain their own database containing all registered water right documents within the state, along with relevant metadata such as the point of diversion and place of use of the water. All western U.S. states have digitized their individual water rights databases, along with the geospatial data describing the spatial units where water rights are managed. Each state maintains and provides their own water rights data in accordance with individual state regulations and standards. We collected water rights databases from 11 western United States states either by downloading them from publicly accessible web portals, or by contacting state water management representatives; detailed descriptions of where and when the data was collected is provided in the README.txt, as well as Lisk et al.(in review). This collection of data are those raw water rights. Each state formats their data differently, meaning that file types, field availability, and names vary from state to state. Note, the data provided here reflects the state of the water rights databases at the time we collected the data; updates have likely occurred in many states. Some pieces of information are common among all states. These are: priority date, volume or flow of water allowed by the right, stated water use of the right, and some means of identifying the geography and source of the water pertaining to the right - typically the coordinates of the Point of Diversion (PoD) of a waterbody or well. Arizona regulates water in a different way than the other 10 states. Outside of some relatively small critical agricultural areas called Active Management Areas (AMAs), Arizona does not maintain any water rights. However, the state does require registration of surface and groundwater pumping devices, which includes disclosing the mechanical specifics of the devices. We used these records as a proxy for water rights. Each state, and their respective water right authorities, have made their water right records available for non-commercial reference uses. In addition, the states make no guarantees as to the completeness, accuracy, or timeliness of their respective databases, let alone the modifications which we, the authors of this paper, have made to the collected records. None of the states should be held liable for using this data outside of its intended use. In addition, the following states have requested specifically worded disclaimers to be included with their data. Colorado: "The data made available here has been modified for use from its original source, which is the State of Colorado. THE STATE OF COLORADO MAKES NO REPRESENTATIONS OR WARRANTY AS TO THE COMPLETENESS, ACCURACY, TIMELINESS, OR CONTENT OF ANY DATA MADE AVAILABLE THROUGH THIS SITE. THE STATE OF COLORADO EXPRESSLY DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. The data is subject to change as modifications and updates are complete. It is understood that the information contained in the Web feed is being used at one's own risk." Montana: "The Montana State Library provides this product/service for informational purposes only. The Library did not produce it for, nor is it suitable for legal, engineering, or surveying purposes. Consumers of this information should review or consult the primary data and information sources to ascertain the viability of the information for their purposes. The Library provides these data in good faith but does not represent or warrant its accuracy, adequacy, or completeness. In no event shall the Library be liable for any incorrect results or analysis; any direct, indirect, special, or consequential damages to any party; or any lost profits arising out of or in connection with the use or the inability to use the data or the services provided. The Library makes these data and services available as a convenience to the public, and for no other purpose. The Library reserves the right to change or revise published data and/or services at any time." Oregon: "This product is for informational purposes and may not have been prepared for, or be suitable for legal, engineering, or surveying purposes. Users of this information should review or consult the primary data and information sources to ascertain the usability of the information." The available data is provided as a series of compressed files, which each containing the full data collected from each state. Some of the files have been renamed, to more easily know which state the data belongs to. The file renaming was also required as some files from different states had the same name. In other cases, the data for a state has been placed in a folder indicating which state it belongs to - as the state organized its data by selected subregions. Below is a brief description of the format of the collected data from each state. ArizonaRights_StatementOfClaimants: A folder containing a database of interconnected CSV files. The soc_erd.pdf file contains a visual flowchart of how the various files are connected, beginning with SOC_MAIN.csv in the center of the page. ArizonaRights_SurfaceWaterRightsData: A folder containing a database of a single Shapefile and 10 associated CSVs. SurfaceWater.pdf contains a visual flowchart of how the various files are connected, beginning with ADWR_SW_APPL_REGRY.csv. ArizonaRights_Well55Registry: A folder containing a database of a single Shapefile and 59 associated CSVs. Wells55.pdf contains a visual flowchart of how the various files are connected, beginning with WellRegistry.shp. CaliforniaRights_eWRIMS_directDatabase: A folder containing a collection of four "series" Microsoft Excel files, as either XLS or XLSX. The four "series": byCounty, byEntity (what type of legal entity holds the right), byUse (stated water use), and byWatershed, are various methods by which the California water rights are organized within the state's database. However, it was observed that by only collecting a single series, not all water rights were being provided. So, essentially, the majority of records within each "series" are copies of each other, with each "series" containing some unique records. ColoradoRights_NetAmounts: A folder containing 78 CSV files, with one file per Colorado Water District. IdahoRights_PointOfDiversion: A Shapefile containing the Points of Diversion for the entire state of Idaho. IdahoRights_PlaceOfUse: A Shapefile containing the Place of Use polygons for the entire state of Idaho. MontanaRights_WaterRights: A Geodatabase file containing the Points of Diversion and Places of Use for the entire state of Montana. The name of the Points of Diversion Feature Layer within the Geodatabase is "WRDIV", and the name of the Places of Use Feature Layer is "WRPOU". NevadaRights_POD_Sites: A Shapefile containing the Points of Diversion for the entire state of Nevada. NewMexicoRights_Points_of_Diversion: A Shapefile containing the Points of Diversion for the entire state of New Mexico. OregonRights_state_shp: A folder containing 36 Shapefiles and are split between "pod" (Point of Diversion) and "pou" (Place of Use) for each water management basin within Oregon. In other words, each basin has one "pod" file and one "pou" file. The "pod" files are point shapes, and the "pou" files are polygons. UtahRights_Points_of_Diversion: A Shapefile containing the Points of Diversion for the entire state of Utah. WashingtonRights_WaterDiversions_ECY_NHD: A Geodatabase file containing both the Points of Diversion for the entire state of Washington. The name of the Feature Layer within the Geodatabase is "WaterDiversions_ECY_NHD". WyomingRights: A folder containing four subdirectories, one for each Wyoming Water Division. Each Division directory includes a varying number of subdirectories for each Wyoming Water District. Each District folder contains two copies of the Point of Diversion records for that area, with one copying being in CSV and one copy in Microsoft Excel XLS format.

Lisk, Matthew↗
DOE OSTI2023

HarDWR - Harmonized Water Rights Records

For a detailed description of the database of which this record is only one part, please see the HarDWR meta-record. Here we present a new dataset of western U.S. water rights records. This dataset provides consistent unique identifiers for each spatial unit of water management across the domain, unique identifiers for each water right record, and a consistent categorization scheme that puts each water right record into one of 7 broad use categories. These data were instrumental in conducting a study of the multi-sector dynamics of intersectoral water allocation changes through water markets (Grogan et al., in review). Specifically, the data were formatted for use as input to a process-based hydrologic model, WBM, with a water rights module (Grogan et al., in review). While this specific study motivated the development of the database presented here, U.S. west water management is a rich area of study (e.g., Anderson and Woosly, 2005; Tidwell, 2014; Null and Prudencio, 2016; Carney et al, 2021) so releasing this database publicly with documentation and usage notes will enable other researchers to do further work on water management in the U.S. west. The raw downloaded data for each state is described in Lisk et al. (in review), as well as here. The dataset is a collection of objects stored within an RDate file, stateWaterRightsHarmonized.RData. Each object either describes how one should interact with the other objects, or contains the harmonized water rights data. Here is each object described in detail: states: A character vector containing the state names for those states in which data was collected for. More importantly, the index of the state name is also the index in which that state's data can be found in the various following list objects. For example, if California is the third index in this object, the data for California will also be in the third index for each accompanying list. rightsByState_ground: A list of data frames with the cleaned ground water rights collected from each state. The column headers for each data frame are: waterRightID - The unique identifying ID of the water right, the same identifier as its state uses. basinNum - The alpha-numeric identifier of the WMA the record belongs to. priorityDate - The priority date associated with the right. allocatedFlow - The maximum flow of the allocation in cubic feet per second (ft3s-1). source - Whether the right is for surface water or groundwater. origWaterUse - The original stated water use from the state. waterUse - The water use category under the unified use categories established here. rightsByState_surface: A list of the cleaned surface water rights collected from each state. See rightsByState_ground for more details on how the data is formatted. fullRightsRecs: A list of the combined groundwater and surface water records for each state. Essentially, these lists are the merging of rightsByState_ground and rightsByState_surface by state. See rightsByState_ground for more details on how the data is formatted. projProj: The spatial projection used for map creation in the beginning of the project. Specifically, the World Geodetic System (WGS84) as a coordinate reference system (CRS) string in PROJ.4 format. wmaStateLabel: The name and/or abbreviation for what each state legally calls their WMAs. h2oUseByState: A list of spatial polygon data frames which contain the area(s) in which each water right is claimed to be used. It should be noted that not all water right records have a listed area(s) of use in this object. Currently, only Idaho and Washington provided valid data to be included in this object. h2oDivByState: A list of spatial points data frames which identifies the location of the Point of Diversion for the state's water rights. It should be noted that not all water right records have a listed Point of Diversion in this object. spatialWMAByState: A list of spatial polygon data frames which contain the spatial WMA boundaries for each state. The only data contained within the table are identifiers for each polygon. It is worth reiterating that Arizona is the only state in which the surface and groundwater WMA boundaries are not the same. wmaIDByState: A list which contains the unique ID values of the WMAs for each state. plottingDim: A character vector used to inform mapping functions for internal map making. Each state is classified as either "tall" or "wide", to maximize space on a typical 8x11 page.

Economics↗
DOE OSTI2020

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗
DOE OSTI2024

HarDWR - Harmonized Water Rights Records

For a detailed description of the database of which this record is only one part, please see the HarDWR meta-record. Here we present a new dataset of western U.S. water rights records. This dataset provides consistent unique identifiers for each spatial unit of water management across the domain, unique identifiers for each water right record, and a consistent categorization scheme that puts each water right record into one of 7 broad use categories. These data were instrumental in conducting a study of the multi-sector dynamics of intersectoral water allocation changes through water markets (Grogan et al., in review). Specifically, the data were formatted for use as input to a process-based hydrologic model, WBM, with a water rights module (Grogan et al., in review). While this specific study motivated the development of the database presented here, U.S. west water management is a rich area of study (e.g., Anderson and Woosly, 2005; Tidwell, 2014; Null and Prudencio, 2016; Carney et al, 2021) so releasing this database publicly with documentation and usage notes will enable other researchers to do further work on water management in the U.S. west. The raw downloaded data for each state is described in Lisk et al. (in review), as well as here. The dataset is a series of various files organized by state sub-directories. The first two characters of each file name is the abbreviation for the state the in which the file contains data for. After the abbreviation is the text which describes the contents of the file. Here is each file type described in detail: XXFullHarmonizedRights.csv: A file of the combined groundwater and surface water records for each state. Essentially, this file is the merging of XXGroundwaterHarmonizedRights.csv and XXSurfaceWaterHarmonizedRights.csv by state. The column headers for each of this type of file are: state - The name of the state the data comes from. FIPS - The two digit numeric state ID code. waterRightID - The unique identifying ID of the water right, the same identifier as its state uses. priorityDate - The priority date associated with the right. origWaterUse - The original stated water use from the state. waterUse - The water use category under the unified use categories established here. source - Whether the right is for surface water or groundwater. basinNum - The alpha-numeric identifier of the WMA the record belongs to. CFS - The maximum flow of the allocation in cubic feet per second (ft3s-1). Arizona is unique among the states, as its surface and groundwater resources are managed with two different sets of boundaries. So, for Arizona, the basinNum column is missing and instead there are two columns: surBasinNum - The alpha-numeric identifier of the surface water WMA the record belongs to. grdBasinNum - The alpha-numeric identifier of the groundwater WMA the record belongs to. XXStatePOD.shp: A shapefile which identifies the location of the Points of Diversion for the state's water rights. It should be noted that not all water right records in XXFullHarmonizedRights.csv have coordinates, and therefore may be missing from this file. XXStatePOU.shp: A shapefile which contains the area(s) in which each water right is claimed to be used. Currently, only Idaho and Washington provided valid data to included within this file. XXGroundwaterHarmonizedRights.csv: A file which contains only harmonized groundwater rights collected from each state. See XXFullHarmonizedRights.csv for more details on how the data is formatted. XXSurfaceWaterHarmonizedRights.csv: A file which contains only harmonized surface water rights collected from each state. See XXFullHarmonizedRights.csv for more details on how the data is formatted. Additionally, one file, stateWMALabels.csv, is not stored within a sub-directory. While we have referred to the spatial boundaries that each state uses to manage its water resources as WMAs, this term is not shared across all states. This file lists the proper name for each boundary set, by state.

Economics↗
DOE OSTI2022

Leaf gas exchange raw data, 2015 - 2017, at Manaus, Brazil

This data package contains raw leaf gas exchange data from the NGEE Tropics K34 tower site on a plateau near Manaus, Brazil. Data was collected using sensor and leaf sample measurements. Leaves on a target tree were chosen by accessing a branch near the tower. In some cases, the branch was naturally positioned near the tower and in other cases, a small rope was used to pull the branch closer to the tower for better access to leaves. Mature leaves were targeted in nearly all cases. The gas exchange system was hauled with a climbing rope to the level of the branch on the tower and the sensor head/leaf enclosure was installed near the branch of interest using a magic arm. We measured leaf gas exchange from 5:00 AM to 5:00 PM, randomly cycling through the approximately 20 leaves in the targeted branch(es). Data were collected over two time periods, detailed within the files. The attached data files contain raw data, sensor manuals, and PDFs with detailed methods and procedures separated into multiple folders. Data is in Excel file format. See data references for data package containing related metadata. This dataset replaces the leaf gas exchange data of two retired packages, NGT0019 and NGT0040.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

SPRUCE Whole Ecosystem Warming (WEW) Environmental Data and Water Table Summaries, Marcell Experimental Forest, Minnesota, 2015-2024

This data set contains observations of photosynthetically active radiation (PAR), precipitation, soil temperature, soil volumetric water content, air temperature, relative humidity, and normalized water table depth that are summarized on a daily, weekly, monthly, and annual basis for each of the SPRUCE plots. Observations span 2015-2024. This dataset draws on several datasets (Hanson et al. 2016; Hanson et al. 2020; and Warren, unpublished data) and compiles these environmental observations into useful formats for data analysis. These environmental metrics can be used to understand the environmental conditions inside SPRUCE environmental chambers throughout the durations of the experiment and can be paired with other data for modeling and analysis. R code used to generate these files is provided as part of the data package. This dataset contains four data files in comma separate (.csv) format and a compressed folder (*.zip) containing three R (*.r) scripts. Additional metadata are provided: one data dictionary and a file-level metadata file in comma separate (.csv) format and a user guide in PDF (*.pdf) format. User note: Users must cite the original dataset/s along with this dataset when publishing any analyses using this dataset. Details on the dataset used to compile each variable are available in the header row of the files and in the user guide.

air temperature↗