Search NASA⌕ Search

SEARCH · Search NASA

Results for “parquet”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

51 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Electricity Baseline 2021 Background Data and Log File

The ElectricityLCI v2 Python package (https://github.com/USEPA/ElectricityLCI/tree/v2.0) was used to generate the 2021 electricity baseline: a regionalized life cycle inventory model of U.S. electricity generation, consumption, and distribution using standardized facility and generation data. ElectricityLCI implements a local data store for downloading and accessing public data on an individual's computer. The data store follows the folder definition provided by USEPA's esupy Python package (https://github.com/USEPA/esupy), which utilizes the appdirs Python dependency (https://pypi.org/project/appdirs/). An overview of the ElectricityLCI data stores may be found on the README (https://github.com/USEPA/ElectricityLCI/blob/v2.0/README.md#data-store). This submission includes the background data used to generate the 2021 electricity baseline inventory. Each zip archive stores the source files as found in their data stores. Sub-folders in each of the data stores are archived separately. For example, stewi.zip contains the JSON files, while stewi.facility.zip is the 'facility' sub-folder of stewi data store that stores the parquet files. To reproduce the data store, extract each zip file and drag-and-drop sub-folders in to their appropriate root folders to recreate the data stores, then copy the root folders to your data store folder (as returned by running the following on the command line: python -c "import appdirs; print(appdirs.user_data_dir())"). The main five data stores include: 'electricitylci', 'facilitymatcher', 'fedelemflowlist', 'stewi', and 'stewicombo'. The log file generated by the 2021 model run is also included, which contains the statements at the DEBUG level and above.

Electricity; LCA; LCI; Life Cycle; data inventory↗
DOE OSTI2025

Electricity Baseline 2020 Background Data and Log File

The ElectricityLCI v2 Python package (https://github.com/USEPA/ElectricityLCI/tree/v2.0) was used to generate the 2020 electricity baseline: a regionalized life cycle inventory model of U.S. electricity generation, consumption, and distribution using standardized facility and generation data. ElectricityLCI implements a local data store for downloading and accessing public data on an individual's computer. The data store follows the folder definition provided by USEPA's esupy Python package (https://github.com/USEPA/esupy), which utilizes the appdirs Python dependency (https://pypi.org/project/appdirs/). An overview of the ElectricityLCI data stores may be found on the README (https://github.com/USEPA/ElectricityLCI/blob/v2.0/README.md#data-store). This submission includes the background data used to generate the 2020 electricity baseline inventory. Each zip archive stores the source files as found in their data stores. Sub-folders in each of the data stores are archived separately. For example, stewi.zip contains the JSON files, while stewi.facility.zip is the 'facility' sub-folder of stewi data store that stores the parquet files. To reproduce the data store, extract each zip file and drag-and-drop sub-folders in to their appropriate root folders to recreate the data stores, then copy the root folders to your data store folder (as returned by running the following on the command line: python -c "import appdirs; print(appdirs.user_data_dir())"). The main five data stores include: 'electricitylci', 'facilitymatcher', 'fedelemflowlist', 'stewi', and 'stewicombo'. The log file generated by the 2020 model run is also included, which contains the statements at the DEBUG level and above.

Electricity; LCA; LCI; Life Cycle; data inventory↗
DOE OSTI2021

Machine Learning Guided Operational Intelligence from Synchrophasors (Final Report)

Schweitzer Engineering Laboratories (SEL) and Oregon State University (OSU) received over 27 terabytes of electrical power system phasor measurement unit (PMU) data for the Eastern, Western, and ERCOT interconnections. The dataset includes measurements spread across 446 PMUs from early 2016 to mid 2018 depending on the interconnect. The full dataset was split into a training and test (holdout) dataset by PNNL. All data was received in the Apache Parquet format. The overarching goal of this project is to develop and execute a strategy to mitigate data anomalies, perform analysis on the dataset, and detect anomalous events in the data.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

On the Existence of the Kolmogorov Inertial Range in the Terrestrial Magnetosheath Turbulence

In the solar wind, power spectral density (PSD) of the magnetic field fluctuations generally follow the so-called Kolmogorov spectrum f {sup −5/3} in the inertial range, where the dynamics is thought to be dominated by nonlinear interactions between counter-propagating incompressible Alfvén wave parquets. These features are thought to be ubiquitous in space plasmas. The present study gives a new and more complex picture of magnetohydrodynamic (MHD) turbulence as observed in the terrestrial magnetosheath. The study uses three years of in situ data from the Cluster mission to explore the nature of the magnetic fluctuations at MHD scales in different locations within the magnetosheath, including flanks and subsolar regions. It is found that the magnetic field fluctuations at MHD scales generally have a PSD close to f {sup −1} (shallower than the Kolmogorov one f {sup −5/3}) down to the ion characteristic scale, which recalls the energy-containing scales of solar wind turbulence. The Kolmogorov spectrum is observed only away from the bow shock toward the flank and the magnetopause regions in 17% of the analyzed time intervals. Measuring the magnetic compressibility, it is shown that only a fraction (35%) of the observed Kolmogorov spectra was populated by shear Alfvénic fluctuations, whereas the majority of the events (65%) was found to be dominated by compressible magnetosonic-like fluctuations, which contrasts with well-known turbulence properties in the solar wind. This study gives a first comprehensive view of the origin of the f {sup −1} and the transition to the Kolmogorov inertial range; both questions remain controversial in solar wind turbulence.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

JHTDB-wind: a web-accessible large-eddy simulation database of a wind farm with virtual sensor querying

This paper introduces JHTDB-wind (https://turbulence.idies.jhu.edu/datasets/windfarms, last access: 11 November 2025), a publicly accessible database containing large-eddy simulation (LES) data from wind farms. Building on the framework of the Johns Hopkins Turbulence Database (JHTDB), which hosts direct numerical simulation (DNS) and some LES datasets of canonical turbulent flows, JHTDB-wind stores the 4D space–time history of the flow and provides users the ability to access and query the data via a web-based virtual sensor interface. The initial dataset comprises LES results from a large wind farm with 10×6 turbines, modeled using a filtered actuator line method, under conventionally neutral atmospheric conditions. These data comprise 1 h (hour) of flow field data (velocity, pressure, potential temperature deviation, subgrid-scale (SGS) eddy viscosity, and turbine forces, approximately 15 TB (terabytes) and wind turbine data – including both turbine-level operational quantities and blade-level aerodynamic quantities (approximately 1.3 TB) – stored in Zarr and Parquet formats, respectively. Data retrieval is facilitated by the giverny Python package, allowing remote users to query the database in Python or MATLAB (C and Fortran support are available for flow field data). This paper details the simulation setup and demonstrates data access through examples that analyze wind farm flow structures and turbine performance. The framework is extensible to future datasets, including the JHTDB-wind diurnal cycle simulation analyzed in Xiao et al. (2025).

17 WIND ENERGY↗
DOE OSTI2024

Gold-et-al_2024_EarthsFuture

Results from Gold et al. submitted to Earth's Future. All code to reproduce the experiment and make the figures can be found here: https://github.com/davidfgold/Gold-etal_2024_EarthsFuture For a detailed guide to data in this repository, see the README.txt file. Shortage output from .xdd files generated by StateMod was compressed into .parquet files. Reservoir output from .xre files can be found in the "Reservoir" directory. We adopted the CDSS naming convention: cm = Upper Colorado River Basin gm = Gunnison River Basin ym = Yampa River Basin wm = White River Basin sj = Southwest Basin

Climate Change↗
DOE OSTI2025

Data for Unraveling the Drivers of Water Shortage across Spatial Scales and Sectors in Colorado's West Slope River Basins

Results from Sunkara et al. submitted to Earth's Future. All code to reproduce the experiment and make the figures can be found here: https://github.com/ssaiveena/Sunkara-etal_2025_WestSlope For a detailed guide to data in this repository, see the README.txt file. Shortage output from .xdd files generated by StateMod was compressed into .parquet files in the Uncertain_scenarios folder. Reservoir output from .xre files can be found in the "Reservoir" directory within the Uncertain_scenarios folder. We adopted the CDSS naming convention: cm = Upper Colorado River Basin gm = Gunnison River Basin ym = Yampa River Basin wm = White River Basin sj = Southwest Basin

Climate Change↗
DOE OSTI2020

dGen (Distributed Generation Market Demand) Model Data: Alpha Release

Open sourced data needed to run the basic alpha release version of the dGen model. Includes a pre-generated agent file of 100,000 agents in pickle file format along with the base schema and table data in parquet format that are needed to create a postgreSQL database for the model to interact with.

14 SOLAR ENERGY↗
DOE OSTI2025

NLR HPC Facility Power Usage Effectiveness (PUE) Data

Timeseries of Energy Systems Integration Facility (ESIF) Data Center Power Usage Effectiveness (PUE) Data provided in Parquet and compressed CSV formats Power Metrics Timeseries Fields: ts: Timestamp cooling_kw: Cooling (kilowatts) - Captures the power used by fans and pipe trace heaters associated with outdoor cooling equipment. The dedicated tower filter pump power is also captured as cooling load. energy_reuse: Energy Reuse Effectiveness hvac_kw: Heating, ventilation, and air conditioning (kilowatts) - Captures fan walls, fan coils that support the data center electrical rooms, and the make-up air unit. it_power_kw: IT equipment (kilowatts) - Captures power used by the IT equipment on the data center floor. plug_and_light_kw: Lights and utility plugs (kilowatts) - Captures power associated with the data center and dedicated mechanical room. The crank-case heater for the emergency standby generator is also captured as light and plug load. pue: Power Usage Effectiveness pump_kw: Pumps (kilowatts) - Captures power from pumps that move water in the data center Energy Recover Water loop and the Tower Water loops, and also captures power used by the boost pumps that circulate water through the fan walls. Note: The tower filter pump runs constantly to filter water from the data center cooling tower system, so 2.67 kilowatts are attributed to this pump and that is not reflected in this data field. day: Day of month Outside Weather Station Timeseries Fields: ts: Timestamp outside_air_humidity: Outside air humidity - Relative humidity percent outside_air_temp: Outside air temperature - Degrees Fahrenheit day: Day of month More detail: High-Performance Computing Data Center Power Usage Effectiveness

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Outdoor Deployment Data for a Four-Terminal GaAs//Si Tandem Solar Mini-Module

This dataset contains the complete outdoor measurement and analysis data for a mechanically stacked, four-terminal (4T) gallium arsenide (GaAs)//silicon (Si) tandem solar mini-module deployed from October 2019 to January 2021 at the Solar Radiation Research Laboratory (SRRL) in Golden, Colorado, USA. The data support a performance modeling and degradation analysis framework for tandem photovoltaic devices, as described in the accompanying publication. The dataset includes: (1) current–voltage (J–V) characteristics of each sub-cell measured approximately every five minutes, with extracted performance parameters; (2) spectral irradiance from an EKO MS-710 WISER spectroradiometer, along with derived spectral mismatch ratios (SMR) and average photon energy (APE); (3) one-minute resolution meteorological data from the co-located SRRL weather station and GPS-derived precipitable water vapor (PWV); (4) pre-deployment laboratory characterization (external quantum efficiency, J–V curves, standard test conditions parameters); (5) outdoor-extracted temperature and PWV correction coefficients; and (6) PVcircuit equivalent-circuit simulation outputs used for model validation. Degradation rates of −4.1 ± 0.2 %/year (GaAs) and −2.5 ± 0.9 %/year (Si) were determined using a filtering and normalization methodology adapted for fixed-tilt tandem modules. All data are provided in open, portable formats (Apache Parquet, CSV, JSON) to enable full reproducibility of the published analysis.

14 SOLAR ENERGY↗
NASA NTRS1986

Stress on the surface of Venus: Analysis of data provided by Venera-15 and Venera-16

There are no large-scale sedimentation and erosion processes on Venus, therefore the observed deformation must be formed on the surface or close to it, and in the ellipsoids of stresses one of the main axes must be perpendicular to the surface. According to the deformation visible on photography from Venera-15, 16, the orientations of the ellipsoids of stresses are established in the different sections of the Montes Maxwell and the northern parquet. A geological model is established, according to which the Montes Maxwell is an arched vault and the region to the south a conjugate bend, separated by a shift zone.

Ronka, L. B.↗
NASA NTRSDate not supplied

Cloud Optimized Data Formats

Cloud computing offers the promise of being able to analyze Big Data earth Observations at scale, by allowing scientists to deploy many nodes at once to analyze the data. However, in order to take full advantage of cloud scalability, it is often necessary to reorganize and reformat the data to enable fine-grained, parallel access to the data in Web Object Storage. NASA recently conducted a study of several formats that are optimized for analysis in the cloud: Parquet, zarr, HDF (Hierarchical Data Format) in the Cloud, and Cloud-Optimized GeoTIFF (Tagged Image File Format). They were compared against non-cloud-optimized formats, netCDF (network Common Data Form) and GeoTIFF, with criteria based both on stewardship and analysis performance.

Christopher Lynnes↗
DOE OSTI2026

Machine learning for slow-extraction uniformity at the Fermilab Delivery Ring

The Muon-to-Electron Conversion Experiment (Mu2e) requires uniform slow extraction from the Fermilab Delivery Ring during each 43 ms spill. This project analyzed 13,737 spills from seven parquet files collected during March and June 2026 commissioning. Each spill contained 430 samples at 0.1 ms resolution and was summarized by the Spill Duty Factor. Null Action and active proportional-integral-derivative conditions established operational baselines. Fixed-ramp sessions were segmented to evaluate reproducibility, drift, and spill-to-spill spread, while a deliberate quadrupole-ramp step identified the beam-response peak and trough. Median profiles were reproducible within fixed conditions but drifted over multi-hour periods. The resulting baselines and response features support future physics-based and machine-learning approaches to real-time spill regulation.

Prescott, Matthew J. [Purdue U., West Lafayette; F↗
DOE OSTI2022

Beyond quantum cluster theories: multiscale approaches for strongly correlated systems

The degrees of freedom that confer to strongly correlated systems their many intriguing properties also render them fairly intractable through typical perturbative treatments. For this reason, the mechanisms responsible for their technologically promising properties remain mostly elusive. Computational approaches have played a major role in efforts to fill this void. In particular, dynamical mean field theory and its cluster extension, the dynamical cluster approximation have allowed significant progress. However, despite all the insightful results of these embedding schemes, computational constraints, such as the minus sign problem in quantum Monte Carlo (QMC), and the exponential growth of the Hilbert space in exact diagonalization (ED) methods, still limit the length scale within which correlations can be treated exactly in the formalism. A recent advance aiming to overcome these difficulties is the development of multiscale many body approaches whereby this challenge is addressed by introducing an intermediate length scale between the short length scale where correlations are treated exactly using a cluster solver such QMC or ED, and the long length scale where correlations are treated in a mean field manner. At this intermediate length scale correlations can be treated perturbatively. This is the essence of multiscale many-body methods. Furthermore, we will review various implementations of these multiscale many-body approaches, the results they have produced, and the outstanding challenges that should be addressed for further advances.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗