SEARCH · Search NASA

Results for “PostgreSQL”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

21 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2025

NLR HPC Eagle Jobs Data and Additional Energy Metrics

Overview: Anonymized job-level records from the Eagle high-performance computing (HPC) system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, resource utilization, CPU/GPU energy consumption, and efficiency metrics. Sensitive fields (user, account, job name) are replaced with cryptographic hashes. System & Timeframe: Eagle was a 2,000-node, 8-petaflop system operated at NLR from 2019–2024. Data covers the full operational lifetime of the system. Slurm data was processed nightly; timestamps are in Mountain Time. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.eagle.job-anon.zip — Core anonymized job records (Hive-partitioned Parquet) esif.hpc.eagle.job-anon-energy-metrics.zip — Same records with additional iLO and Ganglia energy metrics datacard.md — Full dataset documentation ~13.8 million rows, 62 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct through a pipeline: Eagle Jobs API → Redpanda → StreamSets → HPCMON API → PostgreSQL. Node-level power from iLO (HP Integrated Lights-Out); GPU power from Ganglia monitoring, joined to jobs via node lists and time ranges. Preprocessing: Anonymization of name, user, and account fields via cryptographic hashing Derived columns: queue_wait, cpu_eff, max_mem_eff Simplified job state mapping (e.g., "CANCELLED BY 12345" → "CANCELLED") QoS accounting rules (buy-in, standby, or Slurm QoS value) CPU energy estimated from TDP (200W, Intel Xeon Gold 6154, 18 cores) Timezone-aware columns (_tz) sourced from LEX accounting database to correctly handle DST transitions Key Variables: Scheduling: job_id, partition, state_simple, submit_time_tz, start_time_tz, end_time_tz, queue_waitResources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, node_energy_total_watt_hours (iLO), gpu0/1_energy_total_watt_hours (Ganglia) Partitions: bigmem, bigmem-8600, bigscratch, csc, dav, ddn, debug, gpu, haswell, long, mono, short, standard Job States: CANCELLED, COMPLETED, FAILED, NODE_FAIL, OUT_OF_MEMORY, PENDING, RUNNING, TIMEOUT QoS Levels: Unknown, normal, buy-in, debug, penalty, high, standby Important Notes: Non-_tz timestamp columns may be off by one hour across DST boundaries; use _tz columns for time difference calculations Energy fields are null for jobs without monitoring coverage Job step records and raw Slurm JSONB fields are excluded from this extract Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

GBCGE Subsurface Database Explorer and APIs

This submission defines a DOI for the Great Basin Center for Geothermal Energy's (GBCGE) Subsurface Database Explorer web application and underlying data services, and acknowledges the INGENIOUS project as a major source of funding for data compilation and quality assurance. The GBCGE Subsurface Database Explorer is an interactive web mapping application that provides public access to the GBCGE Subsurface Database, and its collection of datasets pertinent to geothermal exploration, oil and gas exploration, critical mineral exploration, and other subsurface characterization for the Great Basin Region, western US. This is a living database, and will be continuously updated with new data and datasets as funding and motivations allow. The underlying database views that populate the web application are on an automated refresh schedule. Data sources and acknowledgements: We thank our partners with the Nevada Division of Minerals (NDOM), the Southern Methodist University (SMU), and Great Basin State Geological Surveys for their active efforts in data curation, schema design, and quality assurance. We also thank contributors among the USGS, Oregon Institute of Technology, State Divisions of Water Resources, State Divisions of Oil, Gas, and Minerals, and State Geological Surveys for open data availability and direct contributions made under the National Geothermal Data System (NGDS).

15 GEOTHERMAL ENERGY↗

DOE OSTI2019

Fascinating Reporting with Postgres psql and sendmail

Among many things, psql, the Postgres command line tool, can create decent output to share with coworkers and customers, and that is where many Postgres users stop. The extreme limits of psql's formatting will be explored, delving into Unicode and HTML reports and the backslash/meta commands. Pretty reports are useless unless good data can be reported. The past and current developers of Postgres have packed a wonderful set functionality into the query language such as array aggregation and un-nesting. The staggering usefulness of "DISTINCT ON ()" will also be shown (without stepping into the quagmire of window functions). Environmentally conscious developers don't want to dump endless reports onto paper, so the power of psql is paired with the power of the UNIX/Linux sendmail command to generate electronic reports to send to coworkers and customers. The icing on the cake will be an example set of simple database tables to manage recipients of an automated reporting system.

97 MATHEMATICS AND COMPUTING↗