SEARCH · Search NASA

Results for “Building benchmarking models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Reduced-order modeling for efficient cross section library development in high-temperature gas reactor pebble-bed depletion analysis

Accurate modeling of running-in and equilibrium conditions in pebble-bed reactors (PBRs) requires precise microscopic multigroup neutron cross sections. In Griffin, deterministic neutronics calculations rely on multivariate interpolation over large cross section libraries, resulting in significant memory usage and performance bottlenecks. This work, together with a companion paper on Griffin integration, explores reduced-order models (ROMs) to replace interpolation with lightweight surrogates. Several ROM techniques are benchmarked, with deep neural networks (DNNs) demonstrating superior memory efficiency, scalability, and predictive accuracy. A total of 295 DNNs were trained to build a comprehensive isotope library, integrated into Griffin through a custom LibTorch interface for depletion analysis. Initial results demonstrate that DNN-based ROMs drastically reduce memory demands while preserving accuracy, enabling finer tabulations and additional state variables without overhead. In conclusion, the framework also supports online cross section generation and real-time DNN updates through transfer learning, improving fidelity by capturing self-shielding and evolving nuclide compositions during burnup.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

BuildingsBench: A Benchmark for Universal Building Load Forecasting [SWR-23-51]

The residential and commercial building stock in the United States is responsible for a significant percentage of energy consumption and greenhouse gas emissions. Electrification of end-uses, as well as decarbonizing the electrical grid through renewable energy sources such as solar and wind, constitutes the pathway to zero-emission buildings. Forecasting day-ahead building energy consumption is an integral part of this solution. Currently, specialized forecasting models are hand-made for each individual building, which is time-consuming, expensive, and leads to duplicated efforts. BuildingsBench is a Python software framework for training and comparing generalized machine learning models for universal building load forecasting. This challenge tasks a single foundational model to generalize its forecasts for a wide variety of buildings, across geographic regions, building types, weather patterns, and more. This software provide code for pre-training such models and subsequently evaluating their performance on a suite of hundreds of diverse real and synthetic buildings. BuildingsBench is a platform for: - Large-scale pretraining with the synthetic Buildings-900K dataset for short-term load forecasting (STLF). Buildings-900K is statistically representative of the entire U.S. building stock and is extracted from the NREL End-Use Load Profiles database. - Benchmarking on two tasks evaluating generalization: zero-shot STLF and transfer learning for STLF. We provide an index-based PyTorch Dataset for large-scale pretraining, easy data loading for multiple real building energy consumption datasets as PyTorch Tensors or Pandas DataFrames, simple (persistence) to advanced (transformer) baselines, metrics management, and more.

Emami, Patrick↗

DOE OSTI2022

Development of a Annual Air Handling Unit Fault Dataset for FDD Tools: Lessons Learned and Considerations for FDD Developers

As energy management and information systems (e.g., automated fault detection and diagnostics [AFDD] tools) become more prevalent in the commercial building stock, it is important to determine the effectiveness of these technologies by benchmarking their performance. The authors have been working to develop the largest publicly available dataset of HVAC fault data for performance benchmarking applications, covering the most common HVAC systems and designs including chiller plants, rooftop packaged units, dual duct air handling units and single duct air handling units. This study covers the development, modeling, and validation of a synthetic fault dataset for a single duct air handling unit (AHU), one of the most common HVAC configurations found in the commercial building stock. Despite this being a common system, real-world time series data are scarce and usually do not span a wide range of weather conditions. Due to this limitation, a detailed AHU model was employed to carry out annual simulations of numerous common sensor and mechanical faults, which were then validated by comparing their effects on system performance to expected symptoms. We summarize the nature of each fault and their impacts under different weather and operation conditions. Finally, we highlight considerations for FDD developers that may want to use this dataset to assess their algorithms’ performance and their improvement over time.

Casillas, Armando↗

DOE OSTI2020

GrAMME: Semisupervised Learning Using Multilayered Graph Attention Models

Modern data analysis pipelines are becoming increasingly complex due to the presence of multiview information sources. While graphs are effective in modeling complex relationships, in many scenarios, a single graph is rarely sufficient to succinctly represent all interactions, and hence, multilayered graphs have become popular. Though this leads to richer representations, extending solutions from the single-graph case is not straightforward. Consequently, there is a strong need for novel solutions to solve classical problems, such as node classification, in the multilayered case. In this article, we consider the problem of semisupervised learning with multilayered graphs. Though deep network embeddings, e.g., DeepWalk, are widely adopted for community discovery, we argue that feature learning with random node attributes, using graph neural networks, can be more effective. To this end, we propose to use attention models for effective feature learning and develop two novel architectures, GrAMME-SG and GrAMME-Fusion, that exploit the interlayer dependences for building multilayered graph embeddings. Using empirical studies on several benchmark data sets, we evaluate the proposed approaches and demonstrate significant performance improvements in comparison with the state-of-the-art network embedding strategies. The results also show that using simple random features is an effective choice, even in cases where explicit node attributes are not available.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Observations of airflow around a supertall curved building and its impact on temperature and humidity in Houston's urban center

Characterization of realistically shaped skyscrapers embedded in non-uniform neighbourhoods experiencing intricate weather patterns remains inadequately investigated. Aiming to close this gap, the Center for Multiscale Applied Sensing team deployed its mobile observatory in the street canyons around the curved Wells Fargo Plaza skyscraper in downtown Houston, TX. Three deployments allowed airflow observations under different inflow wind and thermodynamic stability conditions. Doppler lidar measurements reveal that when inflow hits the curved wall of the skyscraper, perpendicular canyons experience similar vortex configurations creating two windward and two leeward circulations. Windsond measurements support that buoyancy within the deep street canyons can generate thermal updrafts as strong as 2 m s -1 which is sufficient to overturn the mechanical downwash under gentle wind conditions. Canyons experiencing venting during the daytime were observed to be more thermodynamically stable at night while thermodynamically stable canyons during the day were observed to be more thermodynamically unstable at night owing to the accumulation of heat near street-level. Fourier decomposition of the vertical velocity measurements shows that in all cases flow exhibited high Reynolds numbers and was composed of turbulent eddies of predominantly 6 and 15 min periods. Here, this observational dataset provides insights to assess wind load, pedestrian comfort, urban air mobility, and natural ventilation and may be used as a benchmark for numerical model and wind tunnel studies attempting to represent realistically complex urban conditions.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Integral Nuclear Data and Benchmarking Needs for Fusion Energy Systems

Fusion energy systems are currently being designed and optimized using radiation transport codes. To deal with the unique environment inside a fusion-based system, many of these designs incorporate novel materials able to withstand the high radiation fields, ensure adequate cooling and thermal protection, and produce tritium. Validation plays a vital role in building trust in the predictive power of these models and computational methods. Validation of a code consists of modeling documented real-world experiments and comparing the code-predicted response to the measured response. Adequate validation requires measured responses from real-world experiments, also known as integral data, that mimic the system being designed, including materials, impinging radiation, and temperature, among other variables. The most trusted integral data are experimental responses that have been through a rigorous benchmarking process that develops a recommended computational model and evaluates all experimental uncertainties. Finally, there are a few research groups around the world that have been producing integral data for fusion applications, but a substantial investment is needed to address the unique validation needs of the fusion community.

Fusion↗

DOE OSTI2022

Distributed Wind Energy Futures Study

To better understand distributed wind opportunities in the United States, researchers at the U.S. Department of Energy's (DOE's) National Renewable Energy Laboratory explored cost, performance, and valuation benchmarks necessary for distributed wind to achieve widespread commercial viability in the United States by 2035. Building on DOE's prior benchmark report "Assessing the Future of Distributed Wind: Opportunities for Behind-the-Meter Projects", the new Distributed Wind Futures Study employs higher-resolution data and new modeling techniques to highlight geographic trends in technical and economic potential and to compare the cost and performance of distribution-connected, megawatt-scale wind applications (referred to as "front-of-the-meter" applications) and behind-the-meter systems, thereby informing the case for investment. The report identifies the best locations and sectors for behind-the-meter and front-of-the-meter applications and provides data to inform the trade-offs consumers face when deciding between distributed wind and solar photovoltaic (PV).

17 WIND ENERGY↗

DOE OSTI2022

Distributed Wind Energy Futures Study [Slides]

17 WIND ENERGY↗

DOE OSTI2022

Distinctive signals of frustrated dark matter

We study a renormalizable model of Dirac fermion dark matter (DM) that communicates with the Standard Model (SM) through a pair of mediators — one scalar, one fermion — in the representation (6, 1, $\frac{4}{3}$) of the SM gauge group SU(3) c x SU(2) L x U(1) Y . While such assignments preclude direct coupling of the dark matter to the Standard Model at tree level, we examine the many effective operators generated at one-loop order when the mediators are heavy, and find that they are often phenomenologically relevant. We reinterpret dijet and pair-produced resonance and jets + E$^{miss}_{T}$ searches at the Large Hadron Collider (LHC) in order to constrain the mediator sector, and we examine an array of DM constraints ranging from the observed relic density Ω χ h$^{2}_{Planck}$ to indirect and direct searches for dark matter. Tree-level annihilation, available for DM masses starting at the TeV scale, is required in order to produce Ω χ h$^{2}_{Planck}$ through freeze-out, but loops — led by the dimension-five DM magnetic dipole moment — are nonetheless able to produce signals large enough to be constrained, particularly by the XENON1T experiment. In some benchmarks, we find a fair amount of parameter space left open by experiment and compatible with freeze-out. In other scenarios, however, the open space is quite small, suggesting a need for further model-building and/or non-standard cosmologies.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2024

Machine learning in materials research: Developments over the last decade and challenges for the future

The number of studies that apply machine learning (ML) to materials science has been growing at a rate of approximately 1.67 times per year over the past decade. In this review, I examine this growth in various contexts. First, I present an analysis of the most commonly used tools (software, databases, materials science methods, and ML methods) used within papers that apply ML to materials science. The analysis demonstrates that despite the growth of deep learning techniques, the use of classical machine learning is still dominant as a whole. It also demonstrates how new research can effectively build upon past research, particular in the domain of ML models trained on density functional theory calculation data. Next, I present the progression of best scores as a function of time on the matbench materials science benchmark for formation enthalpy prediction. In particular, a dramatic improvement of 7 times reduction in error is obtained when progressing from feature-based methods that use conventional ML (random forest, support vector regression, etc.) to the use of graph neural network techniques. Finally, I provide views on future challenges and opportunities, focusing on data size and complexity, extrapolation, interpretation, access, and relevance.

36 MATERIALS SCIENCE↗

DOE OSTI2021

URANIUM HOLDUP MONITORING WITH COMPTON IMAGING AS FUNCTION OF DEPTH AND MASS

Mass quantification using standoff gamma detection is a logical progression for MC&A applications in uranium processing facilities. While obtaining a ground truth from physical collection and weighing of masses is ideal and routine, it poses a logistical problem from both a standpoint of ALARA programs as well as interruptions to operations. The standard approach is the Generalized Geometry in Holdup (GGH) method that can apply for many cases where geometries of the holdup match simulated models; however, there are many cases where a model may not exist or a feature of the holdup geometry is such that it does not match a model particularly well.1 Gamma imaging at standoff distances and outside process equipment has potential to address these shortcomings and improve mass estimations while reducing personnel exposure; however, algorithms still require a significant degree of benchmarking to build sufficient confidence in algorithms before displacing GGH in some procedures. This paper presents the creation of a set of laboratory standards of depleted U3O8 in fixed size containers covering a range of masses and effective thicknesses up to the infinite thickness for the 185-keV line. These blocks are identically sized aluminum boxes loaded such that a thin, flat face is placed towards the imager, while the back side contains closed cell foam to apply pressure to keep the powder in a fixed position at the front face as the sources are manipulated for measurements. Included is an analysis of the mass estimation using a commercially available Compton imaging instrument where multiple geometries are created showing limitations in current methodologies.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2026

Building Energy Analysis of Manufactured and Multifamily Housing Types in Juneau, Alaska

This report details the results of building energy modeling analysis evaluating the potential energy savings, economic outcomes, and grid-level electricity reduction associated with cold climate air source heat pump (ccASHP) adoption across multifamily and manufactured housing (MMFH) building typologies in the City and Borough of Juneau (CBJ). Three building archetypes were evaluated: multifamily 4-plex apartments, multifamily 8-plex apartments, and manufactured housing units. Building energy models were developed using OpenStudio-HPXML and calibrated to actual utility consumption data and local meteorological data from the Juneau International Airport weather station using an automated calibration tool following the BPI-2400-S-2015 v.2 standard for model calibration. Occupant behaviors present the greatest variability in successful calibrations. Calibrated models were benchmarked against a baseline electric resistance heating condition, with the selected ccASHP modeled as the retrofit condition and typical meteorological year weather data for all results generation.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2025

Towards excitations and dynamical quantities in correlated lattices with density matrix embedding theory

Density matrix embedding theory (DMET) provides a framework to describe ground-state expectation values in strongly correlated systems, but its extension to dynamical quantities is still an open problem. We show one route to obtaining excitations and dynamical spectral functions by using the techniques of DMET to approximate the matrix elements that arise in a single-mode inspired excitation ansatz. We demonstrate this approach in the one-dimensional Hubbard model, comparing the neutral excitations, single-particle density of states, charge, and spin dynamical structure factors to benchmarks from the Bethe ansatz and density matrix renormalization group. Finally, our work highlights the potential of these ideas in building computationally efficient approaches for dynamical quantities.

1-dimensional systems↗

DOE OSTI2026

Performance-Aligned LLMs for Generating Fast HPC Code

Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. Here, we demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP parallel code.

Computer science↗

DOE OSTI2024

Comparative Study of Large Language Model Architectures on Frontier

Large language models (LLMs) have garnered significant attention in both the AI community and beyond. Among these, the Generative Pre-trained Transformer (GPT) has emerged as the dominant architecture, spawning numerous variants. However, these variants have undergone pre-training under diverse conditions, including variations in input data, data preprocessing, and training methodologies, resulting in a lack of controlled comparative studies. Here we meticulously examine two prominent open-sourced GPT architectures, GPT-NeoX and LLaMA, leveraging the computational power of Frontier, the world’s first Exascale supercomputer. Employing the same materials science text corpus and a comprehensive end-to-end pipeline, we conduct a comparative analysis of their training and downstream performance. Our efforts culminate in achieving state-of-the-art performance on a challenging materials science benchmark. Furthermore, we investigate the computation and energy efficiency, and propose a computationally efficient method for architecture design. To our knowledge, these pre-trained models represent the largest available for materials science. Our findings provide practical guidance for building LLMs on HPC platforms.

Yin, Junqi↗

DOE OSTI2020

Efficient, Predictive Tomography of Multi-Qubit Quantum Processors

After decades of R&D, quantum computers comprising more than 2 qubits are appearing. If this progress is to continue, the research community requires a capability for precise characterization (“tomography”) of these enlarged devices, which will enable benchmarking, improvement, and finally certification as mission-ready. As world leaders in characterization -- our gate set tomography (GST) method is the current state of the art – the project team is keenly aware that every existing protocol is either (1) catastrophically inefficient for more than 2 qubits, or (2) not rich enough to predict device behavior. GST scales poorly, while the popular randomized benchmarking technique only measures a single aggregated error probability. This project explored a new insight: that the combinatorial explosion plaguing standard GST could be avoided by using an ansatz of few-qubit interactions to build a complete, efficient model for multi-qubit errors. We developed this approach, prototyped it, and tested it on a cutting-edge quantum processor developed by Rigetti Quantum Computing (RQC), a US-based startup. We implemented our new models within Sandia’s PyGSTi open-source code, and tested them experimentally on the RQC device by probing crosstalk. We found two major results: first, our schema worked and is viable for further development; second, while the Rigetti device is indeed a “real” 8-qubit quantum processor, its behavior fluctuated significantly over time while we were experimenting with it and this drift made it difficult to fit our models of crosstalk to the data.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Machine Learning-enabled Scalable Performance Prediction of Scientific Codes

Hardware architectures become increasingly complex as the compute capabilities grow to exascale. Here, we present the Analytical Memory Model with Pipelines (AMMP) of the Performance Prediction Toolkit (PPT). PPT-AMMP takes high-level source code and hardware architecture parameters as input and predicts runtime of that code on the target hardware platform, which is defined in the input parameters. PPT-AMMP transforms the code to an (architecture-independent) intermediate representation, then (i) analyzes the basic block structure of the code, (ii) processes architecture-independent virtual memory access patterns that it uses to build memory reuse distance distribution models for each basic block, and (iii) runs detailed basic-block level simulations to determine hardware pipeline usage. PPT-AMMP uses machine learning and regression techniques to build the prediction models based on small instances of the input code, then integrates into a higher-order discrete-event simulation model of PPT running on Simian PDES engine. We validate PPT-AMMP on four standard computational physics benchmarks and present a use case of hardware parameter sensitivity analysis to identify bottleneck hardware resources on different code inputs. We further extend PPT-AMMP to predict the performance of a scientific application code, namely, the radiation transport mini-app SNAP. To this end, we analyze multi-variate regression models that accurately predict the reuse profiles and the basic block counts. We validate predicted SNAP runtimes against actual measured times.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Algorithmic and GPU enhancements for molecular dynamics in Cabana and LAMMPS

The focus of this milestone was to implement and test new algorithms and optimizations for improved GPU performance by molecular dynamics (MD) and related particle codes. The work was done in the CoPA Cabana library, the CabanaMD mini-app which uses Cabana to implement and benchmark prototypical MD models, and the LAMMPS MD code which is used by the ECP EXAALT project as well as for materials modeling generally. Efforts included (a) implementation of new or optimized algorithms for fundamental MD kernels like neighbor-list building and FFTs which are also widely used in other particle methods, (b) performance testing of data structure layouts for both simple pairwise and computationally intense neural network potentials, (c) optimizations aimed at improving GPU performance for models with small atom counts, and (d) optimizations specific to the SNAP interatomic potential due to its importance for the EXAALT project.

36 MATERIALS SCIENCE↗