Search NASA⌕ Search

SEARCH · Search NASA

Results for “application benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,693 records · Page 94

From Rules to Reasoning: A Survey of Large Language Model-Based Approaches to Scientific Hypothesis and Idea Generation

Scientific hypothesis generation represents a fundamental challenge in contemporary research due to exponentially expanding literature volumes and increasing disciplinary specialization. Large language models (LLMs) have emerged as transformative tools for automated scientific discovery, moving beyond traditional rule-based and literature-mining approaches. Four paradigmatic approaches define current LLM-driven hypothesis generation: direct prompting and fine-tuning methods, knowledge-enhanced frameworks integrating retrieval-augmented generation (RAG), multi-agent collaborative systems simulating research teams, and reasoning-focused approaches implementing cognitive architectures. Domain-specific applications demonstrate statistical equivalence to human expert performance in social psychology, experimental validation in biomedical research, and near-expert quality in astronomy. Evaluation methodologies encompass human expert assessment, LLM-as-judge frameworks, and comprehensive benchmarking systems. Technical challenges include hallucination management, knowledge integration limitations, and balancing novelty with feasibility. Future directions emphasize hybrid neural-symbolic architectures and sophisticated human-AI collaboration models for responsible scientific discovery acceleration.

AI-driven discovery↗

Benchmarking Demand Flexibility in Commercial Buildings and Flattening the Duck – Addressing Baseline and Commissioning Challenges

With the transition from our traditional electric grid to a cleaner grid with renewable power generation, there is a need to enable building loads to be flexible. Load shedding and shifting will be essential for flattening the “Duck” for decarbonization. This paper explored the trend in the timing of DR events as a reflection of the grid’s needs using recent four years of event data from 203 retail stores in 11 states. The events are becoming significantly shorter with 2-hour duration being the most popular; shifting to late afternoon and early evening is another trend beyond California. Benchmarking will be essential for accounting DF as a reliable grid resource. This paper addresses a challenging aspect of benchmarking – inaccuracies in counterfactual baseline methods can introduce significant DF metrics variations in addition to weather and building characteristics related factors. The conventional “10/10” with adjustment baseline method has inherent limitation by design for load shifting applications. Therefore, it is imperative to identify alternative methods. This study compared three hourly regression baseline methods with “10/10” methods using two groups of commercial buildings that participated in DR programs: (1) 121 big-box retail stores, and (2) 11 office buildings in CA. The 14-day hourly outdoor temperature regression method was found to produce least error in the tested datasets and is promising for load shifting. The paper also pointed out that commissioning issues can also be a significant barrier for achieving consistent DF performance, which building managers and utilities should be aware of.

Liu, Jingjing↗

Methodology for Sensitivity and Uncertainty-Based Criticality Safety Validation

Monte Carlo radiation transport software is used extensively for determining the effective multiplication factor k of fissionable systems towards assessing the criticality safety of operations. Understanding the predictive capability of the Monte Carlo software and nuclear data libraries for calculations of specific applications is vital towards making these assessments. Recently, sensitivity and uncertainty (S/U) methods have been developed for this purpose. A computational tool has been developed that uses the S/U methods with MCNP6 to select critical experiment benchmarks that have similar sources of computational bias, and extreme value theory is then used to determine suitable and conservative calculational margins. The S/U techniques are then used to perform nuclear data adjustments, which are then used to determine a suitable margin of subcriticality from cross section uncertainties. This document gives an overview of the theory of S/U methods used for criticality safety validation. First an overview is given for using the sensitivity coefficients and nuclear co variance data towards determining neutronic similarity factors is then discussed. Next, the extreme value theory (EVT) to determine a calculational margin is given along with how the similarity factors are used in the analysis. Then, the Generalized Linear Least Squares (GLLS) method for performing data adjustment and how that method is used to reject benchmarks and compute adjusted nuclear data uncertainties in k are discussed. Last, a discussion of the margin of subcriticality is given along with how to apply of the GLLS data adjustment methods to quantify the effect of nuclear data variability on k.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Performance Evaluation and Modeling Techniques for Parallel Processors

In practice, the performance evaluation of supercomputers is still substantially driven by singlepoint estimates of metrics (e.g., MFLOPS) obtained by running characteristic benchmarks or workloads. With the rapid increase in the use of time-shared multiprogramming in these systems, such measurements are clearly inadequate. This is because multiprogramming and system overhead, as well as other degradations in performance due to time varying characteristics of workloads, are not taken into account. In multiprogrammed environments, multiple jobs and users can dramatically increase the amount of system overhead and degrade the performance of the machine. Performance techniques, such as benchmarking, which characterize performance on a dedicated machine ignore this major component of true computer performance. Due to the complexity of analysis, there has been little work done in analyzing, modeling, and predicting the performance of applications in multiprogrammed environments. This is especially true for parallel processors, where the costs and benefits of multi-user workloads are exacerbated. While some may claim that the issue of multiprogramming is not a viable one in the supercomputer market, experience shows otherwise. Even in recent massively parallel machines, multiprogramming is a key component. It has even been claimed that a partial cause of the demise of the CM2 was the fact that it did not efficiently support time-sharing. In the same paper, Gordon Bell postulates that, multicomputers will evolve to multiprocessors in order to support efficient multiprogramming. Therefore, it is clear that parallel processors of the future will be required to offer the user a time-shared environment with reasonable response times for the applications. In this type of environment, the most important performance metric is the completion of response time of a given application. However, there are a few evaluation efforts addressing this issue.

Dimpsey, Robert Tod↗

Cost studies for commercial fuselage crown designs

Studies were conducted to evaluate the cost and weight potential of advanced composite design concepts in the crown region of a commercial transport. Two designs from each of three design families were developed using an integrated design-build team. A range of design concepts and manufacturing processes were included to allow isolation and comparison of cost centers. Detailed manufacturing/assembly plans were developed as the basis for cost estimates. Each of the six designs was found to have advantages over the 1995 aluminum benchmark in cost and weight trade studies. Large quadrant panels and cobonded frames were found to save significant assembly labor costs. Comparisons of high- and intermediate-performance fiber systems were made for skin and stringer applications. Advanced tow placement was found to be an efficient process for skin lay up. Further analysis revealed attractive processes for stringers and frames. Optimized designs were informally developed for each design family, combining the most attractive concepts and processes within that family. A single optimized design was selected as the most promising, and the potential for further optimization was estimated. Technical issues and barriers were identified.

Walker, T. H.↗

Microgravity science and applications. Program tasks and bibliography for FY 1994

This annual report includes research projects funded by the Office of Life and Microgravity Sciences and Applications, Microgravity Science and Applications Division, during FY 1994. It is a compilation of program tasks (objective, description, significance, progress, students funded under research, and bibliographic citations) for flight research and ground-based research in five major scientific disciplines: benchmark science, biotechnology, combustion science, fluid physics, and materials science. ATD (Advanced Technology Development) program task descriptions are also included. The bibliography cites the related PI (Principal Investigator) publications and presentations for these program tasks in FY 1994. Three appendices include Table of Acronyms, Guest Investigator Index, and Principal Investigator Index.

Source record↗

Performance modeling & simulation of complex systems (A systems engineering design & analysis approach)

Modeling of the Multi-mission Image Processing System (MIPS) will be described as an example of the use of a modeling tool to design a distributed system that supports multiple application scenarios. This paper examines: (a) modeling tool selection, capabilities, and operation (namely NETWORK 2.5 by CACl), (b) pointers for building or constructing a model and how the MIPS model was developed, (c) the importance of benchmarking or testing the performance of equipment/subsystems being considered for incorporation the design/architecture, (d) the essential step of model validation and/or calibration using the benchmark results, (e) sample simulation results from the MIPS model, and (f) how modeling and simulation analysis affected the MIPS design process by having a supportive and informative impact.

Hall, Laverne↗

Spectroscopic Determination of the Physical Conditions in Hot, Optically Thin Sources

This annual report covers the period from Oct 1, 2000 to Sep 30, 2001. The Astrophysics Plasma Emission Code and Database (APEC/APED), developed in part under this grant, have been upgraded to Version 1.1 and are now beginning to be used outside our research group in applications to X-ray spectral data from Chandra and XMM-Newton. These models represent the best theoretical data currently available and are reasonably complete below about 30 angstroms. Stellar coronae are being used to benchmark the atomic data in APED as part of the Emission Line Project. Initial results suggest that the models for most of the strongest lines are in good agreement with the observations, including the H-like and He-like emission from dominant elements and the Fe L shell emission, both near 1 keV and in the extreme ultraviolet near 100 eV. At this point in time, we define 'good agreement' at the level of accuracy expected from the atomic physics, approximately 20-30%. In order to benchmark the spectral models beyond the canonical theoretical accuracy, we are working closely with the Chandra gratings calibration group to ensure that we are using the optimal calibration, primarily effective areas and line response functions.

Brickhouse, N.↗

Multi-Core Processor Memory Contention Benchmark Analysis Case Study

Multi-core processors dominate current mainframe, server, and high performance computing (HPC) systems. This paper provides synthetic kernel and natural benchmark results from an HPC system at the NASA Goddard Space Flight Center that illustrate the performance impacts of multi-core (dual- and quad-core) vs. single core processor systems. Analysis of processor design, application source code, and synthetic and natural test results all indicate that multi-core processors can suffer from significant memory subsystem contention compared to similar single-core processors.

Simon, Tyler↗

Exploring Cognitive States: Methods for Detecting Physiological Temporal Fingerprints

Cognitive state detection and its relationship to observable physiologically telemetry has been utilized for many human-machine and human-cybernetic applications. This paper aims at understanding and addressing if there are unique psychophysiological patterns over time, a ”physiological temporal fingerprint”, that is associated with specific cognitive states. This preliminary work involves commercial airline pilots completing experimental benchmark task inductions of three cognitive states: 1) Channelized Attention (CA); 2) High Workload (HW); and 3) Low Workload (LW). We approach this objective by modeling these "fingerprints" through the use of Hidden Markov Models and Entropy analysis to evaluate if the transitions over time are complex or rhythmic/predictable by nature. Our results indicate that cognitive states do have unique complexity of physiological sequences that are statistically different from other cognitive states. More specifically, CA has a significantly higher temporal psychophysiological complexity than HW and LW in EEG and ECG telemetry signals. With regards to respiration telemetry, CA has a lower temporal psychophysiological complexity than HW and LW. Through our preliminary work, addressing this unique underpinning can inform whether these underlying dynamics can be utilized to understand how humans transition between cognitive states and for improved detection of cognitive states.

Napoli, Nicholas J.↗

PALMO: An OVERFLOW Machine Learning Airfoil Performance Database

The OVERFLOW Machine Learning Airfoil Performance (PALMO) database has been created to enable robust modeling of airfoil performance in a variety of applications. The PALMO database uses OVERFLOW simulation data second-order accurate in time and fourth-order accurate in space with Spalart-Allmaras turbulence closure. The foundation of the in-development PALMO database is the airfoil base cube. Each base cube includes simulation data parametrized over a range of Mach numbers, Reynolds numbers, and angles-of-attack. This database includes the NACA 4-series airfoils, with parametrization in airfoil thickness and camber from an NACA 0006 to an NACA 4424. In total, 52,480 NACA 4-series OVERFLOW calculations were run on the NASA High-End Compute Capability (HECC) supercomputer. This provides high-order-accurate simulation data covering a wide range of aerospace design applications, which enables users to develop accurate airfoil performance look-up tables without additional high-performance computing. In addition to engineering design and analysis of aerospace vehicles, PALMO is well suited to be a benchmark dataset for the development and testing of machine learning methods in aerospace engineering. This work presents an example PALMO surrogate model that enables accurate airfoil performance predictions for any arbitrary combination of camber, thickness, Mach number, Reynolds number, and angle of attack within the bounds of the database. Airfoil performance tables predicted for an airfoil not used in training the model are used in three-dimensional OVERFLOW simulations to quantify the downstream accuracy on aggregate rotor performance metrics. For the NACA 3415 airfoil, which had no common thickness or camber with the training data, the surrogate predicted and CFD generated tables were within 2.1% of each other in the forward flight lift to drag metric. This suggests that performance tables generated for airfoils within the bounds of the PALMO database will yield aggregate rotor performance predictions on par with tables generated from directly running OVERFLOW airfoil calculations. The PALMO airfoil performance coefficients are available publicly.

Database↗

VERA Benchmarking Results for Watts Bar Nuclear Plant Unit 1 Cycles 1-12

The Consortium for Advanced Simulation of Light Water Reactors, the first US DOE Energy Innovation Hub, is developing and deploying a suite of directly coupled, high-fidelity physics methods known as VERA for performing engineering analyses of operating commercial nuclear power reactors. The centerpiece of VERA is the reactor core simulator, which employs the direct coupling of a 3D neutron transport code, a two-phase sub-channel thermal-hydraulic code, and a well-known isotopic depletion and decay code. Together these tools represent the most rigorous reactor simulator available today. Recently added capabilities include the depletion, shuffling, and discharge of fuel assemblies, enabling VERA to simulate multiple fuel cycles of operation. To demonstrate this capability, VERA has been used to simulate over 18 years of operation of TVA’s Watts Bar Nuclear Unit 1, and the results of these calculations have been used to quantify its performance in an industry-grade benchmarking activity. Comparisons are made with measured critical boron concentrations, control bank reactivity worths, isothermal temperature coefficients, and in-core neutron flux distributions. These results demonstrate that the advanced tools in VERA are capable for application to practical and common industrial problems.

Godfrey, Andrew T↗

Considerations on the Use of Custom Accelerators for Big Data Analytics

Accelerators, including Graphic Processing Units (GPUs) for gen- eral purpose computation, many-core designs with wide vector units (e.g., Intel Phi), have become a common component of many high performance clusters. The appearance of more stable and reliable tools tools that can automatically convert code written in high-level specifications with annotations (such as C or C++) to hardware de- scription languages (High-Level Synthesis - HLS), is also setting the stage for a broader use of reconfigurable devices (e.g., Field Pro- grammable Gate Arrays - FPGAs) in high performance system for the implementation of custom accelerators, helped by the fact that new processors include advanced cache-coherent interconnects for these components. In this chapter, we briefly survey the status of the use of accelerators in high performance systems targeted at big data analytics applications. We argue that, although the progress in the use of accelerators for this class of applications has been sig- nificant, differently from scientific simulations there still are gaps to close. This is particularly true for the ”irregular” behaviors exhibited by no-SQL, graph databases. We focus our attention on the limits of HLS tools for data analytics and graph methods, and discuss a new architectural template that better fits the requirement of this class of applications. We validate the new architectural templates by mod- ifying the Graph Engine for Multithreaded System (GEMS) frame- work to support accelerators generated with such a methodology, and testing with queries coming from the Lehigh University Benchmark (LUBM). The architectural template enables better supporting the task and memory level parallelism present in graph methods by sup- porting a new control model and a enhanced memory interface. We show that out solution allows generating parallel accelerators, pro- viding speed ups with respect to conventional HLS flows. We finally draw conclusions and present a perspective on the use of reconfig- urable devices and Design Automation tools for data analytics.

Data Analytics↗

‘Wasteaware’ benchmark indicators for integrated sustainable waste management in cities

Highlights: • Solid waste management (SWM) is a key utility service, but data is often lacking. • Measuring their SWM performance helps a city establish priorities for action. • The Wasteaware benchmark indicators: measure both technical and governance aspects. • Have been developed over 5 years and tested in more than 50 cities on 6 continents. • Enable consistent comparison between cities and countries and monitoring progress. - Abstract: This paper addresses a major problem in international solid waste management, which is twofold: a lack of data, and a lack of consistent data to allow comparison between cities. The paper presents an indicator set for integrated sustainable waste management (ISWM) in cities both North and South, to allow benchmarking of a city’s performance, comparing cities and monitoring developments over time. It builds on pioneering work for UN-Habitat’s solid waste management in the World’s cities. The comprehensive analytical framework of a city’s solid waste management system is divided into two overlapping ‘triangles’ – one comprising the three physical components, i.e. collection, recycling, and disposal, and the other comprising three governance aspects, i.e. inclusivity; financial sustainability; and sound institutions and proactive policies. The indicator set includes essential quantitative indicators as well as qualitative composite indicators. This updated and revised ‘Wasteaware’ set of ISWM benchmark indicators is the cumulative result of testing various prototypes in more than 50 cities around the world. This experience confirms the utility of indicators in allowing comprehensive performance measurement and comparison of both ‘hard’ physical components and ‘soft’ governance aspects; and in prioritising ‘next steps’ in developing a city’s solid waste management system, by identifying both local strengths that can be built on and weak points to be addressed. The Wasteaware ISWM indicators are applicable to a broad range of cities with very different levels of income and solid waste management practices. Their wide application as a standard methodology will help to fill the historical data gap.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Recent DIII-D progress toward validating models of tungsten erosion, re-deposition, and migration for application to next-step fusion devices

Abstract Fundamental mechanisms governing the erosion and prompt re-deposition of tungsten impurities in tokamak divertors are identified and analyzed to inform the lifetime of tungsten plasma-facing components in ITER and other future devices. Various experiments conducted at DIII-D to benchmark predictive models are presented, leveraging the DiMES removable sample exposure probe capability and the Metal Rings Campaign, in which toroidally symmetric rows of tungsten-coated tiles were installed in the DIII-D divertor. In tokamak divertors, the width of the electric sheath is of the order of the main ion Larmor radius, and a vast majority of sputtered tungsten impurities are typically ionized within the sheath. Therefore, W prompt redeposition is mainly governed by the ratio of the characteristic ionization mean-free path of neutral tungsten to the width of the sheath. In-situ monitoring of the prompt redeposition of tungsten impurities in divertors is demonstrated via the use of WII/WI line ratios and the ionizations/photon (S/XB) method in L-mode discharges. Even with this relatively limited set of emission measurements, net erosion measurements were found to be a consistent upper bound to an analytic scaling based on the ratio of the W ionization length, λ iz , and the width of the magnetic sheath rather than the ratio of λ iz and the W + gyro-radius. In the far-scrape-off layer (SOL) of the ITER divertor, however, it is calculated that the measurement of photon emissions associated with the ionization of tungsten impurities up to W 5 + may be required. Finally, W deposition patterns on DiMES collector probes, interpreted via DIVIMP-WallDYN modelling, reveal the key roles of progressive W erosion/re-deposition staps and E × B drifts in regulating long-range high-Z material migration.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Mathematical efficiency calibration with uncertain source geometries using smart optimization

The In Situ Object Counting Software (ISOCS), a mathematical method developed by CANBERRA, is a well established technique for computing High Purity Germanium (HPGe) detector efficiencies for a wide variety of source shapes and sizes. In the ISOCS method, the user needs to input the geometry related parameters such as: the source dimensions, matrix composition and density, along with the source-to-detector distance. In many applications, the source dimensions, the matrix material and density may not be well known. Under such circumstances, the efficiencies may not be very accurate since the modeled source geometry may not be very representative of the measured geometry. CANBERRA developed an efficiency optimization software known as 'Advanced ISOCS' that varies the not well known parameters within user specified intervals and determines the optimal efficiency shape and magnitude based on available benchmarks in the measured spectra. The benchmarks could be results from isotopic codes such as MGAU, MGA, IGA, or FRAM, activities from multi-line nuclides, and multiple counts of the same item taken in different geometries (from the side, bottom, top etc). The efficiency optimization is carried out using either a random search based on standard probability distributions, or using numerical techniques that carry out a more directed (referred to as 'smart' in this paper) search. Measurements were carried out using representative source geometries and radionuclide distributions. The radionuclide activities were determined using the optimum efficiency and compared against the true activities. The 'Advanced ISOCS' method has many applications among which are: Safeguards, Decommissioning and Decontamination, Non-Destructive Assay systems and Nuclear reactor outages maintenance. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

2-D Circulation Control Airfoil Benchmark Experiments Intended for CFD Code Validation

A current NASA Research Announcement (NRA) project being conducted by Georgia Tech Research Institute (GTRI) personnel and NASA collaborators includes the development of Circulation Control (CC) blown airfoils to improve subsonic aircraft high-lift and cruise performance. The emphasis of this program is the development of CC active flow control concepts for both high-lift augmentation, drag control, and cruise efficiency. A collaboration in this project includes work by NASA research engineers, whereas CFD validation and flow physics experimental research are part of NASA s systematic approach to developing design and optimization tools for CC applications to fixed-wing aircraft. The design space for CESTOL type aircraft is focusing on geometries that depend on advanced flow control technologies that include Circulation Control aerodynamics. The ability to consistently predict advanced aircraft performance requires improvements in design tools to include these advanced concepts. Validation of these tools will be based on experimental methods applied to complex flows that go beyond conventional aircraft modeling techniques. This paper focuses on recent/ongoing benchmark high-lift experiments and CFD efforts intended to provide 2-D CFD validation data sets related to NASA s Cruise Efficient Short Take Off and Landing (CESTOL) study. Both the experimental data and related CFD predictions are discussed.

Englar, Robert J.↗

Hierarchically Porous Multimetal‐Based Carbon Nanorod Hybrid as an Efficient Oxygen Catalyst for Rechargeable Zinc–Air Batteries

Abstract The lack of efficient strategies to address the intrinsic activity, site accessibility, and structural stability issues of metal‐carbon hybrid catalysts is restricting their real‐world implementation on the basis of rechargeable zinc–air batteries. Herein, a dual metal–organic frameworks (MOFs) pyrolysis strategy is developed to regulate the intrinsic activity and porous structure of the derived catalysts, where a Fe 2 Ni_MIL‐88@ZnCo_zeolitic imidazolate framework (ZIF), with a hierarchically porous structure, multifunctional components, and an integrated architecture, acts as an ideal precursor to obtain multimetal based porous nanorod (FeNiCo@NC‐P). Benefitting from the synergetic effect of the multimetal components, facilitated reactant accessibility, and the well‐retained integrated structure, the resultant FeNiCo@NC‐P catalyst exhibits an oxygen reduction reaction half‐wave potential of 0.84 V as well as an oxygen evolution reaction potential of 1.54 V at 10 mA cm –2 . Furthermore, the practical application of FeNiCo@NC‐P in the zinc–air battery displays a low voltage gap and long‐term durability (over 130 h at a current density of 10 mA cm –2 ), which outperforms the commercial noble metal benchmarks. This work not only affords a competitive bifunctional oxygen electrocatalyst for zinc–air batteries but also paves a new way to design and fabricate MOF‐derived materials with tunable catalytic properties.

Ren, Dezhang↗