Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation methodology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Identifying Outliers in AI-based Image Compression

Image compression using artificial intelligence (AI) is becoming increasingly prevalent across various fields, including scientific research. Scientific instruments can generate hundreds of images per second, and effectively compressing these images with high compression ratios is crucial for facilitating scientific discoveries. However, automatically detecting outlier cases, where compression may not have succeeded or where interesting scientific phenomena are present, poses a significant challenge. To address this, we have developed a methodology based on unsupervised machine learning techniques for detecting outlier compressed images. This methodology utilizes metrics such as peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), structural texture similarity index measure (STSIM), and deep image and structural texture similarity index (DISTS). We have evaluated our methodology on several unlabeled datasets, including microscopy and x-ray images, and have successfully identified multiple outlier images using our proposed approach. Furthermore, our approach has enabled us to identify image semantics that are valuable for post-experiment analysis by scientists.

Data Analysis↗

Antimicrobial properties of nanostructured surfaces – demonstrating the need for a standard testing methodology

Bioinspired nanostructured materials that exhibit antimicrobial properties are being synthesized and tested at increasing rates for use in healthcare, manufacturing processes, and diagnostics. Although progress has been made in improving and understanding their bactericidal activity, arguably, the biggest problem currently in the field is the lack of a standard testing methodology that allows for optimal characterization and better comparison of emerging nanostructures. In this work, we examine two forms of nanostructured silicon that vary in their ability to kill certain bacterial species due to different physical mechanisms and derive guidelines for the comparative testing. We perform a comprehensive evaluation of methodologies used extensively in the field (e.g., colony counting and live dead analysis) and the novel application of high-throughput flow cytometry. The data reveal how the techniques are complementary but not always directly equivalent or correlative. Therefore, comparison of results obtained using different methodologies on different materials can be grossly misleading. We report significant variations in bactericidal efficiencies depending on experimental environments (medium type, etc.) and methodologies employed. In addition, we demonstrate how cytometry is yet another powerful complementary tool that can aid the mechanistic understanding of antimicrobial activities of rough surfaces. Besides standardization for comparison, ultimately, evaluation methods need to consider anticipated applications. Then and only then can the true potential (or limitation) of a novel material be determined for its suitability for advancement in a particular field of use.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Port of New York and New Jersey Drayage Electrification Analysis

The National Renewable Energy Laboratory (NREL) evaluated the potential for drayage electrification in the Port of New York and New Jersey (PoNYNJ), with a focus on operators: Harbor Freight Transport (HF), Safeway Trucking (SWT), and International Motor Freight Inc (IMF). This report summarizes the data collection and electrification evaluation of all three drayage operators, includes detailed operational data, and identifies the performance requirements for battery electric tractors (BETs) and corresponding infrastructure operated within the context of PoNYNJ drayage operation. This report also details a methodology to evaluate opportunities, strategies, and challenges associated with future expansions of BETs in meeting PANYNJ emissions goals. The Port Authority has established a goal of achieving Net Zero carbon emissions by 2050 across all facilities, including from tenant and stakeholder sources such as drayage trucks. NREL used real-world performance data collected on the three PoNYNJ drayage operations, along with modeling and analysis tools to compare BET to diesel trucks. From March to July 2021, NREL collected 1Hz vehicle and engine data from 46 drayage trucks at the three operators totaling nearly 121,000 miles of operation, providing enough information to assess vehicle operations for electrification potential. A Future Automotive Systems Technology Simulator (FASTSim) electric truck powertrain model was validated using PoNYNJ data and scenarios were run to evaluate drayage truck electrification requirements over the real-world cycles. The first scenario examined BET viability with minimal changes to existing operations. This assumes the trucks charge when stopped for two hours or longer, have a functional battery size of 375 kWh, and can charge at 270 kilowatts (kW) average which are the specification of the commercially available Freightliner eCascadia. The second scenario looked at what operational, charging infrastructure, and BET technology changes would be needed to fully electrify. Finally, detailed analysis was run on charging rate structure to understand operational costs to the fleets. The studied drayage trucks averaged 5.1 MPG, spent roughly 9% of their energy at idle, and drove an average of 140 miles per day with a maximum daily distance of 573 miles. The FASTSim model results indicate a comparable BET would use 417 kWh of energy per day on average accounting for cargo weight, which is close to the full usable capacity of the eCascadia currently available on the market. Based on the daily average operating data, partial fleet electrification is possible with current technology. However, some specific days of operation would require over 1,600 kWh of energy due to longer distances traveled by the trucks and more intense operation. Trucks used for long distance and intense operation cannot be readily electrified with current technology without operational changes. Full adoption of BETs could reduce CO 2 emissions from these fleets by roughly 75% today, eliminating 76 metric tons of CO 2 (MTCO 2 ) per vehicle each year, which equates to 24,100 MTCO 2 per year for all three operators. Commercially available direct current fast chargers (DCFC) have charge rates up to 350 kW. Based on the average daily modeled energy use for each operator, current industrial rate structures, and the assumption of 350 kW peak charging, full drayage electrification would increase electricity consumption. In addition, peak demand usage would increase with unmanaged charging along with cost of electricity having a direct impact on cost per mile for electric vehicles. The resulting cost per mile for BETs along with comparable cost per mile for conventional diesel trucks are also examined at $\$$4.00 per gallon of diesel. It will be important for PANYNJ and the drayage operators within the PoNYNJ to consider these load impacts to their existing electrical infrastructure and devise operational strategies that avoid coincident charging of vehicles to mitigate demand charges. Despite these electricity cost increases, savings from reductions in diesel consumption will help offset the costs of this increased electricity consumption. However, prices of both electricity and diesel are subject to change based on various factors meaning the realized savings will vary over time. This shows BETs could be cost-competitive on an energy cost per mile basis for all scenarios while diesel is above $\$$3.00/gal. Further, if diesel prices dropped to the 15-year low of $2.33/gal, it would still be cost competitive to operate the EVs with electricity costs of 16.3 ¢/kWh or less.

33 ADVANCED PROPULSION SYSTEMS↗

Seasonal performance evaluation of zero-superheat active refrigerant charge control for variable-speed heat pumps

Heat pumps account for a significant portion of electricity consumption in buildings, making their energy efficiency a critical area of research. Control optimization is recognized as an effective approach for enhancing the efficiency of building HVAC systems. Specifically, since the efficiency of heat pumps increases with reduced superheat and the optimal refrigerant charge level varies with operating conditions, zero-superheat active charge control is a promising strategy to maximize the system’s COP. However, due to its complexity, this control strategy remains in the research phase and has only been tested under limited operating conditions. This paper proposes a methodology to evaluate the seasonal performance improvement of zero-superheat active charge control across a full range of heating conditions in buildings. The methodology includes the development of an optimal controller that utilizes an accumulator to store excess refrigerant and manages the active charge level at zero superheat through subcooling control, as well as the creation of a detailed physics-based heat pump model. Targeted a 3-ton variable-speed heat pump with R410A refrigerant, we selected ten U.S. cities representing different climate regions for seasonal performance evaluation. Our simulations indicate that, depending on the climate conditions, the optimal control strategy could achieve energy savings ranging from 5.8 % to 8.0 % over the entire heating season. These results suggest that zero-superheat active charge control, combined with coordinated compressor and fan speeds, could substantially improve the energy efficiency of heat pump systems while maintaining the required heating capacity.

Guo, Fangzhou↗

Uncertainties and Limitations of Experimental Thrust and Pressure Gain Measurements in a Rotating Detonation Combustor

Experimentally measuring the pressure gain through the method of equivalent available pressure is prone to many experimental uncertainties. This study performs a detailed uncertainty analysis of the thrust and pressure gain measurements utilizing data from a H2/air operated rotating detonation combustor with an axial air inlet and a 50% exit constriction. The measured (negative) pressure gain values agree with the results in literature for similar inlet-to-exit area ratios. Specific considerations are given to the uncertainty in thrust measurement introduced by the base drag correction on the inner-body. The assumed unity Mach number utilized in the equivalent available pressure methodology is evaluated experimentally with two different methods using time-averaged measurements at the exhaust plane. A definitive demonstration of gain will be challenging given the assessed uncertainties, but recommendations are given to increase the precision of the equivalent available pressure methodology to estimate pressure gain.

33 ADVANCED PROPULSION SYSTEMS↗

Methodology to assess “no-touch” building audit software using simulated utility data

Building audits are conducted in many commercial buildings to identify opportunities to reduce energy costs and improve building operation. Because audits require significant effort by building engineers, they are usually only affordable for larger commercial buildings. “No-touch” building audit tools have thus been developed to identify potential savings based on a simplified analysis of building energy consumption patterns via high-level energy data such as monthly utility bills. This paper presents a comprehensive and standardized methodology to evaluate the accuracy of no-touch audit tools in detecting and diagnosing building energy problems and quantifying potential energy savings. The test suite is based on output data from a well-characterized set of building energy models, and the methodology is illustrated by applying it to a representative no-touch building audit tool. Results show that the tool estimates building energy end uses with reasonable accuracy but is less accurate in identifying probably causes of high energy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Finite element procedure for thermomechanical and structural integrity analysis of beam intercepting devices subjected to free electron laser

High-energy particles, including photons (x-ray, γ-ray, bremsstrahlung), electrons, and protons, possess the capability to penetrate materials and deposit energy within them. The degree of absorption depends on both the energy and type of particles, as well as the properties of the materials with which they interact. This energy deposition can manifest either at the material's surface or throughout its volume, potentially resulting in various failure modes. The primary aim of this paper is to establish a structured analysis methodology for evaluating the structural integrity of beam-intercepting devices when subjected to high-energy particles. Here, the paper also reviews some of the underlying physics, pertinent to the scope of the thermomechanical analysis, potential failure modes, and introduces verification and validation methodologies. Engineers and researchers can utilize the guidelines presented in this paper to effectively plan the development of beam intercepting devices, thereby ensuring their reliability and performance in the presence of high-energy particle exposure.

Finite element analysis↗

Design methodologies and engineering applications for ecosystem biomimicry: an interdisciplinary review spanning cyber, physical, and cyber-physical systems

Abstract Ecosystem biomimicry is a promising pathway for sustainable development. However, while typical form- and process-level biomimicry is prevalent, system-level ecosystem biomimicry remains a nascent practice in numerous engineering fields. This critical review takes an interdisciplinary approach to synthesize trends across case studies, evaluate design methodologies, and identify future opportunities when applying ecosystem biomimicry to engineering practices, including cyber systems (CS), physical systems (PS), and cyber-physical systems (CPS). After systematically sourcing publications from major databases, the papers were first analyzed at a meta level for their bibliographic context and for statistical correlations among categorical variables. Then, we investigated deeper into the engineering applications and design methodologies. Results indicate that CPS most frequently mimic organisms and ecosystems, while CS and PS frequently mimic populations-communities and molecules-tissues-organ systems, respectively (statistically highly significant). An indirect approach is most often used for mimicry at organizational levels from populations to ecosystems, while a direct approach frequently suits levels from molecules to organisms (highly significant). Dominant themes across engineering applications include symbiotic organism search algorithms for CS and ecological network analysis for CPS, while PS are highly diverse. For design methodologies, this work summarizes and details ten well-documented biomimetic process models among literature, which addresses an outdated concern for a lack of systematic methods for ecosystem biomimicry. In addition to the Biomimetics Standard ISO 18458, these methods include the Natural Step and Techno-Ecological Synergy framework, among others. Further, the analyses revealed future opportunities from less utilized design methods (e.g. interdisciplinary teams tackling indirect, ecosystem-level projects) to well-established engineering concepts ready for technological advancement (e.g. implementing membrane computing for physical applications). For future studies, this review provides a comprehensive reference for ecosystem biomimetic design practices and application opportunities across multiple engineering domains.

42 ENGINEERING↗

Evaluation of the performance of event reconstruction algorithms in the JSNS 2 experiment using a 252 Cf calibration source

JSNS 2 investigates short-baseline neutrino oscillations using a 24-meter baseline and a 17-tonne Gd-loaded liquid scintillator target. Accurate event-reconstruction algorithms are crucial for analyzing experimental data. The algorithms undergo meticulous validation through calibration with a 252 Cf source. This paper outlines the methodology and evaluates the reconstruction performance, focusing on neutrino interactions up to approximately 50 MeV for sterile neutrino searches. Both 252 Cf and Michel electron events are studied to evaluate reconstruction accuracy. The analysis concludes that the uncertainty of the fiducial volume, with an appropriate correction, is much less than the requirement of JSNS 2 requirement (10%). Furthermore, the energy resolution is measured to be 3.3 ± 0.1% for the Michel electron endpoint and 4.3 ± 0.1% for the n-Gd peak in the central region.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Laboratory testing methods to evaluate the reliability of occupancy sensors for commercial building applications

The energy performance of commercial buildings is greatly influenced by occupants which are highly variable and among the most unpredictable components of a building's operation. While most building control systems use fixed, predetermined occupancy schedules, these fixed occupancy levels can be quite different from actual occupancy. This can cause unnecessary energy consumption, particularly from heating, ventilation, and air conditioning (HVAC) and lighting systems which are responsible for approximately 60% of commercial buildings' energy use. The use of occupancy counting sensor systems integrated with building management system controls is one method that can be used to improve the energy-consuming performance of buildings. However, there is no standardized universal methodology and metrics to evaluate their reliability. The aim of this research is to develop a uniform evaluation methodology to assess the reliability of occupancy counting sensor systems in a controlled laboratory environment. The developed testing methodology includes both “typical” scenarios representing the occupancy scenarios of a typical commercial building, and “failure” testing scenarios which represent a range of potential scenarios that may impact a sensor system's reliability. These methods were then implemented in a case study to evaluate the performance of two novel occupancy counting sensor systems (i.e., door-centric, and camera-based). Results suggest that typical testing results can be used to compare the overall performance of the occupancy counting sensor systems; however, failure testing is also important to understand the weaknesses of the sensor system in order to select the suitable one for the intended use of the commercial building. In addition, the proposed methodology includes a modified confusion matrix which enables the ability to identify if failures are caused by over or under counting occupants and to what extent this occurs over the testing period.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Developing a Prototype Methodology to Rank CO2-EOR Wells and Assess Their Reuse Potential for Geologic Carbon Sequestration

This study presents a prototype methodology for evaluating the reuse potential of Class II CO₂-enhanced oil recovery (CO₂-EOR) wells as Class VI wells for geologic carbon sequestration. The approach focuses on assessing wellbore construction materials—casing, cement, tubing, and packers—based on U.S. Environmental Protection Agency (EPA) Class VI well conversion guidelines. Utilizing Python scripts and JSON representations, the methodology automates checks on digitized Texas Railroad Commission (TRRC) data to rank wells based on regulatory and integrity criteria. Key factors include casing integrity, cementing techniques, tubing compatibility, and packer selection. Due to limitations in digitized data, manual verification is required for certain sections. Future enhancements include incorporating non-digitized data via web scraping and machine learning. This research provides a practical framework for well owners and regulators, supporting informed decision-making for sustainable CO₂ storage.

carbon sequestration↗

AviSense: A Real-time System for Detection, Classification, and Analysis of Aviation Signals

Wireless systems are an integral part of aviation. Apart from their apparent use in air-to-ground communication, wireless systems play a crucial role in avionic functions including navigation and landing. An interference-free wireless environment is therefore critical for the uninterrupted operation and safety of an aircraft. Hence, there is an urgency for airport facilities to acquire the capability to continuously monitor aviation frequency bands for real-time detection of interference and anomalies. To meet this critical need, we design and build AviSense, an SDR-based real-time, versatile system for monitoring aviation bands. AviSense detects and characterizes signal activities to enable practical and effective anomaly detection. We identify and tackle the challenges posed by a diverse set of critical aviation bands and technologies. We evaluate our methodology with real-world aviation signal measurements and two custom datasets of anomalous signals. We find that our signal classification capability achieves a true positive rate of ~99%, with few exceptions, and a false positive rate of less than 4%. We also demonstrate that AviSense can effectively distinguish between different types of anomalies. We build and evaluate a prototype implementation of AviSense that supports distributed monitoring.

42 ENGINEERING↗

Development of Integrated Safety and Security Models for Comprehensive Reliability and Resiliency Evaluation

The security of the electric grid and supporting energy systems is crucial to national security. One of the complexities in analyzing the security of energy systems is the safety consequences that may result from accidents. For energy systems, the goal is to ensure that they operate as intended and that any consequences are mitigated or prevented. The integration of safety and security is paramount to protecting these systems from attacks and ensuring that large consequences are prevented. This report describes an integrated safety and security methodology to evaluate cybersecurity events that can lead to large consequences. This novel approach first describes how Systems-Theoretic Process Analysis (STPA) provides a digital causal analysis for Bayesian Networks (BNs). The use of STPA causal analysis provides a systematic approach to constructing BNs that adequately model cyber scenarios that result in consequences. When combined with the technical principles described in Risk-Informed Management of Enterprise Systems (RIMES), a comprehensive risk-informed cybersecurity analysis results that allows decision-makers to prioritize systems that most impact risk.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Quantitative evaluation of common cause failures in high safety-significant safety-related digital instrumentation and control systems in nuclear power plants

Digital instrumentation and control (DI&C) systems at nuclear power plants (NPPs) have many advantages over analog systems. They are proven to be more reliable, cheaper, and easier to maintain given obsolescence of analog components. However, they also pose new engineering and technical challenges, such as possibility of common cause failures (CCFs) unique to digital systems. Here this paper proposes a Platform for Risk Assessment of DI&C (PRADIC) that is developed by Idaho National Laboratory (INL). A methodology for evaluation of software CCFs in high safety-significant safety-related DI&C systems of NPPs was developed as part of the framework. The framework integrates three stages of a typical risk assessment—qualitative hazard analysis and quantitative reliability and consequence analyses. The quantified risks compared with respective acceptance criteria provide valuable insights for system architecture alternatives allowing design optimization in terms of risk reduction and cost savings. A comprehensive case study performed to demonstrate the framework's capabilities is documented here in this paper. Results show that the PRADIC is a powerful tool capable to identify potential digital-based CCFs, estimate their probabilities, and evaluate their impacts on system and plant safety.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Evaluation of AI-enabled Digital Documented Safety Analysis

The National Reactor Innovation Center (NRIC) is leading a transformative initiative to accelerate advanced reactor deployment by fundamentally reimagining how nuclear safety basis documentation is developed, reviewed, and maintained. Traditional Documented Safety Analysis (DSA) processes for DOE-authorized facilities rely on static, document-centric workflows that consume significant time and resources, exemplified by recent major licensing efforts requiring hundreds of thousands of staff hours and millions of pages of documentation review. These conventional approaches create barriers to the rapid, cost-effective deployment of advanced reactors that America's future energy needs demand. NRIC's DOE Authorization Digital Transformation Project addresses these challenges through an innovative framework that integrates artificial intelligence (AI), digital engineering, and systems-based data management into a cohesive digital ecosystem. This white paper presents NRIC's methodology for evaluating AI-enabled document generation capabilities within this broader digital infrastructure, using the Demonstration of Microreactor Experiments (DOME) facility as a pilot case study. The evaluation will assess an AI tool's ability to generate a Preliminary Documented Safety Analysis (PDSA) through progressive integration stages—from standalone document processing to full digital thread connectivity—while maintaining rigorous verification, validation, and regulatory acceptance standards. By establishing dynamic, traceable connections between design data and safety documentation, NRIC's approach has the potential to reduce both document development time and regulatory review cycles by as much as 50%, while simultaneously improving accuracy, consistency, and traceability. This initiative represents a critical step toward establishing reusable digital infrastructure that reactor developers can leverage to accelerate their path from concept to commercial operation, directly supporting NRIC's mission to demonstrate and deploy advanced nuclear energy technologies.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Evaluation of Mesoscale Convective Systems in Climate Simulations: Methodological Development and Results from MPAS-CAM over the U.S.

In this study, a process-oriented approach is developed to evaluate warm-season mesoscale convective system (MCS) precipitation and their favorable large-scale meteorological patterns (FLSMPs) over the U.S. This approach features a novel observation-driven MCS-tracking algorithm using infrared brightness temperature and precipitation feature at 12, 25 and 50 km resolution and metrics to evaluate the model large-scale environment favorable for MCS initiation. The tracking algorithm successfully reproduces the observed MCS statistics from a reference 4-km radar MCS database. To demonstrate the utility of the new methodologies in evaluating MCS in climate simulations with mesoscale resolution, the process-oriented approach is applied to two climate simulations produced by the Variable-Resolution Model for Prediction Across Scales coupled to the Community Atmosphere Model physics, with refined horizontal grid spacing at 50 km and 25 km over North America. With the tracking algorithm applied to simulations and observations at equivalent resolutions, the simulated number of MCS and associated precipitation amount, frequency and intensity are found to be consistently underestimated in the Central U.S., particularly from May to August. The simulated MCS precipitation shows little diurnal variation and lasts too long, while MCS precipitation area is too large and intensity is too weak. The model is able to simulate four types of observed FLSMP associated with frontal systems and low-level jets (LLJ) in spring, but the frequencies are underestimated because of low-level dry bias and weaker LLJ. Precipitation simulated under different FLSMPs peak during daytime, in contrast to the observed nocturnal peak. Implications of these findings for future model development and diagnostics are discussed.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Skyshine from an Accelerator Facility: Dependence on Distance and Angle

Neutron skyshine from Linac Coherent Light Source II 4 GeV electron beam operation at SLAC National Accelerator Laboratory can contribute to prompt radiation exposure to the public at distances far beyond the accelerator tunnel housing. One of the shielding design requirements at SLAC is that the annual dose to a member of the public is no more than 0.05 mSv y -1 . Our work uses Monte Carlo code FLUKA to simulate the generation of neutrons from 4 GeVelectron beam losses on a thick copper target inside a generalized geometry of the Linac Coherent Light Source II Beam Transport Hall accelerator tunnel section. The effective dose from neutron skyshine was characterized as a function of both distance from the tunnel wall (up to 1 km away) and angle relative to the beam direction (between 0° and 180°). Furthermore, this new methodology for evaluating neutron skyshine dose is applicable to high-energy GeV-range electron accelerator facilities both at SLAC and elsewhere.

43 PARTICLE ACCELERATORS↗

“One Table to Rule Them All”: How a Single Table can Enable Extensive Insights, Analytics and Assessment on Human Mobility Data

While much research has been conducted in Human Mobility Science, most studies on the analytics/insights part generally focus on one of the following: processing and analytics on human stop-trip behavior, design of individual mobility metrics (often in silos), calculation and characterization of only a handful (typically 5-6) of human mobility metrics on geospatial-temporal human mobility data of interest. Although human mobility research offers a vast and diverse array of available metrics, most individual studies typically compute only a small subset of five or six metrics at a time when analyzing trajectory datasets of human mobility across different areas of interest. This paper is motivated by the critical need to repeatedly compute an extensive array of human mobility metrics across several trajectory datasets and perform individual metric-level benchmarking to establish a new, standardized Test and Evaluation (T&E) suite for the field of Human Mobility Science. We first present our findings on the minimal yet sufficient pre-processing required to reliably and efficiently compute a wide range of human mobility metrics. The key findings are specifically related to the proposed Composite Stop Locations table, which serves as a core pre-processing data layer. Subsequently, we present a case study demonstrating how the Composite Stop Locations table facilitates computation of at least 14 distinct human mobility metrics (unlike 5-6 different set of metrics used for studies in the literature) using the popular and open-source OpenPFLOW dataset. Finally, we have also presented an example of our benchmarking methodology to evaluate the quality and performance of the trajectory dataset of interest, assessed across multiple human mobility metrics.

De, Debraj [ORNL] (ORCID:0000000233630020)↗