Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation methodology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Efforts to enhance reproducibility in a human performance research project

Background: Ensuring the validity of results from funded programs is a critical concern for agencies that sponsor biological research. In recent years, the open science movement has sought to promote reproducibility by encouraging sharing not only of finished manuscripts but also of data and code supporting their findings. While these innovations have lent support to third-party efforts to replicate calculations underlying key results in the scientific literature, fields of inquiry where privacy considerations or other sensitivities preclude the broad distribution of raw data or analysis may require a more targeted approach to promote the quality of research output. Methods: We describe efforts oriented toward this goal that were implemented in one human performance research program, Measuring Biological Aptitude, organized by the Defense Advanced Research Project Agency's Biological Technologies Office. Our team implemented a four-pronged independent verification and validation (IV&V) strategy including 1) a centralized data storage and exchange platform, 2) quality assurance and quality control (QA/QC) of data collection, 3) test and evaluation of performer models, and 4) an archival software and data repository. Results: Our IV&V plan was carried out with assistance from both the funding agency and participating teams of researchers. QA/QC of data acquisition aided in process improvement and the flagging of experimental errors. Holdout validation set tests provided an independent gauge of model performance. Conclusions: In circumstances that do not support a fully open approach to scientific criticism, standing up independent teams to cross-check and validate the results generated by primary investigators can be an important tool to promote reproducibility of results.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Flexible FlueCO2

Carbon dioxide (CO2) emission reductions remain a significant challenge on the path to clean energy. There are increasing legislative, social, and environmental factors motivating CO2 emissions reduction from power plants with carbon capture and storage (CCS). CCS in natural gas combined cycle (NGCC) power plants is critical to achieve a net-zero carbon electricity grid. Enhanced 45Q tax credits provide new incentives, but currently available technologies are unable to profitably operate in grids with deep variable renewable penetration which require flexible NGCC operation. Luna Labs has developed the FlueCO2 membrane to enable a profitable NGCC-CCS process. The FlueCO2 membrane couples steam transport across the membrane to CO2 transport in the opposite direction, enabling high capture efficiencies and low energy costs even at low CO2 concentrations. The dual-phase membrane can operate in the range of typical flue gas temperatures and pressures and does not require temperature or pressure cycling. Luna Labs’ FlueCO2 technology enables flexible and profitable operation of NGCC plants with lower capital investment and impact on electricity prices. In this Phase 1 project, Luna Labs utilized experimental testing, modeling, process simulation, and standardized costing methodologies to evaluate the techno-economic value of a 650 MW greenfield NGCC plant with FlueCO2 (NGCC-FlueCO2). Key design requirements for operation were established and plant performance under load-leveling conditions was validated through computational fluid dynamics and process modeling. Luna Labs developed a dynamic modeling tool which modeled plant operational modes across a variety of tax structures and electricity pricing scenarios to project the overall Net Present Value (NPV) of the NGCC-FlueCO2. FlueCO2 minimizes the impact of CCS integration on plant operation by integrating directly into the NGCC heat recovery steam generator (HRSG). By tapping into the plant’s low-pressure (LP) steam, operators can divert LP steam to the greenfield NGCC and/or CCS process in response to dynamic markets. Since FlueCO2 will not significantly affect HRSG (or NGCC) operation, CCS only turns off during peak power demand (>$250/MWh). Under baseload conditions, FlueCO2 lowers the capital (37%), energy (36%) and carbon capture (<$40/tonne) costs and can increase the overall plant lifetime NPV by approximately ~$1B in comparison with NGCC solvent-based capture reference cases (NETL Case 31B). Luna Labs has shared its costing tools with several interested partners and customers, which follows a generalizable approach to costing analysis.

Kelly, Jesse↗
DOE OSTI2025

MRCI Subtask 2.4/2.5: Regional/Subregional Analysis and Risk Assessment Final Technical Summary Report

The objective of the Midwest Regional Carbon Initiative (MRCI) project is to implement a collaborative Regional Initiative (RI) to accelerate the deployment of carbon capture, and storage (CCS) in the Midwest-Northeastern quadrant of the United States. This report is a Technical Summary report describing work performed on Tasks 2.4 (Conducting Regional/Subregional Analysis) and 2.5 (Assessing and Managing Risk) during the MRCI project. In Task 2.4, detailed numerical reservoir simulation models were developed for selected carbon storage (CS) systems identified under Task 2.1 (Battelle, 2021a) in the MRCI study area. The objective of Task 2.4 is to demonstrate a dynamic modeling methodology for evaluating the suitability of the (selected) CS systems in the MRCI region for hosting a commercial-scale storage project. In this study, CO2 injectivity was evaluated for different CS systems with an annual injection rate of 1 million metric tonnes (MMT) of CO2 considered as the minimum requirement for a commercial scale project. The objective of Task 2.5 is to assess key risks associated with storing CO2 in the different CS systems across the MRCI region and to demonstrate a method(s) for assessing these risks that may be used by developers of future CO2 storage projects in the region. The risk analysis was limited to evaluating two types of leakage risks (i.e., wellbore leakage, flow across unfractured caprock) at three modeled sites considered in Task 2.4.

MRCI,Report,Summary,Technical Challenges,dynamic m↗
DOE OSTI2025

Trace Element Analyses of Micron-Size Particles and Statistical Determination of Minimum Detection Limits

Savannah River National Laboratory (SRNL) has developed expertise in producing homogeneous, ca. 1 m-diameter spherical particles of mixed-element components, wherein dopants can be varied from a trace constituent (ppm) to wt.% concentrations. The samples used for this work are nickel-doped cerium oxide microspheres produced by SRNL. They were initially selected as analogs for plutonium-doped uranium oxide particles and analyzed as part of a larger study to evaluate whether electron probe microanalyzers (EPMA) can be used to characterize nuclear materials as an alternative or complementary method to mass spectrometers. The five samples used in this study contained nominal compositions of 0, 0.004, 0.04, 0.4 and 4 wt.% Ni. They were analyzed by both an Agilent 7900 Q-ICP-MS at SRNL and the JEOL JXA&#x2;8530F Plus EPMA at the University of Minnesota. In addition to EPMA results (calibrated with high-precision Q-ICP-MS analyses) suggesting that the EPMA could address outstanding nuclear material characterization needs, these samples 1) showcase the ability of the EPMA to quantify not just trace concentrations, but trace concentrations in microparticles (1 m diameter, Fig. 1), and 2) offer a unique opportunity to evaluate the methodology for assessing the minimum detection limits of EPMA analyses.

McSwiggen, Peter [JEOL USA, 11 Dearborn Road, Peab↗
DOE OSTI2023

Complete Performance Comparison Between the Optimized Image Construction Algorithm (U-MBIR) and the Existing Reconstruction Algorithm for Detecting Defects and Damage in Concrete

Reinforced concrete (RC) is a composite material subjected to mechanical, thermal, and chemical loads throughout its service life. Because of these external stressors and the susceptibility of RC structural members to shrinkage and microcracking, the material degrades throughout its life cycle. This deterioration can lead to a decrease in member capacity and, ultimately, poses a threat to the structural integrity. Thus, it is crucial that the damage caused by aging and degradation be monitored and assessed at regular intervals throughout the material’s service life. Since coring of the material is typically not feasible for in-service structural systems, non-destructive evaluation (NDE) methodologies are used to assess remaining structural capacity. NDE methods enable surface and subsurface examination without damaging or degrading the medium. Moreover, RC is a critical component of nuclear power plants; thus, its safety and reliability must be thoroughly examined throughout the life cycle of the structural system. Ultrasonic measurements have been an industry standard for both surface and subsurface inspections. To this end, Oak Ridge National Laboratory (ORNL) has researched and developed advanced image reconstruction algorithms to capture internal damage. The results and discussion presented herein summarize the current state of the ultrasonic model–based iterative reconstruction (U-MBIR) algorithm developed at ORNL. More specifically, this report presents a comparison between reconstruction images produced via a widely employed ultrasonic NDE technique—the synthetic aperture focusing technique (SAFT)—and the ORNL-developed U-MBIR algorithm. These NDE methodologies are demonstrated using ultrasonic data collected from four concrete specimens. Overall, the U-MBIR algorithm eliminates artifacts and noise that are typically present within the SAFT reconstructions, and it shows defects and anomalies more clearly than the SAFT images. In conclusion, this algorithm is suitable for identifying concrete defects, although more improvements and optimization could be done to better define internal defects.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Unique safety features and licensing requirements of the NuScale small modular reactor

Small modular reactors (SMR) offer a novel approach to the construction and operation of nuclear power plants. The NuScale VOYGR™ plant uses a simplified SMR design that is based on proven light-water reactor technology with substantial improvements in nuclear safety. It consists of a 250 MWt reactor core housed with other primary system components in an integral reactor pressure vessel surrounded by a steel containment vessel, all of which is immersed in a large pool of water that also serves as the ultimate heat sink. At the core of the NuScale safety case are three primary safety systems: the decay heat removal system, the emergency core cooling system, and the containment. The ability of the NuScale Power Module (NPM) passive safety systems to remove core decay heat for an unlimited duration is demonstrated through analysis of a beyond-design-basis extended loss of AC power with no replenishment of water to the ultimate heat sink or operator actions. The NuScale methodology to evaluate an indefinite loss of AC power uses the proprietary NRELAP5 systems analysis computer code. Analysis results show that the reactor coolant system liquid level above the core is maintained and that containment pressure remains below the vessel design pressure. Once full passive air cooling is established, containment pressure and temperature will decrease over time with decreasing core decay heat. NuScale received standard design approval in September 2020 and design certification in January 2023 for its 50 MWe NPM configured as a 12 module plant. NuScale is currently seeking standard design approval to increase its core power to 250 MWt, nominally 77 MWe per module, in a 6-module plant (VOYGR™-6) configuration. The high-level safety of NuScale’s SMR technology is foundational to a new standard of nuclear power plant resilience.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Deep learning methods for drug response prediction in cancer: Predominant and emerging trends

Cancer claims millions of lives yearly worldwide. While many therapies have been made available in recent years, by in large cancer remains unsolved. Exploiting computational predictive models to study and treat cancer holds great promise in improving drug development and personalized design of treatment plans, ultimately suppressing tumors, alleviating suffering, and prolonging lives of patients. A wave of recent papers demonstrates promising results in predicting cancer response to drug treatments while utilizing deep learning methods. These papers investigate diverse data representations, neural network architectures, learning methodologies, and evaluations schemes. However, deciphering promising predominant and emerging trends is difficult due to the variety of explored methods and lack of standardized framework for comparing drug response prediction models. To obtain a comprehensive landscape of deep learning methods, we conducted an extensive search and analysis of deep learning models that predict the response to single drug treatments. A total of 61 deep learning-based models have been curated, and summary plots were generated. Based on the analysis, observable patterns and prevalence of methods have been revealed. This review allows to better understand the current state of the field and identify major challenges and promising solution paths.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Impact Analysis of High-Altitude Electromagnetic Pulse Coupling Effects on Power Grid Protection Relays

Protection relays are important equipment used for protection, control, and metering functions in the power grid. These relays are used to protect critical and difficult-to-replace equipment, like generators, transformers, and capacitor banks. Once the protection devices are disturbed or damaged, a high risk of power generation interruption occurs. Therefore, it is important and necessary to study the relay’s immunity to electromagnetic pulse (EMP) events. As a preliminary step toward empirical experimentation on actual equipment, this manuscript outlines an economical and efficient methodology for evaluating the impact of an EMP. An impedance measurement strategy was employed to model the equipment, setting the stage for subsequent immunity analyses. These analyses included the pulse current injection (PCI) method, which utilized an injecting probe to introduce the transient, and frequency domain electromagnetic (FEKO) simulation, which integrated electromagnetic coupling effects into the transient simulation. The impedance measurement and simulation results in this paper provide a reliable basis for gauging equipment performance in the face of HEMP threats. The results obtained using the PCI and FEKO simulations demonstrated the performance of different port responses under a high-altitude EMP, indicating the requirement for some protection to ensure the reliable operation of relays.

42 ENGINEERING↗
DOE OSTI2024

Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology Using Large Language Models—A Case in Optimizing Intermodal Freight Transportation

The incorporation of Artificial Intelligence (AI) models into various optimization systems is on the rise. However, addressing complex urban and environmental management challenges often demands deep expertise in domain science and informatics. This expertise is essential for deriving data and simulation-driven insights that support informed decision-making. In this context, we investigate the potential of leveraging the pre-trained Large Language Models (LLMs) to create knowledge representations for supporting operations research. By adopting ChatGPT-4 API as the reasoning core, we outline an applied workflow that encompasses natural language processing, Methontology-based prompt tuning, and Generative Pre-trained Transformer (GPT), to automate the construction of scenario-based ontologies using existing research articles and technical manuals of urban datasets and simulations. From these ontologies, knowledge graphs can be derived using widely adopted formats and protocols, guiding various tasks towards data-informed decision support. The performance of our methodology is evaluated through a comparative analysis that contrasts our AI-generated ontology with the widely recognized pizza ontology, commonly used in tutorials for popular ontology software. We conclude with a real-world case study on optimizing the complex system of multi-modal freight transportation. Our approach advances urban decision support systems by enhancing data and metadata modeling, improving data integration and simulation coupling, and guiding the development of decision support strategies and essential software components.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Predictive Analytics for Hydropower Fleet Intelligence

A primary challenge in hydropower industry is the ability to maintain cost-competitiveness, reliability, and security of hydropower assets through evolving power system contexts and aging of the fleet. Maintaining cost-effective and reliable operations under these conditions is expected to require new modernization and maintenance paradigms for changing contexts. Changes in existing practices for O&M will require an understanding of the current state and health of hydropower assets, and the impact of changing paradigms on asset health and reliability. The Hydropower Fleet Intelligence project is developing and evaluating standardized methodologies and analysis tools for data-driven asset reliability and management technologies for hydropower, leading to eventual predictive maintenance planning, repair/replacement decision making, and asset-reliability and cost-optimized operations. A key question is the feasibility of using existing data sets at hydropower facilities to perform assessments of asset reliability. This document uses data from hydropower facilities to assess the potential for using available analytics methods for asset reliability estimates. In addition to reliability assessments, the feasibility of using existing analytics techniques for several other potential applications is discussed. Finally, a case study that a data-driven model is trained to learn nominal operations via vibration data from an asset of a certain plant, and then utilized to identify anomalies on a similar asset from a different plant, highlighting the generic use of proposed Prognostics and Health Management (PHM) approaches.

Yucesan, Yigit↗
DOE OSTI2021

Thermal and Reliability Characterization of an Epoxy Resin-Based Double-Side Cooled Power Module

Wide-Band Gap (WBG) power devices have become a promising option for high-power applications due to the superior material properties over traditional Silicon. To not limit WBG devices’ mother nature, a rugged and high-performance power device packaging solution is necessary. This study proposes a Double-Side Cooled (DSC) 1.2 kV half-bridge power module having dual epoxy resin insulated metal substrate (eIMS) for solving convectional power module challenges and providing a cost-effective solution. The thermal performance outperforms traditional Alumina (Al2O3) Direct Bonded Copper (DBC) DSC power module due to moderate thermal conductivity (10 W/mK) and thin (120 mm) epoxy resin composite dielectric working as the IMS insulation layer. This novel organic dielectric can withstand high voltage (5 kVAC @ 120 μm) and has a Glass Transition Temperature (Tg) of 300°C, which is suitable for high-power applications. In the thermal-mechanical modeling, the organic DSC power module can pass the thermal cycling test over 1,000 cycles by optimizing the mechanical properties of the encapsulant material. In conclusion, this article not only proposes a competitive organic-based power module but also a methodology of evaluation for thermal and mechanical performance.

42 ENGINEERING↗
DOE OSTI2026

Condition monitoring of wind turbine drivetrains: state-of-the-art technologies, recent trends, and future outlook

As global wind capacity expands, reducing operations and maintenance costs is critical to lowering the levelized cost of energy. This paper explores the state of the art in condition monitoring and prognostic strategies for wind turbine drivetrains, which are among the most failure-prone and maintenance-intensive subsystems. Current diagnostic methodologies are evaluated, covering supervisory control and data acquisition (SCADA) data, high-frequency vibration and acoustic analysis, machine learning and digital twin frameworks. Finally, practical challenges are identified that limit wide-scale industrial adoption, in order to guide future research and industrial efforts.

17 WIND ENERGY↗
DOE OSTI2026

Resilience Metrics Framework for Solar Photovoltaics

This presentation was given at the Photovoltaic Specialist Conference (PVSC) 54 in New Orleans, Louisiana. Photovoltaic (PV) systems are routinely exposed to extreme weather, including wind and hail storms. Historically, most systems have proven to be resilient to such events, but some storms have damaged PV systems, leading to physical and financial loss. Storm hardening measures and specific system attributes can reduce this risk. This work introduces a set of resilience metrics and a framework for quantifying, comparing, and predicting PV system resilience. The framework is divided into two parts: 1) predictive, attribute metrics based on site and component characteristics, and 2) impact metrics that assess post-storm performance. Metrics are weighted and aggregated, producing hazard-specific resilience scores. We derive damage functions from storm-impacted PV systems, establishing a baseline against which post-storm performance can be compared. This damage was widely variable across hail and wind intensities, and field hail damage was less than predicted by laboratory tests, suggesting that system features - in addition to storm conditions - influence damage likelihood. Finally, the metrics framework is demonstrated using three case studies of storm damaged PV systems. Although additional data are needed to create attribute specific damage functions and establish metric weights, this study presents a methodology for evaluating PV resilience and contributes new damage functions to the literature.

14 SOLAR ENERGY↗
DOE OSTI2026

Africa Battery Energy Storage Systems (BESS) Capacity Building Utility-Scale Storage: BESS Valuation, Tariffs, and Remuneration [Slides]

Utility-scale Battery Energy Storage Systems (BESS) are key to enhancing grid reliability, integrating renewable energy, and providing operational flexibility. Designing effective valuation, remuneration, and tariff frameworks is essential to ensure both system benefits and financial viability for developers. This presentation outlines a structured methodology for evaluating BESS projects, covering policy and legal considerations, cost and revenue analysis, benchmarking, financial sensitivity, and risk assessment, while ensuring alignment with public interest. It also explores valuation of multiple storage services - bulk energy, ancillary services, and infrastructure support - and monetization strategies through capacity payments, energy tariffs, tolling, arbitrage, and non-wires alternative payments. Technical factors, including round-trip efficiency, degradation, and storage duration, are integrated into financial and operational modeling to quantify both system-wide and project-level benefits. Through case studies and simulation-based approaches, this framework provides regulators, utilities, and developers with practical guidance for tariff design, payment structures, and investment decisions, maximizing the economic and societal value of BESS deployment.

25 ENERGY STORAGE↗
DOE OSTI2021

Wash Vehicle Fleet Sizing for Contingency Planning Against Dust Storms: Preprint

Wash vehicles containing either high- or low-pressure water sprayers, a collection of rotating brushes, or a combination of these, are frequently utilized in concentrating solar power (CSP) plants to maintain a high level of optical efficiency in the solar field. In recent years, multiple modeling approaches have been developed to obtain fleet sizes and mirror-washing schedules that optimize the tradeoff of vehicle capital and use costs and labor versus lost revenues due to soiling. These planning models cover normal operating conditions well but do not consider rare events such as dust storms which can a significant reduction in receiver productivity, or shut down operations until most or all of the solar field’s mirrors have been cleaned. To that end, we propose a methodology that evaluates whether additional capital should be deployed to hedge against these events by weighing the net present value of the expected benefits against the capital costs. The output of this method is a breakeven frequency, a metric we sue to determine whether an additional vehicle should be purchased to address the contingency of dust storms by comparing it to the expected annual storm frequency We develop a small collection of case studies using commercial-scale CSP tower plants and obtain breakeven frequencies that mostly fall between 0.1 and 1.0 storms per year, depending on the existing fleet size and storm severity.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2022

Privacy policy robustness to reverse engineering

Differential privacy policies allow one to preserve data privacy while sharing and analyzing data. However, these policies are susceptible to an array of attacks. In particular, often a portion of the data desired to be privacy protected is exposed online. Access to these pre-privacy protected data samples can then be used to reverse engineer the privacy policy. With knowledge of the generating privacy policy, an attacker can use machine learning to approximate the full set of originating data. Bayesian inference is one method for reverse engineering both model and model parameters. We present a methodology for evaluating and ranking privacy policy robustness to Bayesian inference-based reverse engineering, and demonstrated this method across data with a variety of temporal trends.

Kusne, Aaron Gilad↗
DOE OSTI2021

CCE and Resilience Strategies

Consequence-driven Cyber-informed Engineering (CCE) is a methodology focused on securing the nation’s critical infrastructure systems. Developed at Idaho National Laboratory, CCE starts with the assumption that if a critical infrastructure system is targeted by a skilled and determined adversary, the targeted network can and will be penetrated. This think like the adversary approach provides critical infrastructure owners, operators, vendors and manufacturers with a disciplined methodology to: evaluate complex systems; determine what must be fully safeguarded; and apply proven engineering strategies to isolate and protect an industry’s most critical assets.

42 ENGINEERING↗