Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation protocol”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Analysis of Ion-Exchanged ZSM-5, BEA, and SSZ-13 Zeolite Trapping Materials under Realistic Exhaust Conditions

An industry-defined evaluation protocol was used to evaluate the hydrocarbon trapping (HCT) and passive NOx adsorption (PNA) potential for BEA, ZSM-5, and SSZ-13 zeolites with ion-exchanged Pd or Ag. All materials underwent 700 °C degreening prior to exposure to an industry-derived protocol gas stream, which included NOx, ethylene, toluene, and decane as measured trapping species as well as common exhaust gasses CO, H2O, O2, CO2, and H2. Evaluation showed that BEA and ZSM-5 zeolites were effective at trapping hydrocarbons (HCs), as saturation was not achieved after 30 min of exposure. SSZ-13 also stored HCs but was only able to adsorb 20–25% compared to BEA and ZSM-5. The presence of Ag or Pd did not impact the overall HC uptake, particularly in the first three minutes. Pd/zeolites had significantly lower THC release temperature, and it aided in the conversion of the released HCs; Ag only had a moderate effect in both areas. With respect to NOx adsorption, the level of uptake was much lower than HCs on all samples, and Ag or Pd was necessary with Pd being notably more effective. Additionally, only Pd/ZSM-5 and Pd/SSZ-13 continue to store a portion of the NOx above 200 °C, which is critical for downstream selective catalytic NOx reduction (SCR). Hydrothermal aging (800 °C for 50 h) of a subset of the samples were performed: BEA, Pd/BEA, ZSM-5, Pd/ZSM-5, and Pd/SSZ-13. There was a minimal effect on the HC storage, ~10% reduction in capacity with no effect on release temperature; however, only Pd/SSZ-13 showed significant NOx storage after aging.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Modeling Protocol for Evaluation of Air Emissions for the Idaho National Laboratory Site-wide Permit to Construct and Facility Emission Cap (PTC/FEC)

This document outlines the methods to be used for calculating non-radiological air quality impacts from proposed emissions at the Idaho National Laboratory (INL) Site in support of applying for a renewal to a Permit to Construct with a Facility Emissions Cap (PTC/FEC). The INL Site is seeking to maintain a state enforceable limit on emissions to maintain synthetic minor status in accordance with Idaho Administrative Procedures Act (IDAPA) 58.01.01.175-181 and IDAPA 58.01.01.200-228.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Best Practices and Testing Protocols for Benchmarking ORR Activities of Fuel Cell Electrocatalysts Using Rotating Disk Electrode

Abstract Thin-film-rotating disk electrodes (TF-RDEs) are the half-cell electrochemical system of choice for rapid screening of oxygen reduction reaction (ORR) activity of novel Pt supported on carbon black supports (Pt/C) electrocatalysts. It has been shown that the magnitude of the measured ORR activity and reproducibility are highly dependent on the system cleanliness, evaluation protocols, and operating conditions as well as ink formulation, composition, film drying, and the resultant film thickness and uniformity. Accurate benchmarks of baseline Pt/C catalysts evaluated using standardized protocols and best practices are necessary to expedite ultra-low-platinum group metal (PGM) catalyst development that is crucial for the imminent commercialization of fuel cell vehicles. We report results of evaluation in three independent laboratories of Pt/C electrocatalysts provided by commercial fuel cell catalyst manufacturers (Johnson Matthey, Umicore, Tanaka Kikinzoku Kogyo—TKK). The studies were conducted using identical evaluation protocols/ink formulation/film fabrication albeit employing unique electrochemical cell designs specific to each laboratory. The ORR activities reported in this work provide a baseline and criteria for selection and scale-up of novel high activity ORR electrocatalysts for implementation in proton exchange membrane fuel cells (PEMFCs).

Kocha, Shyam S.↗
DOE OSTI2015

Performance Test Protocol for Evaluating Inverters Used in Grid-Connected Photovoltaic Systems

Interconnection equipment that connects distributed resources (DR) to an electrical power system (EPS) are expected to do so efficiently. Standardized test procedures are necessary to establish methods for verifying inverter performance that leads to comparable results. These test procedures are provided as a repeatable, independent means of measuring inverter performance regarding maximum continuous power rating, conversion efficiency, and tare loss characteristics.

14 SOLAR ENERGY↗
DOE OSTI2022

Durability Evaluation of Advanced Fenestration Technologies

In the U.S., more than 40% of primary energy and 70% of electricity is consumed in residential and commercial buildings, resulting in annual energy costs of more than $430 billion. Approximately 35% of this consumption can be attributed to losses through the building envelope, of which windows are currently the weakest link. Multiple technologies are under development to improve this performance. This includes dynamic and photovoltaic glazing as well as emerging highly insulating technologies including vacuum insulating glass (VIG), aerogels and thin-glass based multi-pane configurations. While windows are specified based on expected performance as installed, the energy savings realized by high performance windows are delivered over time. This makes it critical to understand and maximize the durability of high-performance windows to ensure those projected energy savings are delivered. Present methods for evaluating durability are based on existing technologies. These methods may not apply or be adequate for newer emerging technologies which often present novel failure mechanisms that need to be understood and evaluated differently. In this presentation, we will discuss our efforts to define appropriate methods to improve the evaluation of many existing technologies as well as evaluate the durability of emerging window technologies. This work results from the combination of an extensive review of various international standards as well as existing scientific literature. In addition, input was gathered from multiple industry stakeholders regarding present practice as well as feedback on proposed improvements to existing methods. Here we will share these findings and present proposed improvements to developing and existing durability evaluation protocols.

building envelope↗
DOE OSTI2022

Model Assumptions and Data Characteristics: Impacts on Domain Adaptation in Building Segmentation

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influences the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as a case study, we perform a large-scale study across over 200 DA scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the performances of two DA methods based on adversarial learning that differ only in their assumptions about source and target domains. Informed by metadata and data characteristics unveiled using traditional computer vision (CV) techniques as well as pretrained deep models, we provide a detailed meta-analysis of experiments highlighting the importance of accurately considering data assumptions for DA in RS segmentation tasks. As demonstrated by a “cherry-picking” exercise, different claims regarding which model is best could be made by selecting different subsets of evaluation scenarios. While well-calibrated assumptions can be beneficial, mismatching assumptions can lead to negative biases in DA applications. Furthermore, this study intends to motivate the community toward more consistent evaluation protocols while providing recommendations and insights toward creating novel benchmark datasets, documenting data characteristics, application-specific knowledge, and model assumptions.

42 ENGINEERING↗
DOE OSTI2026

Scaling open-weight large language models for hydropower regulatory information extraction: A systematic analysis

Information extraction from regulatory and technical documents using large language models (LLMs) involves practical trade-offs between extraction quality and computational cost. We evaluate eight open-weight LLMs spanning 0.6B–70B parameters on hydropower licensing documents and report deployment-oriented evidence under a unified extraction schema and evaluation protocol. Across the model set, we observe clear scale-dependent trends in both baseline extraction quality and the effectiveness of reflective reasoning (self-checking) under our fixed-prompt, no-augmentation setting. Mid-scale models often provide a favorable balance of accuracy and efficiency, whereas the smallest models show limited or inconsistent gains from the reasoning variants tested. Larger models achieve the highest overall F1 scores but incur substantially greater compute and infrastructure requirements. We further find that reliability failure modes can distort conventional metrics in this domain: in particular, high recall can coincide with systematic extraction errors when models fabricate values for fields that are absent from the source text, underscoring the importance of conservative null handling and evidence-grounded evaluation. Overall, our study provides a reproducible resource–performance comparison for open-weight LLM-based extraction in hydropower regulatory documentation and offers practical guidance for model selection under different deployment constraints.

Evaluation protocol↗
DOE OSTI2014

SU-E-J-35: Clinical Performance Evaluation of a Phase II Proton CT Scanner

Purpose: To develop the methodology to evaluate the clinical performance of a Phase II Proton CT scanner Methods: Range errors on the order of 3%-5% constitute a major uncertainty in current charged particle treatment planning based on Hounsfield Unit (HU)-relative stopping power (RSP) calibration curves. Within our proton CT collaboration, we previously developed and built a Phase I proton CT scanner that provided a sensitive area of 9 cm (axial) × 18 cm (in-plane). This scanner served to get initial experience with this new treatment planning tool and to incorporate lessons learned into the next generation design. A Phase II scanner was recently completed and is now undergoing initial performance testing. It will increase the proton acquisition rate and provide a larger detection area of 9 cm x 36 cm. We are now designing a comprehensive evaluation program to test the image quality, imaging dose, and range uncertainty associated with this scanner. The testing will be performed along the lines of AAPM TG 66. Results: In our discussion of the evaluation protocol we identified the following priorities. The image quality of proton CT images, in particular spatial resolution and low-density contrast discrimination, will be evaluated with the Catphan600 phantom. Initial testing showed that the Catphan uniformity phantom did not provide sufficient uniformity; it was thus replaced by a cylindrical water phantom. The imaging dose will be tested with a Catphan dose module, and compared to a typical cone beam CT dose for comparable image quality. Lastly, we developed a dedicated dosimetry range phantom based on the CIRS pediatric head phantom HN715. Conclusion: A formal evaluation of proton CT as a new tool for proton treatment planning is an important task. The availability of the new Phase II proton CT scanner will allow us to perform this task. This research is supported by the National Institute of Biomedical Imaging and Bioengineering of the NIH under award number R01EB013118. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Exponential Backoff and Its Security Implications for Safety-Critical OT Protocols over TCP/IP Networks

The convergence of Operational Technology (OT) and Information Technology (IT) networks has become increasingly prevalent with the growth of Industrial Internet of Things (IIoT) applications. This shift, while enabling enhanced automation, remote monitoring, and data sharing, also introduces new challenges related to communication latency and cybersecurity. Oftentimes, legacy OT protocols were adapted to the TCP/IP stack without an extensive review of the ramifications to their robustness, performance, or safety objectives. To further accommodate the IT/OT convergence, protocol gateways were introduced to facilitate the migration from serial protocols to TCP/IP protocol stacks within modern IT/OT infrastructure. However, they often introduce additional vulnerabilities by exposing traditionally isolated protocols to external threats. This study investigates the security and reliability implications of migrating serial protocols to TCP/IP stacks and the impact of protocol gateways, utilizing two widely used OT protocols: Modbus TCP and DNP3. Our protocol analysis finds a significant safety-critical vulnerability resulting from this migration, and our subsequent tests clearly demonstrate its presence and impact. A multi-tiered testbed, consisting of both physical and emulated components, is used to evaluate protocol performance and the effects of device-specific implementation flaws. Through this analysis of specifications and behaviors during communication interruptions, we identify critical differences in fault handling and the impact on time-sensitive data delivery. The findings highlight how reliance on lower-level IT protocols can undermine OT system resilience, and they inform the development of mitigation strategies to enhance the robustness of industrial communication networks.

DNP3↗
DOE OSTI2025

An Evaluation of Representation Learning Methods in Particle Physics Foundation Models

We present a systematic evaluation of representation learning objectives for particle physics within a unified framework. Our study employs a shared transformer-based particle-cloud encoder with standardized preprocessing, matched sampling, and a consistent evaluation protocol on a jet classification dataset. We compare contrastive (supervised and self-supervised), masked particle modeling, and generative reconstruction objectives under a common training regimen. In addition, we introduce targeted supervised architectural modifications that achieve state-of-the-art performance on benchmark evaluations. This controlled comparison isolates the contributions of the learning objective, highlights their respective strengths and limitations, and provides reproducible baselines. We position this work as a reference point for the future development of foundation models in particle physics, enabling more transparent and robust progress across the community.

Chen, Michael [Caltech]↗
DOE OSTI2015

SU-E-P-15: Technique Factor Modulation and Reference Plane Air Kerma Rates in Response to Simulated Patient Thickness Variations for a Sample of Current Generation Fluoroscopes

Purpose: To evaluate and compare approaches to technique factor modulation and air kerma rates in response to simulated patient thickness variations for four state-of-the-art and one previous-generation interventional fluoroscopes. Methods: A polymethyl methacrylate (PMMA) phantom was used as a tissue surrogate for the purposes of determining fluoroscopic reference plane air kerma rates, kVp, mA, and spectral filtration over a wide range of simulated tissue thicknesses. Data were acquired for each fluoroscopic and acquisition dose curve within a default abdomen or body imaging protocol. Results: The data obtained indicated vendor- and model-specific variations in the approach to technique factor modulation and reference plane air kerma rates across a range of tissue thicknesses. Some vendors have made hardware advances increasing the radiation output capabilities of their fluoroscopes; this was evident in the acquisition air kerma rates. However, in the imaging protocol evaluated, all of the state-of-the-art systems had relatively low air kerma rates in the fluoroscopic low-dose imaging mode as compared to the previous-generation unit. Each of the newest-generation systems also employ copper filtration in the selected protocol in the acquisition mode of imaging; this is a substantial benefit, reducing the skin entrance dose to the patient in the highest dose-rate mode of fluoroscope operation. Conclusion: Understanding how fluoroscopic technique factors are modulated provides insight into the vendor-specific image acquisition approach and provides opportunities to optimize the imaging protocols for clinical practice. The enhanced radiation output capabilities of some of the fluoroscopes may, under specific conditions, may be beneficial; however, these higher output capabilities also have the potential to lead to unnecessarily high dose rates. Therefore, all parties involved in imaging, including the clinical team, medical physicists, and imaging vendors, must work together to ensure that adequate but not excessive radiation doses are used.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2014

TU-PIS-Exhibit Hall-01: CT Dose Optimization Technologies II

Partners in Solutions is an exciting new program in which AAPM partners with our vendors to present practical “hands-on” information about the equipment and software systems that we use in our clinics. The imaging topic this year is CT scanner dose optimization capabilities. Note that the sessions are being held in a special purpose room built on the Exhibit Hall Floor, to encourage further interaction with the vendors. Siemens‘ Commitment to the Right Dose in Computed Tomography Presentation Time: 11:15 - 11:45 AM Providing sustainable clinical results at highest patient safety: This is the challenge in medical imaging. Especially for Computed Tomography this means applying not simply the lowest, but the right dose for sound diagnostic imaging. Consequently, Siemens is committed to deliver the right dose in CT. In order to reduce radiation to the right dose, the first step is to provide the right dose technology. Through decades of research and development in CT imaging, Siemens CT has constantly introduced new ideas leading to a comprehensive portfolio of unique CARE technologies to deliver the right dose. For example automated kV adjustment based on patient size and the clinical question with CARE kV and three generations of iterative reconstruction. Based on the right dose technology, the next step is to actually scan at the right dose. For this, it is key to know the right dose targets for every examination. Siemens continuously involves CT experts to push developments further and outline how users can best adapt their procedures to the right dose. For users to know whether they met the right dose targets, it is therefore important to understand and monitor the actual absolute dose values. All scanners are delivered with defined default protocols which automatically use the available right dose technologies. Finally, to deliver the right dose not just in singular cases, but ideally to patients everywhere, organizations need then to manage dose across their institutions. Here Siemens offers the right dose management tools for administrations to analyze and document dose, to train their clinical staff and to continuously optimize the application of the right dose in clinical routine. Siemens CT is not only adhering to the ALARA principle, but is inspired by it. With CARE Right Siemens show their commitment to the right dose in CT. Toshiba Aquilion CT Dose Management Tools Presentation Time: 11:45 - 12:15 PM A practical review of Toshiba’s CT dose management technologies from the physicist’s perspective. This presentation will include an overview of Toshiba’s CT dose reduction technologies and dose safeguards. Content will include answers to frequently asked questions relating to dose management, dose tracking, CTDI measurement, ACR accreditation, ACR QC, and protocol review on Toshiba CT systems. Specific topics will include the following: Toshiba’s mechanism of tube current modulation; Exposure record format and interpretation of the values; Dose reporting and dose safeguard tools; Use of AIDR 3D iterative reconstruction; The configuration of task specific image quality target and its integration with AIDR 3D; Tips for scanning the ACR phantom with wide beam geometry; Measuring CTDI for wide beam geometry; Discussion of daily QC procedures and specifications; Tips for annual QC evaluation; Protocol management tools.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Taming the Wild West: Assessing Impacts-Relevant Climate Data Products (Abbreviated Report)

Impacts-relevant Earth system data refers to observational and ESM data that are downscaled, debiased, validated, and provisioned for use by decision-makers. Impacts-relevant Earth system data is essential for mitigation and adaptation planning across a variety of regions and sectors. A vast number of these data products have emerged in recent years, which has led to confusion among stakeholders and scientists as to the best product to use. With no standard evaluation protocol available for these products, the decision on which product to use was sometimes made because it was pragmatic rather than the best product to use. This project sought to develop foundational capabilities around impacts-relevant data products that would support more informed selection and application of these products. This work has been immensely successful, driving several academic publications and supported the development of a community of practice around impacts-relevant data products. Over the project’s three years we have addressed six tasks: First, the development of standard evaluation metrics for impacts-relevant climate data; second, the development of a novel suite of atmospheric river metrics; third, the development of novel metrics for precipitation feature analysis; fourth, the development of novel metrics for assessing co-variances between temperature and precipitation; fifth, the development of a dashboard for interactive examination of impacts-relevant climate data; and sixth, the establishment of a community of practice around impacts-relevant climate data that will continue beyond the conclusion of this project.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2012

Concurrent Chemoradiotherapy Followed by Consolidation Chemotherapy With Bi-Weekly Docetaxel and Carboplatin for Stage III Unresectable, Non-Small-Cell Lung Cancer: Clinical Application of a Protocol Used in a Previous Phase II Study

Purpose: To assess the clinical applicability of a protocol evaluated in a previously reported phase II study of concurrent chemoradiotherapy followed by consolidation chemotherapy with bi-weekly docetaxel and carboplatin in patients with stage III, unresectable, non-small-cell lung cancer (NSCLC). Methods and Materials: Between January 2000 and March 2006, 116 previously untreated patients with histologically proven, stage III NSCLC were treated with concurrent chemoradiotherapy. Radiation therapy was administered in 2-Gy daily fractions to a total dose of 60 Gy in combination with docetaxel, 30 mg/m{sup 2}, and carboplatin at an area under the curve value of 3 every 2 weeks during and after radiation therapy. Results: The median survival time for the entire group was 25.5 months. The actuarial 2-year and 5-year overall survival rates were 53% and 31%, respectively. The 3-year cause-specific survival rate was 60% in patients with stage IIIA disease, whereas it was 35% in patients with stage IIIB disease (p = 0.007). The actuarial 2-year and 5-year local control rates were 62% and 55%, respectively. Acute hematologic toxicities of Grade {>=}3 severity were observed in 20.7% of patients, while radiation pneumonitis and esophagitis of Grade {>=}3 severity were observed in 2.6% and 1.7% of patients, respectively. Conclusions: The feasibility of the protocol used in the previous phase II study was reconfirmed in this series, and excellent treatment results were achieved.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2014

Phase 2 Trial of Induction Gemcitabine, Oxaliplatin, and Cetuximab Followed by Selective Capecitabine-Based Chemoradiation in Patients With Borderline Resectable or Unresectable Locally Advanced Pancreatic Cancer

Purpose: To evaluate, in a phase 2 study, the safety and efficacy of induction gemcitabine, oxaliplatin, and cetuximab followed by selective capecitabine-based chemoradiation in patients with borderline resectable or unresectable locally advanced pancreatic cancer (BRPC or LAPC, respectively). Methods and Materials: Patients received gemcitabine and oxaliplatin chemotherapy repeated every 14 days for 6 cycles, combined with weekly cetuximab. Patients were then restaged; “downstaged” patients with resectable disease underwent attempted resection. Remaining patients were treated with chemoradiation consisting of intensity modulated radiation therapy (54 Gy) and concurrent capecitabine; patients with borderline resectable disease or better at restaging underwent attempted resection. Results: A total of 39 patients were enrolled, of whom 37 were evaluable. Protocol treatment was generally well tolerated. Median follow-up for all patients was 11.9 months. Overall, 29.7% of patients underwent R0 surgical resection (69.2% of patients with BRPC; 8.3% of patients with LAPC). Overall 6-month progression-free survival (PFS) was 62%, and median PFS was 10.4 months. Median overall survival (OS) was 11.8 months. In patients with LAPC, median OS was 9.3 months; in patients with BRPC, median OS was 24.1 months. In the group of patients who underwent R0 resection (all of which were R0 resections), median survival had not yet been reached at the time of analysis. Conclusions: This regimen was well tolerated in patients with BRPC or LAPC, and almost one-third of patients underwent R0 resection. Although OS for the entire cohort was comparable to that in historical controls, PFS and OS in patients with BRPC and/or who underwent R0 resection was markedly improved.

62 RADIOLOGY AND NUCLEAR MEDICINE↗