Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation methodology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

KSC management training system project

The stated objectives for the summer of 1993 were: to review the Individual Development Plan Surveys for 1994 in order to automate the analysis of the Needs Assessment effort; and to develop and implement evaluation methodologies to perform ongoing program-wide course-to-course assessment. This includes the following: to propose a methodology to develop and implement objective, performance-based assessment instruments for each training effort; to mechanize course evaluation forms and develop software to facilitate the data gathering, analysis, and reporting processes; and to implement the methodology, forms, and software in at lease one training course or seminar selected among those normally offered in the summer at KSC. Section two of this report addresses the work done in regard to the Individual Development Plan Surveys for 1994. Section three presents the methodology proposed to develop and implement objective, performance-based assessment instruments for each training course offered at KSC.

Sepulveda, Jose A.↗
DOE OSTI2023

Operational resilience metrics for power systems with penetration of renewable resources

Abstract Modern power grid is evolving towards carbon neutrality by deploying increasing amount of renewable energy resources. However, the impact of renewable generation on power system planning and operation is not sufficiently investigated, especially the capability of renewable penetrated power systems to resist and recover from major disturbances, which is a critical concern for system operators. Novel metrics and evaluation methodologies are needed to depict systems’ ability in response to events caused by natural disasters, and quantitatively evaluate system performance in various time scales. In this paper, operational resilience metrics are proposed for power systems with penetration of renewable energy resources based on transient stability principles. A systematic methodology is proposed to quantitatively assess the evolution of system performance during various stages of the disaster process. Based on the proposed metrics, a resilience‐oriented disaster management strategy is designed and validated using the modified IEEE 39‐bus test system. The simulation results demonstrate the validity of the proposed metrics and strategy, and show that the system resilience is enhanced during the mitigation of fault conditions.

Gui, Jianzhong↗
NASA NTRS2000

SeaWiFS Postlaunch Calibration and Validation Analyses

The effort to resolve data quality issues and improve on the initial data evaluation methodologies of the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) Project was an extensive one. These evaluations have resulted, to date, in three major reprocessings of the entire data set where each reprocessing addressed the data quality issues that could be identified up to the time of each reprocessing. The number of chapters (21) needed to document this extensive work in the SeaWiFS Postlaunch Technical Report Series requires three volumes. The chapters in Volumes 9, 10, and 11 are in a logical order sequencing through sensor calibration, atmospheric correction, masks and flags, product evaluations, and bio-optical algorithms. The first chapter of Volume 9 is an overview of the calibration and validation program, including a table of activities from the inception of the SeaWiFS Project. Chapter 2 describes the fine adjustments of sensor detector knee radiances, i.e., radiance levels where three of the four detectors in each SeaWiFS band saturate. Chapters 3 and 4 describe the analyses of the lunar and solar calibration time series, respectively, which are used to track the temporal changes in radiometric sensitivity in each band. Chapter 5 outlines the procedure used to adjust band 7 relative to band 8 to derive reasonable aerosol radiances in band 7 as compared to those in band 8 in the vicinity of Lanai, Hawaii, the vicarious calibration site. Chapter 6 presents the procedure used to estimate the vicarious calibration gain adjustment factors for bands 1-6 using the waterleaving radiances from the Marine Optical Buoy (MOBY) offshore of Lanai. Chapter 7 provides the adjustments to the coccolithophore flag algorithm which were required for improved performance over the prelaunch version. Chapter 8 is an overview of the numerous modifications to the atmospheric correction algorithm that have been implemented. Chapter 9 describes the methodology used to remove artifacts of sun glint contamination for portions of the imagery outside the sun glint mask. Finally, Chapter 10 explains a modification to the ozone interpolation method to account for actual time differences between the SeaWiFS and Total Ozone Mapping Spectrometer (TOMS) orbits.

Hooker, Stanford B.↗
DOE OSTI2025

EVALUATION OF HRA METHODOLOGIES FOR APPLICATION IN SDP WORK

This study critically evaluates human reliability analysis (HRA) methodologies applicable to regulatory probabilistic safety assessment (PSA) model, with a particular focus on their role in supporting the significance determination process (SDP) in nuclear safety assessment. Firstly, three widely utilized HRA methods – IDHEAS-ECA, SPAR-H, and ASEP/THERP – were qualitatively and quantitatively assessed. Qualitative assessments were conducted using attributes from the NEA/CSNI/R(2015)1 report, while quantitative evaluations employed regression and correlation analyses to compare predicted human error probabilities (HEPs) against empirical data. Results reveal distinct strengths, for example, IDHEAS-ECA’s robust predictive accuracy and K-HRA’s alignment with operational practices. In addition, dependency analysis and recovery analysis were critically evaluated. For dependency analysis, the methods’ handling of inter-task dependencies and their impact on HEPs were examined, while recovery analysis highlighted strategies for mitigating failure events. Furthermore, strategies were proposed to evaluate performance-shaping factors under conditions of reduced human performance, such as stress, fatigue, or cognitive overload, addressing specific challenges faced in SDP evaluations. Human errors from KINS’s operational performance information system event reports were evaluated as a case study. This study identifies gaps and provides actionable insights to ensure their validity and applicability in SDP HRA applications. This paper is a part of research conducted by KINS, and it should be noted that this result does not represent the regulatory position of KINS.

99 - GENERAL AND MISCELLANEOUS↗
NASA NTRS2003

Evaluating Behaviorally Oriented Aviation Maintenance Resource Management (MRM) Training and Programs: Methods, Results, and Conclusions

Assessment of the impact of Aviation Resource Management Programs on aviation culture and performance has compelled a considerable body of research (Taylor & Robertson, 1995; Taylor, 1998; Taylor & Patankar, 2001). In recent years new methods have been applied to the problem of maintenance error precipitated by factors such as the need for self-assessment of communication and trust. The present study - 2002 -- is an extension of that past work. This research project was designed as the conclusion of a larger effort to help understand, evaluate and validate the impact of Maintenance Resource Management (MRM) training programs, and other MRM interventions on participant attitudes, opinions, behaviors, and ultimately on enhanced safety performance. It includes research and development of evaluation methodology as well as examination of psychological constructs and correlates of maintainer performance. In particular, during 2002, three issues were addressed. First, the evaluation of two (independent & different) MRM programs for changing behaviors was undertaken. In one case we were able to further apply the approach to measuring written communication developed during 2001 (Taylor, 2002; Taylor & Thomas, 2003). Second, the MRM/TOQ surveys were made available for completion on the internet. The responses from these on-line surveys were automatically linked to a results calculator (like the one developed and described in Taylor, 2002) to aid industry users in analyzing and evaluating their local survey data on the internet. Third, the main trends and themes from our research about MRM programs over the past dozen years were reviewed.

Taylor, James C.↗
DOE OSTI2025

Initial Design Curves for Alloy 709 for an Improved Creep-fatigue Design Method

Creep-fatigue (CF) interaction damage is the primary damage mode for high-temperature structural components subjected to cyclic loading. Over the past several decades, researchers within the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code (BPVC), Section III, Division 5, have focused on developing elevated temperature code rules to ensure conservative structural designs that mitigate CF failure in high-temperature reactors. The existing CF evaluation methodologies in the Code are based on the creep and fatigue damage diagram approach, which is complex and often excessively conservative. The alternative CF evaluation approach proposed here is intended to significantly simplify the evaluation procedure while reducing conservatism in high-temperature component design analysis. This alternative CF evaluation method integrates the elastic–perfectly plastic (EPP) analysis approach with the simplified model test (SMT) CF design concept, leveraging the advantages of both methods. This report presents the preliminary analysis and the approach for developing CF design curves for Alloy 709, utilizing fatigue and CF data generated for the 100,000-hr Code Case to support its qualification to ASME Section III, Division 5 for Class A construction of high temperature reactors. This study is to support the incorporation of Alloy 709 in this alternative CF evaluation method. Recommendations for the remaining work needed to complete the effort are also provided.

36 MATERIALS SCIENCE↗
NASA NTRS2008

Validation of On-Orbit Methodology for the Assessment of Cardiac Function and Changes in the Circulating Volume Using Ultrasound and "Braslet-M" Occlusion Cuffs

Recent advances in remotely guided imaging techniques on ISS allow the acquisition of high quality ultrasound data using crewmember operators with no medical background and minimal training. However, ongoing efforts are required to develop and validate methodology for complex imaging protocols to ensure their repeatability, efficiency, and suitability for use aboard the ISS. This Station Developmental Test Objective (SDTO) tests a cardiovascular evaluation methodology that takes advantage of the ISS Ultrasound capability, the Braslet-M device, and modified respiratory maneuvers (Valsalva and Mueller), to broaden the spectrum of anatomical and functional information on human cardiovascular system during long-duration space missions. The proposed methodology optimizes and combines new and previously demonstrated methods, and is expected to benefit medically indicated assessments, operational research protocols, and data collections for science. Braslet-M is a current Russian operational countermeasure that compresses the upper thigh to impede the venous return from lower extremities. The goal of the SDTO is to establish and validate a repeatable ultrasound-based methodology for the assessment of a number of cardiovascular criteria in microgravity. Braslet-M device is used as a means to acutely alter volume distribution while focused ultrasound measurements are performed. Modified respiratory maneuvers are done upon volume manipulations to record commensurate changes in anatomical and functional parameters. The overall cardiovascular effects of the Braslet-M device are not completely understood, and although not a primary objective of this SDTO, this effort will provide pilot data regarding the suitability of Braslet-M for its intended purpose, effects, and the indications for its use.

Bogomolov, V. V.↗
NASA NTRS1991

Space transportation main engine cycle assessment process

The Advanced Launch System (ALS) program selection process for a space transportation main engine (STME) power cycle is described in terms of the methodology employed. Low cost, robustness, and high reliability are the primary parameters for engine choice, suggesting simplicity of design and efficient fabrication methods as the crucial characteristics. An evaluation methodology is developed based on the Pugh (1981) process and the King (1989) matrices. The cycle configurations considered are the gas generator (GG), the closed expander, and the open expander. The cycle assessment team determined that the GG cycle is favored by most cycle discriminators, based on an assessment of the characteristics in terms of ALS goals. The lower development risk of the GG-cycle STME is consistent with the goals of the ALS program in terms of reliability and cost efficiency.

Mcconnaughey, H. V.↗
DOE OSTI2021

Utilizing the Dynamic Networks Data Processing and Analysis Experiment (DNE18) to Establish Methodologies for the Comparison of Automatic Infrasonic Signal Detectors

The Dynamic Networks Experiment 2018 (DNE18) was a collaborative effort between Los Alamos National Laboratory (LANL), Sandia National Laboratories (SNL), Lawrence Livermore National Laboratory (LLNL) and Pacific Northwest National Laboratory (PNNL) designed to evaluate methodologies for multi-modal data ingestion and processing. One component of this virtual experiment was a quantitative assessment of current capabilities for infrasound data processing, beginning with the establishment of a baseline for infrasound signal detection. To produce such baselines, SNL and LANL exploited a common dataset of infrasound data recorded across a regional network in Utah from December 2010 through February 2011. We utilize two automated signal detectors, the Adaptive F-Detector (AFD) and the Multivariate Adaptive Learning Detector (MALD) to produce automated signal detection catalogs and an analyst-produced catalog. Comparisons indicate that automatic detectors may be able to identify small amplitude, low SNR events that cannot be identified by analyst review. We document detector performance in terms of precision and recall, demonstrating that the AFD is more precise, but the MALD has higher recall. We use a synthetic dataset of signals embedded in pink noise in order to highlight shortcomings in assessing detection algorithms for low signal to noise ratio signals which are commonly of interest to the nuclear monitoring community. For comparisons utilizing the synthetic dataset, the AFD has higher recall while precision is equal for both detectors. These results indicate that both detectors perform well across a variety of background noise environments; however, both detectors fail to identify repetitive, short duration signals arriving from similar backazimuths. These failures represent specific scenarios that could be targeted for further detector development.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

$\mathcal{R}$ -matrix analysis of η + nat Cl reactions relevant to molten-salt reactor designs [Abstract]

Performed with support from the U.S. Nuclear Criticality Safety Program in an effort to provide improved chlorine cross section and corresponding covariance data, the R-matrix analysis of neutron induced reactions for two stable chlorine isotopes, 35,37 Cl, was performed in the energy range of thermal up to 1.2 MeV. Starting from the repository of the ENDF/B-VIII.0 library and following recent measurement campaigns, this work represents a significant improvement in the evaluation of the ( η , p ) reaction channel. The evaluation methodology uses the $\mathcal{R}$ -matrix code SAMMY to generate a set of Reich-Moore resonance parameters. As predicted by recently measured 35 Cl( η , p ) data, the presented evaluation features a dramatic increase in the magnitude of the ( η , p ) reaction channel over the ENDF/B-VIII.0 and previous nuclear data ENDF/B released libraries. Together with details of the evaluation procedure, the impact of the updates in the outgoing proton emissions on reactivity coefficients for different molten-salt reactor designs is presented and discussed.

charged-particle↗
DOE OSTI2025

R -matrix analysis of n + nat Cl reactions up to 1.2 MeV

The R -matrix analysis of neutron-induced reactions for two stable chlorine isotopes ( 35 Cl and 37 Cl) was performed in the energy range of thermal up to 1.2 MeV. Starting from the repository of the ENDF/B-VIII.0 library and following recent measurement series, this work represents a significant improvement, particularly in the evaluation of the ( n , p ) reaction channel. The evaluation methodology used the R -matrix code SAMMY to generate a set of Reich–Moore resonance parameters for both stable chlorine isotopes. Consistent with recently measured 35 Cl( n , p ) data, the presented evaluation features a dramatic increase in the magnitude of the ( n , p ) reaction channel over the ENDF/B-VIII.1 nuclear data library and previous ENDF/B libraries.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1980

Methodology for urban rail and construction technology research and development planning

A series of transit system visits, organized by the American Public Transit Association (APTA), was conducted in which the system operators identified the most pressing development needs. These varied by property and were reformulated into a series of potential projects. To assist in the evaluation, a data base useful for estimating the present capital and operating costs of various transit system elements was generated from published data. An evaluation model was developed which considered the rate of deployment of the research and development project, potential benefits, development time and cost. An outline of an evaluation methodology that considered benefits other than capital and operating cost savings was also presented. During the course of the study, five candidate projects were selected for detailed investigation; (1) air comfort systems; (2) solid state auxiliary power conditioners; (3) door systems; (4) escalators; and (5) fare collection systems. Application of the evaluation model to these five examples showed the usefulness of modeling deployment rates and indicated a need to increase the scope of the model to quantitatively consider reliability impacts.

Rubenstein, L. D.↗
DOE OSTI2020

Bringing Automated Fault Detection and Diagnostics Tools for HVAC&R Into the Mainstream

Heating, ventilation, and air-conditioning (HVAC) systems consume over 5 quads of energy annually, representing 30% of energy consumption in the U.S. commercial buildings. Additionally, commercial refrigeration (R) systems add about 1 quad to commercial buildings energy consumption. Most HVAC systems operate with one or more faults that result in increased energy consumption. Fault detection and diagnostics (FDD) tools have been developed to address this national issue, and many tools are commercially available. FDD tools have the potential to save considerable energy for an existing commercial rooftop unit (RTU) and refrigeration systems. These devices can be used for both retro commissioning and, when faults are addressed, continuous commissioning. However, there appears to be multiple market barriers for this technology. Although there are efforts to develop FDD tool standards, currently there are no standards and methods to define functions, capabilities, accuracy, and reliability of FDD tools in the field. Moreover, most of the commercial FDD tools have not been verified in the field independently. This paper presents a comprehensive approach for bringing HVAC FDD tools into the mainstream. The approach involves demonstrating ten commercially available FDD tools installed at ten different sites, independent testing and evaluation of the FDD tools, communication with various stakeholders, identifying and assessing market barriers, creating a process evaluation methodology, and assisting utility companies in developing incentive programs. Furthermore, the preliminary baseline results from the case study demonstrate how the use of an independent monitoring system (IMS) can be used for ground-truth in evaluating FDD tools in the field.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS1992

Training effectiveness assessment: Methodological problems and issues

The U.S. military uses a large number of simulators to train and sustain the flying skills of helicopter pilots. Despite the enormous resources required to purchase, maintain, and use those simulators, little effort has been expended in assessing their training effectiveness. One reason for this is the lack of an evaluation methodology that yields comprehensive and valid data at a practical cost. Some of these methodological problems and issues that arise in assessing simulator training effectiveness, as well as problems with the classical transfer-of-learning paradigm were discussed.

Cross, Kenneth D.↗
NASA NTRS1990

A test manager's perspective of a test concept for a heavy lift vehicle

The developmment of a test concept is a significant part of the advanced planning activities accomplished for the Initial Operational Test and Evaluation (IOT&E) of new systems. A test concept is generally viewed as a description, including rationale, of the test structure, evaluation methodology and management approach required to plan and conduct the IOT&E of a program such as a new heavy lift launch vehicle system. The test concept as presented in this paper is made up of an operations area, a test area, an evaluation area, and a management area. The description presented here is written from the perspective of one test manager, and represents his views of a possible framework of a test concept using examples for a potential IOT&E of a heavy lift launch vehicle.

Pargeon, John I., Jr.↗
NASA NTRS2002

Tools and Techniques for Evaluating the Effects of Maintenance Resource Management (MRM) in Air Safety

This research project was designed as part of a larger effort to help Human Factors (HF) implementers, and others in the aviation maintenance community, understand, evaluate, and validate the impact of Maintenance Resource Management (MRM) training programs, and other MRM interventions; on participant attitudes, opinions, behaviors, and ultimately on enhanced safety performance. It includes research and development of evaluation methodology as well as examination of psychological constructs and correlates of maintainer performance. In particular, during 2001, three issues were addressed. First a prototype process for measuring performance was developed and used. Second an automated calculator was developed to aid the HF implementer user in analyzing and evaluating local survey data. These results include being automatically compared with the experience from all MRM programs studied since 1991. Third the core survey (the Maintenance Resource Management Technical Operations Questionnaire, or 'MRM/TOQ') was further developed and tested to include topics of added relevance to the industry.

Taylor, James C.↗
DOE OSTI2025

Physics-informed machine learning for building performance simulation-A review of a nascent field

Building performance simulation (BPS) is critical for understanding building dynamics and behavior, analyzing the performance of the built environment, optimizing energy efficiency, improving demand flexibility, and enhancing building resilience. However, conducting BPS is not trivial. Traditional BPS relies on accurate building energy models, which are primarily physics-based and heavily dependent on detailed building information, expert knowledge, and case-by-case model calibrations, significantly limiting their scalability. With the development of sensing technology and the increased availability of data, there is growing attention and interest in data-driven BPS. However, purely data-driven models often suffer from limited generalization ability and a lack of physical consistency, resulting in poor performance in real-world applications. To address these limitations, recent studies have begun integrating physics priors into data-driven models, a methodology known as physics-informed machine learning (PIML). PIML is an emerging field where its definitions, methodologies, evaluation criteria, application scenarios, and future directions remain open. To bridge those gaps, this study systematically reviews the state-of-the-art PIML for BPS, offering a comprehensive definition of PIML and comparing it to traditional BPS approaches regarding data requirements, modeling effort, performance, and computational cost. We also summarize the commonly used methodologies, validation approaches, application domains, available data sources, open-source packages, and testbeds. In addition, this study provides a general guideline for selecting appropriate PIML models based on BPS applications. Finally, this study identifies key challenges and outlines future research directions, providing a solid foundation and valuable insights to advance R&D of PIML in BPS.

Jiang, Zixin↗
NASA NTRS2004

Assurance Technology Challenges of Advanced Space Systems

The initiative to explore space and extend a human presence across our solar system to revisit the moon and Mars post enormous technological challenges to the nation's space agency and aerospace industry. Key areas of technology development needs to enable the endeavor include advanced materials, structures and mechanisms; micro/nano sensors and detectors; power generation, storage and management; advanced thermal and cryogenic control; guidance, navigation and control; command and data handling; advanced propulsion; advanced communication; on-board processing; advanced information technology systems; modular and reconfigurable systems; precision formation flying; solar sails; distributed observing systems; space robotics; and etc. Quality assurance concerns such as functional performance, structural integrity, radiation tolerance, health monitoring, diagnosis, maintenance, calibration, and initialization can affect the performance of systems and subsystems. It is thus imperative to employ innovative nondestructive evaluation methodologies to ensure quality and integrity of advanced space systems. Advancements in integrated multi-functional sensor systems, autonomous inspection approaches, distributed embedded sensors, roaming inspectors, and shape adaptive sensors are sought. Concepts in computational models for signal processing and data interpretation to establish quantitative characterization and event determination are also of interest. Prospective evaluation technologies include ultrasonics, laser ultrasonics, optics and fiber optics, shearography, video optics and metrology, thermography, electromagnetics, acoustic emission, x-ray, data management, biomimetics, and nano-scale sensing approaches for structural health monitoring.

Chern, E. James↗