SEARCH · Search NASA

Results for “evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRS2023

A Framework for Evaluating Distributed Electric Propulsion on the SUSAN Electrofan Aircraft

This work presents a framework for evaluating models and algorithms for Distributed Electric Propulsion (DEP) on the SUSAN Electrofan Aircraft. Throughout the development of the SUSAN aircraft, the performance of various configurations of the aircraft will need to be analyzed. However, the static behavior alone is not sufficient to describe the performance of these configurations. Therefore, simulation with fully integrated subsystem models is required. The proposed framework considers the vehicle aerodynamic, propulsion, and control subsystems. The presented framework automatically generates control laws for any vehicle configuration in response to changes in these subsystems. To compare these different vehicle configurations, various time and frequency domain performance metrics are compared. Three different system modifications are used as cases to evaluate this framework. The first modification integrates the propulsion control system with the flight controller to enable differential thrust without stalling the main engine. This evaluation case is used to validate the framework for aircraft configurations with coupled subsystems. The second modification compares the effect of the vertical tail size on open and closed loop performance. This evaluation case is used to validate the framework for controlling different configurations and tuning towards comparable closed loop performance despite changes to the aircraft's aerodynamic model. The third modification implements two different control allocation schemes. This evaluation case demonstrates the framework's ability to evaluate allocation modifications needed to take advantage of DEP. The first evaluation case is used to show that controller integration enables differential thrust, improving realized wingfan bandwidth by up to 40\% in simulation. The second evaluation case demonstrates that the framework can stabilize the reduced tail size aircraft with closed loop control. The third evaluation case demonstrates that a pseudoinverse control allocation scheme improves lateral velocity settling time by approximately 17~seconds over a symmetric-thrust allocation. These cases show that the framework is useful for evaluating the performance of integrated system designs, enabling analyses of new models and algorithms for the SUSAN distributed electric propulsion vehicle.

Nicholas C Ogden↗

NASA NTRS1974

Engineering flight and guest pilot evaluation report, phase 2

Prior to the flight evaluation, the two-segment profile capabilities of the DC-8-61 were evaluated and flight procedures were developed in a flight simulator at the UA Flight Training Center in Denver, Colorado. The flight evaluation reported was conducted to determine the validity of the simulation results, further develop the procedures and use of the area navigation system in the terminal area, certify the system for line operation, and obtain evaluations of the system and procedures by a number of pilots from the industry. The full area navigation capabilities of the special equipment installed were developed to provide terminal area guidance for two-segment approaches. The objectives of this evaluation were: (1) perform an engineering flight evaluation sufficient to certify the two-segment system for the six-month in-service evaluation; (2) evaluate the suitability of a modified RNAV system for flying two-segment approaches; and (3) provide evaluation of the two-segment approach by management and line pilots.

Morrison, J. A.↗

NASA NTRS1986

NASA PC software evaluation project

The USL NASA PC software evaluation project is intended to provide a structured framework for facilitating the development of quality NASA PC software products. The project will assist NASA PC development staff to understand the characteristics and functions of NASA PC software products. Based on the results of the project teams' evaluations and recommendations, users can judge the reliability, usability, acceptability, maintainability and customizability of all the PC software products. The objective here is to provide initial, high-level specifications and guidelines for NASA PC software evaluation. The primary tasks to be addressed in this project are as follows: to gain a strong understanding of what software evaluation entails and how to organize a structured software evaluation process; to define a structured methodology for conducting the software evaluation process; to develop a set of PC software evaluation criteria and evaluation rating scales; and to conduct PC software evaluations in accordance with the identified methodology. Communication Packages, Network System Software, Graphics Support Software, Environment Management Software, General Utilities. This report represents one of the 72 attachment reports to the University of Southwestern Louisiana's Final Report on NASA Grant NGT-19-010-900. Accordingly, appropriate care should be taken in using this report out of context of the full Final Report.

Dominick, Wayne D.↗

NASA NTRS2012

Towards Reliable Evaluation of Anomaly-Based Intrusion Detection Performance

This report describes the results of research into the effects of environment-induced noise on the evaluation process for anomaly detectors in the cyber security domain. This research was conducted during a 10-week summer internship program from the 19th of August, 2012 to the 23rd of August, 2012 at the Jet Propulsion Laboratory in Pasadena, California. The research performed lies within the larger context of the Los Angeles Department of Water and Power (LADWP) Smart Grid cyber security project, a Department of Energy (DoE) funded effort involving the Jet Propulsion Laboratory, California Institute of Technology and the University of Southern California/ Information Sciences Institute. The results of the present effort constitute an important contribution towards building more rigorous evaluation paradigms for anomaly-based intrusion detectors in complex cyber physical systems such as the Smart Grid. Anomaly detection is a key strategy for cyber intrusion detection and operates by identifying deviations from profiles of nominal behavior and are thus conceptually appealing for detecting "novel" attacks. Evaluating the performance of such a detector requires assessing: (a) how well it captures the model of nominal behavior, and (b) how well it detects attacks (deviations from normality). Current evaluation methods produce results that give insufficient insight into the operation of a detector, inevitably resulting in a significantly poor characterization of a detectors performance. In this work, we first describe a preliminary taxonomy of key evaluation constructs that are necessary for establishing rigor in the evaluation regime of an anomaly detector. We then focus on clarifying the impact of the operational environment on the manifestation of attacks in monitored data. We show how dynamic and evolving environments can introduce high variability into the data stream perturbing detector performance. Prior research has focused on understanding the impact of this variability in training data for anomaly detectors, but has ignored variability in the attack signal that will necessarily affect the evaluation results for such detectors. We posit that current evaluation strategies implicitly assume that attacks always manifest in a stable manner; we show that this assumption is wrong. We describe a simple experiment to demonstrate the effects of environmental noise on the manifestation of attacks in data and introduce the notion of attack manifestation stability. Finally, we argue that conclusions about detector performance will be unreliable and incomplete if the stability of attack manifestation is not accounted for in the evaluation strategy.

cyber defense↗

DOE OSTI2025

Re-evaluating the prompt fission neutron spectrum of spontaneously fissioning 252 Cf

The prompt fission neutron spectrum (PFNS) of spontaneously fissioning 252 Cf is a Neutron Data Standards observable. Nearly all fission spectra of actinides were measured relative to it, using efficiencies derived from it, or analyzed with simulations validated by it. The current Standards evaluation was published by W. Mannhart in 1987. It could not be updated because the evaluation input, experimental mean values and covariances, were lost. First, we attempt to reproduce it. However, Mannhart’s evaluation can only be reproduced within its one-σ uncertainties as some of its aspects (e.g., experimental covariances, rejected data points) remain unknown. Therefore, a new evaluation is presented: We revisit all existing experimental 252 Cf(sf) PFNS data, including those published after the release of the current Standards evaluation, and re-estimate associated covariances. The newly evaluated 252 Cf(sf) PFNS differs distinctly from Mannhart’s below 300 keV and extends it to lower and higher outgoing neutron energies (500 eV–25 MeV). The new evaluated uncertainties are larger from 3–9 MeV and smaller otherwise. Spectrum averaged cross sections of importance to the International Reactor Dosimetry and Fusion File community calculated with the new spectrum are close to those calculated with Mannhart’s evaluation and agree with experimental values well within their uncertainties.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2026

ICSBEP evaluation lessons learned and good practice

This document describes lessons learned during my nearly 20 years of experience in International Criticality Safety Benchmark Evaluation Project (ICSBEP) meetings and in the development of several evaluations employing the IPEN/MB-01 reactor. The evaluations cover a wide range of possible configurations at the reactor core of this facility. Most of the evaluations are related to critical configurations, although two of them concern subcritical measurements. Examples of the evaluations submitted and approved for ICSBEP publication are the critical configurations employed in the standard core; critical configurations employing heavy reflectors composed of stainless steel, nickel, and carbon steel; and critical configurations employing the standard fuel rods and fuel rods composed of UO 2 -Gd 2 O 3 , among several others. Additionally, this document addresses good practices for the development of ICSBEP evaluations; the intention is to help beginners who are going to start evaluations for ICSBEP. The sections of the ICSBEP evaluation are described and illustrated with several examples.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

NASA NTRS1993

Postflight hardware evaluation 360T026 (RSRM-26, STS-47)

The final report for the Clearfield disassembly evaluation and a continuation of the KSC postflight assessment for the 360T026 (STS-47) Redesigned Solid Rocket Motor (RSRM) flight set is provided. All observed hardware conditions were documented on PFOR's and are included in Appendices A, B, and C. Appendices D and E contain the measurements and safety factor data for the nozzle and insulation components. This report, along with the KSC Ten-Day Postflight Hardware Evaluation Report (TWR-64203), represents a summary of the 360T026 hardware evaluation. The as-flown hardware configuration is documented in TWR-60472. Disassembly evaluation photograph numbers are logged in TWA-1987. The 360T026 flight set disassembly evaluations described were performed at the RSRM Refurbishment Facility in Clearfield, Utah. The final factory joint demate occurred on 12 April 1993. Detailed evaluations were performed in accordance with the Clearfield Postflight Engineering Evaluation Plan (PEEP), TWR-50051, Revision A. All observations were compared against limits that are also defined in the PEEP. These limits outline the criteria for categorizing the observations as acceptable, reportable, or critical. Hardware conditions that were unexpected and/or determined to be reportable or critical were evaluated by the applicable CPT and tracked through the PFAR system.

Nielson, Greg↗

NASA NTRS1993

Final postflight hardware evaluation report RSRM-28 (STS-53)

The final report for the Clearfield disassembly evaluation and a continuation of the KSC postflight assessment for the RSRM-28 (STS-53) RSRM flight set is presented. All observed hardware conditions were documented on PFOR's and are included in Appendices A through C. Appendices D and E contain the measurements and safety factor data for the nozzle and insulation components. This report, along with the KSC Ten-Day Postflight Hardware Evaluation Report (TWR-64215), represents a summary of the RSRM-28 hardware evaluation. The as-flown hardware configuration is documented in TWR-63638. Disassembly evaluation photograph numbers are logged in TWA-1989. The RSRM-28 flight set disassembly evaluations described were performed at the RSRM Refurbishment Facility in Clearfield, Utah. The final factory joint demate occurred on July 15, 1993. Additional time was required to perform the evaluation of the stiffener rings per special issue 4.1.5.2 because of the washout schedule. The release of this report was after completion of all special issues per program management direction. Detailed evaluations were performed in accordance with the Clearfield PEEP, TWR-50051, Revision A. All observations were compared against limits that are also defined in the PEEP. These limits outline the criteria for categorizing the observations as acceptable, reportable, or critical. Hardware conditions that were unexpected and/or determined to be reportable or critical were evaluated by the applicable team and tracked through the PFAR system.

Starrett, William David, Jr.↗

NASA NTRS1994

Rule based design of conceptual models for formative evaluation

A Human-Computer Interface (HCI) Prototyping Environment with embedded evaluation capability has been investigated. This environment will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. This environment, which allows for rapid prototyping and evaluation of graphical interfaces, includes the following four components: (1) a HCI development tool, (2) a low fidelity simulator development tool, (3) a dynamic, interactive interface between the HCI and the simulator, and (4) an embedded evaluator that evaluates the adequacy of a HCI based on a user's performance. The embedded evaluation tool collects data while the user is interacting with the system and evaluates the adequacy of an interface based on a user's performance. This paper describes the design of conceptual models for the embedded evaluation system using a rule-based approach.

Moore, Loretta A.↗

NASA NTRS1994

Rule based design of conceptual models for formative evaluation

A Human-Computer Interface (HCI) Prototyping Environment with embedded evaluation capability has been investigated. This environment will be valuable in developing and refining HCI standards and evaluating program/project interface development, especially Space Station Freedom on-board displays for payload operations. This environment, which allows for rapid prototyping and evaluation of graphical interfaces, includes the following four components: (1) a HCI development tool; (2) a low fidelity simulator development tool; (3) a dynamic, interactive interface between the HCI and the simulator; and (4) an embedded evaluator that evaluates the adequacy of a HCI based on a user's performance. The embedded evaluation tool collects data while the user is interacting with the system and evaluates the adequacy of an interface based on a user's performance. This paper describes the design of conceptual models for the embedded evaluation system using a rule-based approach.

Moore, Loretta A.↗

NASA NTRS1999

Microgravity Workstation and Restraint Evaluations

Confined workstations, where the operator has limited visibility and physical access to the work area, may cause prolonged periods of unnatural posture. Impacts on performance, in terms of fatigue and posture, may occur especially if the task is tedious and repetitive or requires static muscle loading. The glovebox design is a good example of the confined workstation concept. Within the scope of the 'Microgravity Workstation and Restraint Evaluation' project, funded by the NASA Headquarters Life Sciences Division, it was proposed to conduct a series of evaluations in ground, KC-135 and Shuttle environments to investigate the human factors issues concerning confined/unique workstations, such as gloveboxes, and also including crew restraint requirements. As part of the proposed integrated evaluations, two Shuttle Detailed Supplementary Objectives (DSOs) were manifested; one on Space Transportation System (STS)-90 and one on STS-88. The DSO on STS-90 evaluated use of the General Purpose Workstation (GPWS). The STS-88 mission was planned to evaluate a restraint system at the Remote Manipulator System (RMS). In addition, KC- 1 35 flights were conducted to investigate user/workstation/restraint integration for long-duration microgravity use. The scope of these evaluations included workstations and restraints to be utilized in the ISS environment, but also incorporated other workstations/ restraints in an attempt to provide findings/requirements with broader applications across multiple programs (e.g., Shuttle, ISS, and future Lunar-Mars programs). In addition, a comprehensive electronic questionnaire has been prepared and is under review by the Astronaut Office which will compile crewmembers' lessons learned information concerning glovebox and restraint use following their missions. These evaluations were intended to be complementary and were coordinated with hardware developers, users (crewmembers), and researchers. This report is intended to provide a summary of the findings from each of the evaluations.

Chmielewski, C.↗

NASA NTRS1998

Objective Situation Awareness Measurement Based on Performance Self-Evaluation

The research was conducted in support of the NASA Safe All-Weather Flight Operations for Rotorcraft (SAFOR) program. The purpose of the work was to investigate the utility of two measurement tools developed by the British Defense Evaluation Research Agency. These tools were a subjective workload assessment scale, the DRA Workload Scale and a situation awareness measurement tool. The situation awareness tool uses a comparison of the crew's self-evaluation of performance against actual performance in order to determine what information the crew attended to during the performance. These two measurement tools were evaluated in the context of a test of innovative approach to alerting the crew by way of a helmet mounted display. The situation assessment data are reported here. The performance self-evaluation metric of situation awareness was found to be highly effective. It was used to evaluate situation awareness on a tank reconnaissance task, a tactical navigation task, and a stylized task used to evaluated handling qualities. Using the self-evaluation metric, it was possible to evaluate situation awareness, without exact knowledge the relevant information in some cases and to identify information to which the crew attended or failed to attend in others.

DeMaio, Joe↗

NASA NTRS2004

Efficient Evaluation Functions for Multi-Rover Systems

Evolutionary computation can be a powerful tool in cresting a control policy for a single agent receiving local continuous input. This paper extends single-agent evolutionary computation to multi-agent systems, where a collection of agents strives to maximize a global fitness evaluation function that rates the performance of the entire system. This problem is solved in a distributed manner, where each agent evolves its own population of neural networks that are used as the control policies for the agent. Each agent evolves its population using its own agent-specific fitness evaluation function. We propose to create these agent-specific evaluation functions using the theory of collectives to avoid the coordination problem where each agent evolves a population that maximizes its own fitness function, yet the system has a whole achieves low values of the global fitness function. Instead we will ensure that each fitness evaluation function is both "aligned" with the global evaluation function and is "learnable," i.e., the agents can readily see how their behavior affects their evaluation function. We then show how these agent-specific evaluation functions outperform global evaluation methods by up to 600% in a domain where a set of rovers attempt to maximize the amount of information observed while navigating through a simulated environment.

Agogino, Adrian↗

NASA NTRS2010

Risk Evaluation in the Pre-Phase A Conceptual Design of Spacecraft

Typically, the most important decisions in the design of a spacecraft are made in the earliest stages of its conceptual design the Pre-Phase A stages. It is in these stages that the greatest number of design alternatives is considered, and the greatest number of alternatives is rejected. The focus of Pre-Phase A conceptual development is on the evaluation and comparison of whole concepts and the larger-scale systems comprising those concepts. This comparison typically uses general Figures of Merit (FOMs) to quantify the comparative benefits of designs and alternative design features. Along with mass, performance, and cost, risk should be one of the major FOMs in evaluating design decisions during the conceptual design phases. However, risk is often given inadequate consideration in conceptual design practice. The reasons frequently given for this lack of attention to risk include: inadequate mission definition, lack of rigorous design requirements in early concept phases, lack of fidelity in risk assessment methods, and under-evaluation of risk as a viable FOM for design evaluation. In this paper, the role of risk evaluation in early conceptual design is discussed. The various requirements of a viable risk evaluation tool at the Pre-Phase A level are considered in light of the needs of a typical spacecraft design study. A technique for risk identification and evaluation is presented. The application of the risk identification and evaluation approach to the conceptual design process is discussed. Finally, a computational tool for risk profiling is presented and applied to assess the risk for an existing Pre-Phase A proposal. The resulting profile is compared to the risks identified for the proposal by other means.

Fabisinski, Leo L., III↗

NASA NTRS2022

Evaluation of remote sensing-based evapotranspiration products at low-latitude eddy covariance sites

Remote sensing-based evapotranspiration (ET) products have been evaluated primarily using data from northern middle latitudes; therefore, little is known about their performance at low latitudes. To address this bias, an evaluation dataset was compiled using eddy covariance data from 40 sites between latitudes 30° S and 30° N. The flux data were obtained from the emerging network in Mexico (MexFlux) and from openly available databases of FLUXNET, AsiaFlux, and OzFlux. This unique reference dataset was then used to evaluate remote sensing-based ET products in environments that have been underrepresented in earlier studies. The evaluated products were: MODIS ET (MOD16, both the discontinued collection 5 (C5) and the latest collection (C6)), Global Land Evaporation Amsterdam Model (GLEAM) ET, and Atmosphere-Land Exchange Inverse (ALEXI) ET. Products were compared with unadjusted fluxes (ETorig) and with fluxes corrected for the lack of energy balance closure (ETebc). Three common statistical metrics were used: coefficient of determination (R2), root mean square error (RMSE), and percent bias (PBIAS). The effect of a vegetation mismatch between pixel and site on product evaluation results was investigated by examining the relationship between the statistical metrics and product-specific vegetation match indexes. Evaluation results of this study and those published in the literature were used to examine the performance of the products across latitudes. Differences between the MOD16 collection 5 and 6 datasets were generally smaller than differences with the other products. Performance and ranking of the evaluated products depended on whether ETorig or ETebc was used. When using ETorig, GLEAM generally had the highest R2, smallest PBIAS, and best RMSE values across the studied land cover types and climate zones. Neither MOD16 nor ALEXI performed consistently better than the other. When using ETebc, none of the products stood out in terms of both low bias and strong correlations. The use of ETebc instead of ETorig affected the biases more than the correlations. The product evaluation results showed no significant relationship with the degree of match between the vegetation at the pixel and site scale. The latitudinal comparison showed tendencies of lower R2 (all products) but better PBIAS and normalized RMSE values (MOD16 and GLEAM) for forests at low latitudes than for forests at northern middle latitudes. For non-forest vegetation, the products showed no clear latitudinal differences in performance.

Diego Salazar-Martínez↗

NASA NTRSDate not supplied

Evaluation of the NASA Artemis Regions of Interest for ISRU Water Mine Potential

The NASA Artemis Campaign has a stated goal to return to the Moon to maintain a sustainable presence; In-Situ Resource Utilization (ISRU) is a key part of sustainability. The regions of interest identified for the Artemis campaign are at Lunar the South Pole where water ice, a valuable resource for ISRU, has been identified. As such, a preliminary evaluation of the ISRU ice mining potential has been performed for of these regions of interest. A set of ground rules for this evaluation were developed to align with current assumptions for customer needs, hardware capabilities, an initially limited infrastructure, and lunar environments/terrain. These ground rules, and the evaluation of six regions of interest, are presented here. The site selections (ISRU and customer assets) and their associated traverses are notional and were intended only to provide a broad preliminary evaluation of the water ISRU potential of the regions. Evaluation of these regions are subject to change as decisions regarding utilization are made. Water ISRU is possible at all regions, though the degree to which each criterion are met is variable. The two regions near Shackleton ranked highest in this evaluation, while the de Gerlache region presented the most difficulties meeting the current criteria. The regions were not explicitly ranked due to the nuances associated with the high number of variables but evaluation summaries of each are presented. It should also be noted that all ISRU ‘mine’ sites in this analysis focused on smaller (few kilometer) size permanently shadowed regions (PSRs). This was necessary to meet proximity requirements between these PSRs and the highly illuminated regions needed for customers and ISRU processing. The areas identified in this study are meant to focus exploration and reconnaissance efforts needed to better evaluate the ISRU potential.

In-situ resource utilization↗

NASA NTRSDate not supplied

Evaluation of the NASA Artemis Regions of Interest for ISRU Water Mine Potential

In-situ resource utilization↗

DOE OSTI2025

Evaluation of HVAC & refrigeration system fault behaviors and impacts: A systematic review

Achieving the goals of green buildings critically depends on the fault-free operation of heating, ventilation and air conditioning and refrigeration (HVAC&R) systems. However, faults frequently occur in these systems, causing a range of negative consequences, including increased energy consumption, diminished operational performance, compromised indoor environmental quality, higher operational costs, and shortened system lifespan. The evaluation of fault behaviors and impacts plays a critical role in revealing fault characteristics and consequently supports many research areas, including the design of the high-performance equipment, development of fault detection and diagnostics (FDD) and robust control approaches, as well as the enhancement of maintenance decision-making activities. This paper systematically reviews 112 research publications that reported the analysis and evaluation of fault behaviors and impacts in HVAC&R systems over the past thirty years. Here, we designed a review approach to address five crucial research questions, namely: 1) the objectives of analysis and evaluation of fault behaviors and impacts, 2) data sources, 3) equipment/system types and fault types, 4) evaluation methods including evaluation measures and associated metrics, and 5) challenges and future directions in the research on evaluating of fault behaviors and impacts. In-depth discussions on these questions help bridge the gap between the evaluation of fault behaviors and impacts and their practical applications, such as the development of high-performance systems, fault models, FDD methods, and maintenance decision-making tools within the HVAC&R FDD domain.

Chen, Yimin [Oak Ridge National Laboratory (ORNL),↗