SEARCH · Search NASA

Results for “evaluation methodology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Improving the Advancement of Women in Computer and Computational Science Research with the CRA-W Career Mentoring Workshops (Final Report)

The mission of the Computing Research Association’s Committee on Widening Participation in Computing Research (CRA-WP) is to widen the participation and improve the access, opportunities, and positive experiences of individuals from groups underrepresented in computing research and education. CRA-WP programs serve this overarching goal at all career stages; in addition, CRA-WP, through the formation of the CRA Center for Evaluating the Research Pipeline (CERP), has developed a methodology for thoroughly evaluating the success of its programs by comparing a nationwide sample of students, researchers, and faculty (non-participants) to program participants. Achieving these objectives requires that an increasing number of individuals from populations underrepresented in computing start and progress to the next stage while understanding and supporting the myriad computing pathways. CRA-WP offers programs for participants from undergraduate to senior professional levels. Different career stages need different types of interventions, and the goal of all Alliance program activities can be described within the unifying framework of Social Cognitive Career Theory, which finds that interest in and choice of a particular career path will be increased by interventions that improve one or more of the following: (1) outcome expectations (understanding and valuing the rewards of a particular outcome), (2) self-efficacy (a belief that one can successfully achieve an outcome), and (3) social supports that help one persist and overcome obstacles.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2004

COTS Ceramic Chip Capacitors: An Evaluation of the Parts and Assurance Methodologies

Commercial-Off-The-Shelf (COTS) multilayer ceramic chip capacitors (MLCCs) are continually evolving to reduce physical size and increase volumetric efficiency. Designers of high reliability aerospace and military systems are attracted to these attributes of COTS MLCCs and would like to take advantage of them while maintaining the high standards for long-term reliable operation they are accustomed io when selecting military qualified established reliability (MIL-ER) MLCCs. However, MIL-ER MLCCs are not available in the full range of small chip sizes with high capacitance as found in today's COTS MLCCs. The objectives for this evaluation were to assess the long-term performance of small case size COTS MLCCs and to identify effective, lower-cost product assurance methodologies. Fifteen (15) lots of COTS X7R dielectric MLCCs from four (4) different manufacturers and two (2) MIL-ER BX dielectric MLCCs from two (2) of the same manufacturers were evaluated. Both 0805 and 0402 chip sizes were included. Several voltage ratings were tested ranging from a high of 50 volts to a low of 6.3 volts. The evaluation consisted of a comprehensive screening and qualification test program based upon MIL-PRF-55681 (i.e., voltage conditioning, thermal shock, moisture resistance, 2000-hour life test, etc.). In addition, several lot characterization tests were performed including Destructive Physical Analysis (DPA), Highly Accelerated Life Test (HALT) and Dielectric Voltage Breakdown Strength. The data analysis included a comparison of the 2000-hour life test results (used as a metric for long-term performance) relative to the screening and characterization test results. Results of this analysis indicate that the long-term life performance of COTS MLCCs is variable -- some lots perform well, some lots perform poorly. DPA and HALT were found to be promising lot characterization tests to identify substandard COTS MLCC lots prior to conducting more expensive screening and qualification tests. The results indicate that lot- specific screening and qualification are still recommended for high reliability applications. One significant and concerning observation is that MIL- type voltage conditioning (100 hours at twice rated voltage, 125 C) was not an effective screen in removing infant mortality parts for the particular lots of COTS MLCCs evaluated.

Brusse, Jay A.↗

NASA NTRS1995

Infrared algorithm development for ocean observations

Efforts continue under this contract to develop algorithms for the computation of sea surface temperature (SST) from MODIS infrared retrievals. This effort includes radiative transfer modeling, comparison of in situ and satellite observations, development and evaluation of processing and networking methodologies for algorithm computation and data accession, evaluation of surface validation approaches for IR radiances, and participation in MODIS (project) related activities. Efforts in this contract period have focused on radiative transfer modeling, evaluation of atmospheric correction methodologies, involvement in field studies, production and evaluation of new computer networking strategies, and objective analysis approaches.

Brown, Otis B.↗

DOE OSTI2021

Evaluate the impact of sensor accuracy on model performance in data-driven building fault detection and diagnostics using Monte Carlo simulation

The performance of data-driven fault detection and diagnostics (FDD) is heavily dependent on sensors. However, sensor inaccuracy and sensor faults are pervasive in building operation: inaccurate and missing sensor readings deteriorate FDD performance; sensor inaccuracy will also affect the selection of sensor for data-driven FDD in the model training process, which is another key factor of data-driven FDD performance. Sensor accuracy and sensor selection individually are well-studied research topics in this field, but the impact of sensor accuracy on sensor selection and its further impact on FDD performance has not been evaluated and quantified. In this paper, we developed a novel analysis methodology that comprehensively evaluates sensor fault on sensor selection and FDD accuracy. Monte Carlo simulation is applied to deal with multiple stochastic sensor inaccuracy and provide probabilistic analysis results of the impact of sensor inaccuracy on sensor selection and FDD accuracy. This methodology focuses on the net impact of fault states across a full sensor set. The developed methodology can be used for the early-stage sensor design and operation-stage sensor maintenance. Furthermore, a case study is conducted to demonstrate the analysis methodology using a commercial building model crated to Flexible Research Platform located at Oak Ridge National Laboratory, USA.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NASA NTRS1997

Knowledge-Based Aircraft Automation: Managers Guide on the use of Artificial Intelligence for Aircraft Automation and Verification and Validation Approach for a Neural-Based Flight Controller

The ultimate goal of this report was to integrate the powerful tools of artificial intelligence into the traditional process of software development. To maintain the US aerospace competitive advantage, traditional aerospace and software engineers need to more easily incorporate the technology of artificial intelligence into the advanced aerospace systems being designed today. The future goal was to transition artificial intelligence from an emerging technology to a standard technology that is considered early in the life cycle process to develop state-of-the-art aircraft automation systems. This report addressed the future goal in two ways. First, it provided a matrix that identified typical aircraft automation applications conducive to various artificial intelligence methods. The purpose of this matrix was to provide top-level guidance to managers contemplating the possible use of artificial intelligence in the development of aircraft automation. Second, the report provided a methodology to formally evaluate neural networks as part of the traditional process of software development. The matrix was developed by organizing the discipline of artificial intelligence into the following six methods: logical, object representation-based, distributed, uncertainty management, temporal and neurocomputing. Next, a study of existing aircraft automation applications that have been conducive to artificial intelligence implementation resulted in the following five categories: pilot-vehicle interface, system status and diagnosis, situation assessment, automatic flight planning, and aircraft flight control. The resulting matrix provided management guidance to understand artificial intelligence as it applied to aircraft automation. The approach taken to develop a methodology to formally evaluate neural networks as part of the software engineering life cycle was to start with the existing software quality assurance standards and to change these standards to include neural network development. The changes were to include evaluation tools that can be applied to neural networks at each phase of the software engineering life cycle. The result was a formal evaluation approach to increase the product quality of systems that use neural networks for their implementation.

Broderick, Ron↗

DOE OSTI2023

Exploring Microsimulation Process for Energy Impact Evaluation of Connected and Automated Vehicles

In this paper, the authors present a microsimulation-based methodological approach for evaluating the energy impact of connected and automated vehicles (CAVs). They use an open-source micro-simulator, SUMO, and provide a way to set up a simulation environment that emulates real-world traffic dynamics. They also employ the Intelligent Driver Model to represent human drivers and calibrate its driving behavior using real-world traffic data and driving statistics. The authors conduct extensive simulation studies considering different penetration rates of CAVs, different car-following models, and varying car-following model parameters. Using the state-of-the-art Future Automotive System Technology Simulator (FASTSim), they estimate the fuel economy of each vehicle and analyze the energy impact of the given CAV implementation. Finally, the authors analyze the possible factors affecting the simulation results, and also discuss limitations and future work.

ADVANCED PROPULSION SYSTEMS↗

NASA NTRS1992

An improved approach for flight readiness certification: Methodology for failure risk assessment and application examples, volume 1

An improved methodology for quantitatively evaluating failure risk of spaceflight systems to assess flight readiness and identify risk control measures is presented. This methodology, called Probabilistic Failure Assessment (PFA), combines operating experience from tests and flights with engineering analysis to estimate failure risk. The PFA methodology is of particular value when information on which to base an assessment of failure risk, including test experience and knowledge of parameters used in engineering analyses of failure phenomena, is expensive or difficult to acquire. The PFA methodology is a prescribed statistical structure in which engineering analysis models that characterize failure phenomena are used conjointly with uncertainties about analysis parameters and/or modeling accuracy to estimate failure probability distributions for specific failure modes. These distributions can then be modified, by means of statistical procedures of the PFA methodology, to reflect any test or flight experience. Conventional engineering analysis models currently employed for design of failure prediction are used in this methodology. The PFA methodology is described and examples of its application are presented. Conventional approaches to failure risk evaluation for spaceflight systems are discussed, and the rationale for the approach taken in the PFA methodology is presented. The statistical methods, engineering models, and computer software used in fatigue failure mode applications are thoroughly documented.

Moore, N. R.↗

NASA NTRS1992

An improved approach for flight readiness certification: Methodology for failure risk assessment and application examples. Volume 2: Software documentation

An improved methodology for quantitatively evaluating failure risk of spaceflight systems to assess flight readiness and identify risk control measures is presented. This methodology, called Probabilistic Failure Assessment (PFA), combines operating experience from tests and flights with engineering analysis to estimate failure risk. The PFA methodology is of particular value when information on which to base an assessment of failure risk, including test experience and knowledge of parameters used in engineering analyses of failure phenomena, is expensive or difficult to acquire. The PFA methodology is a prescribed statistical structure in which engineering analysis models that characterize failure phenomena are used conjointly with uncertainties about analysis parameters and/or modeling accuracy to estimate failure probability distributions for specific failure modes, These distributions can then be modified, by means of statistical procedures of the PFA methodology, to reflect any test or flight experience. Conventional engineering analysis models currently employed for design of failure prediction are used in this methodology. The PFA methodology is described and examples of its application are presented. Conventional approaches to failure risk evaluation for spaceflight systems are discussed, and the rationale for the approach taken in the PFA methodology is presented. The statistical methods, engineering models, and computer software used in fatigue failure mode applications are thoroughly documented.

Moore, N. R.↗

NASA NTRS1992

An improved approach for flight readiness certification: Methodology for failure risk assessment and application examples. Volume 3: Structure and listing of programs

Moore, N. R.↗

DOE OSTI2023

Technical Specification Surveillance Interval Extension Using Self-Diagnostics

As part of the Light Water Reactor Sustainability program, an ongoing research effort is being conducted on technical specifications surveillance interval extension of digital equipment in nuclear power plants. The research team is led by Idaho National Laboratory and includes Pacific Northwest National Laboratory, Technology Resources, and Oak Ridge National Laboratory. This research focuses on developing methods for applying the U.S. Nuclear Regulatory Commission (NRC)–approved guidance to implement a licensee-controlled, risk-informed surveillance frequency change program in digital instrumentation and control (I&C) systems that include self-diagnostics and online monitoring (OLM) capabilities. Although approved methods exist for extending technical specifications (TS) surveillance test intervals (STIs) for general equipment, including analog I&C equipment, gaps remain in technology and guidance on crediting newer digital equipment’s internal self-diagnostics and OLM characteristics. Previous research described a general methodology for crediting internal self-diagnostics for extending surveillance test intervals. The methodology used self-diagnostics to detect—and credited recovery from—failure. Self-diagnostics were also applied for performance monitoring during the extended surveillance interval. This report discusses the status of recent activities to evaluate the previously developed methodology using a pilot study. Although both a utility partner for a pilot study and a specific digital asset were identified in FY2020, delays in obtaining proprietary information resulted in a limited ability to fully evaluate the methodology, and further interactions were complicated by the COVID pandemic. Therefore, at that time, the use of public-domain information—along with current processes for surveillance interval extension through a surveillance frequency control program—identified the need to fully assess diagnostic coverage as part of the pilot study. Furthermore, self-diagnostics were also identified as a potential option to replace the drift analyses conducted as part of current STI extension procedures. In FY2022, the project was reconstituted with the industry partner, and information and data were made available by the industry partner to the research team for review. The shared information included failure event descriptions and data for a digital I&C system since its implementation, as well as recent STI extension interval reports developed by the utility partner on that digital I&C system. This report presents an evaluation of this information and data and describes an application of the proposed methodology cited above. The methodology seeks to take advantage of the self-diagnostics and OLM capabilities to reduce risk or reduce the level of qualitative monitoring assessment needed to perform a risk-informed STI extension using existing NRC approved guidance or both. Addressing these issues of STI extension by crediting self-diagnostics is likely to result in benefits for current and future nuclear power plant (NPP) operations, including lowering the barriers to adoption of digital I&C systems and increasing cost savings by deferring or eliminating unneeded preventive maintenance (tasks or checks or activities). Specifically, self-diagnostic and OLM capabilities of newer digital equipment being installed in non-safety and safety applications are designed to detect failures, provide early warning of potential failures, and notify plant operators to take appropriate action to reduce out of service (OOS) time thus protecting safety margins. Moreover, the equipment is expected to provide information that time-related operational degradation is identified early to ensure timely and planned corrective actions instead of a reactive and unplanned approach ahead of an extended-surveillance interval.

42 ENGINEERING↗

NASA NTRS1984

Flight test technique for evaluation of gust load alleviation analysis methodology

A technique for gust load alleviation (GLA) flight testing that will approximate a turbulence-like excitation of the wing has been devised. An artificial excitation is produced by randomly deflecting inboard control surfaces on the wing, thereby producing incremental loads on the wing. This presentation covers the background and development of the flight test technique and analyses performed to date.

Perry, B., III↗

NASA NTRS1989

Performance evaluation in full-mission simulation - Methodological advances and research challenges

The crew-factors research program at NASA Ames has developed a methodology for studying the impact of a variety of variables on the effectiveness of crews flying realistic but high workload simulated trips. The validity of investigations using the methodology is enhanced by careful design of full-mission scenarios, performance assessment using converging sources of data, and recruitment of representative subjects. Recently, portions of this methodology have been adapted for use in assessing the effectiveness of crew coordination among participants in line-oriented flight training.

Chidester, Thomas R.↗

NASA NTRS1992

6th AIAA Biennial Flight Test Conference, Hilton Head Island, SC, Aug. 24-26, 1992, Selected Papers

Topics presented include training for avionics evaluation, flight testing in the '90s, methodology for the critical evaluation of flight test data, flight testing of an electric powered vehicle, and the flying qualities of a remotely piloted vehicle. Also presented are airspeed calibration using GPS, the design and conduct of a windshear detection flight experiment, the 757 fly-by-wire demonstrator flight test, and an update of the B-52 flight test program.

American Institute of Aeronautics and Astronautics↗

NASA NTRS1997

Evaluation of a Progressive Failure Analysis Methodology for Laminated Composite Structures

A progressive failure analysis methodology has been developed for predicting the nonlinear response and failure of laminated composite structures. The progressive failure analysis uses C plate and shell elements based on classical lamination theory to calculate the in-plane stresses. Several failure criteria, including the maximum strain criterion, Hashin's criterion, and Christensen's criterion, are used to predict the failure mechanisms. The progressive failure analysis model is implemented into a general purpose finite element code and can predict the damage and response of laminated composite structures from initial loading to final failure.

Sleight, David W.↗

NASA NTRS2018

The Evaluation of Cryogenic Force Balance Calibration Methodologies with Respect to Wind Tunnel Results

Independent tests of the NASA Common Research Model (CRM) at NASA's National Transonic Facility (NTF) and the European Transonic Windtunnel (ETW) revealed discrepancies at low operating temperatures and high Reynolds numbers that warranted further investigation. Since each facility used their own force balance for their tunnel entry, one suggestion for the discrepancy was the temperature compensation methodology developed and applied for each balance. This hypothesis is explored through simulation and experimentally. Independent calibrations of NASA's NTF-118A balance at NASA Langley and ETW reveal discrepancies in the thermal compensation of the normal force and pitching moment primary sensitivities with temperature, while the axial force primary sensitivities are in good agreement. The application of the force balance calibrations performed at NASA and ETW to the prior wind tunnel data suggests that the thermal compensation discrepancies are an order of magnitude less than the discrepancies observed between the wind tunnel aerodynamic coefficients.

Burns, Devin E.↗

NASA NTRS2020

Advancing Methodologies for Applying Machine Learning and Evaluating Spatiotemporal Models of Fine Particulate Matter (PM 2.5 ) Using Satellite Data Over Large Regions

Reconstructing the distribution of fine particulate matter (PM 2.5 ) in space and time, even far from ground monitoring sites, is an important exposure science contribution to epidemiologic analyses of PM 2.5 health impacts. Flexible statistical methods for prediction have demonstrated the integration of satellite observations with other predictors, yet these algorithms are susceptible to overfitting the spatiotemporal structure of the training datasets. We present a new approach for predicting PM 2.5 using machine-learning methods and evaluating prediction models for the goal of making predictions where they were not previously available. We apply extreme gradient boosting (XGBoost) modeling to predict daily PM 2.5 on a 1 x 1 km 2 resolution for a 13 state region in the Northeastern USA for the years 2000–2015 using satellite-derived aerosol optical depth and implement a recursive feature selection to develop a parsimonious model. We demonstrate excellent predictions of withheld observations but also contrast an RMSE of 3.11 μg/m 3 in our spatial cross-validation withholding nearby sites versus an overfit RMSE of 2.10 μg/m 3 using a more conventional random ten-fold splitting of the dataset. As the field of exposure science moves forward with the use of advanced machine-learning approaches for spatiotemporal modeling of air pollutants, our results show the importance of addressing data leakage in training, overfitting to spatiotemporal structure, and the impact of the predominance of ground monitoring sites in dense urban sub-networks on model evaluation. The strengths of our resultant modeling approach for exposure in epidemiologic studies of PM 2.5 include improved efficiency, parsimony, and interpretability with robust validation while still accommodating complex spatiotemporal relationships.

air pollution↗

NASA NTRS1986

NASA PC software evaluation project

The USL NASA PC software evaluation project is intended to provide a structured framework for facilitating the development of quality NASA PC software products. The project will assist NASA PC development staff to understand the characteristics and functions of NASA PC software products. Based on the results of the project teams' evaluations and recommendations, users can judge the reliability, usability, acceptability, maintainability and customizability of all the PC software products. The objective here is to provide initial, high-level specifications and guidelines for NASA PC software evaluation. The primary tasks to be addressed in this project are as follows: to gain a strong understanding of what software evaluation entails and how to organize a structured software evaluation process; to define a structured methodology for conducting the software evaluation process; to develop a set of PC software evaluation criteria and evaluation rating scales; and to conduct PC software evaluations in accordance with the identified methodology. Communication Packages, Network System Software, Graphics Support Software, Environment Management Software, General Utilities. This report represents one of the 72 attachment reports to the University of Southwestern Louisiana's Final Report on NASA Grant NGT-19-010-900. Accordingly, appropriate care should be taken in using this report out of context of the full Final Report.

Dominick, Wayne D.↗

DOE OSTI2022

Evaluation of Heat Exchanger Technology for Integrated Energy System: Methodology and Preliminary Application

This report proposes a novel heat exchanger evaluation method, based on integrated QFD-AHP technique, to support strategic optimal selection of heat exchanger (HX) technology for integrated energy systems. The term “optimal selection” is defined as selecting a product (i.e., HX) that best meets various aspects of end user demand, including both technical and economic requirements. Compared to previous HX evaluation methods, the proposed approach allows decision makers to explicitly take into account the practical demand of customers (i.e., HX end users) which may vary depending on the HX application target. In addition, the proposed evaluation method can be used not only for high-level decisions to select HX type, but also for low-level decisions such as the selection of the key components including HX material or heat transfer fluid. The focus of this study is to describe the new HX evaluation method and to illustrate the overall HX evaluation and selection processes using the three nuclear heat application cases that represent the thermal integration between advanced nuclear reactors and industrial processes. As a preliminary study, three different types of HX, i.e., shell-and-tube heat exchanger, printed circuit heat exchanger, and heat pipe heat exchanger, were compared.

42 ENGINEERING↗