Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation of test systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Icing simulation: A survey of computer models and experimental facilities

A survey of the current methods for simulation of the response of an aircraft or aircraft subsystem to an icing encounter is presented. The topics discussed include a computer code modeling of aircraft icing and performance degradation, an evaluation of experimental facility simulation capabilities, and ice protection system evaluation tests in simulated icing conditions. Current research focused on upgrading simulation fidelity of both experimental and computational methods is discussed. The need for the increased understanding of the physical processes governing ice accretion, ice shedding, and iced aerodynamics is examined.

Potapczuk, M. G.↗

Automatic tools for system testing

As spacecraft control and other space-related ground systems become increasingly complex, the effort required in testing and validation also increases. Implementation of a spacecraft control system normally involves a number of incremental deliveries. In addition kernel or general purpose software may also be involved, which must itself be considered in the integration and testing program. Tools can be used to assist this testing. These can reduce the effort required or alternatively they can ensure that for a given level of effort, a better job is done. Great benefit could be derived by automating certain types of testing (interactive software) which up to now has been performed manually at a terminal. This paper reports on an on-going study. The study examines means of automating spacecraft control system testing, evaluates relevant commercial tools and aims to prototype basic automatic testing functions.

Peccia, N. M.↗

Using an Advanced Distribution Management System Test Bed to Evaluate the Impact of Model Quality on Volt/VAR Optimization: Preprint

In this paper, we present a test bed for evaluating existing and future advanced distribution management system (ADMS) applications in a realistic laboratory setting, including other utility management systems and field equipment. We present an example of using it to evaluate the impact of the ADMS network model quality on a Volt/VAR optimization (VVO) application. The test bed integrates a commercial ADMS with a real-time simulation model of a utility distribution feeder. Representative power and controller hardware are integrated through hardware-in-the-loop (HIL) techniques. The performance of the ADMS VVO application is also evaluated for different levels of measurement density. Initial results indicate that a higher model quality achieves the highest possible energy savings while avoiding voltage violations, whereas a lower model quality results in increased energy savings but at the expense of more voltage violations.

ADMS↗

Evaluation Applied to Reliability Analysis of Reconfigurable, Highly Reliable, Fault-Tolerant, Computing Systems for Avionics

Emulation techniques are proposed as a solution to a difficulty arising in the analysis of the reliability of highly reliable computer systems for future commercial aircraft. The difficulty, viz., the lack of credible precision in reliability estimates obtained by analytical modeling techniques are established. The difficulty is shown to be an unavoidable consequence of: (1) a high reliability requirement so demanding as to make system evaluation by use testing infeasible, (2) a complex system design technique, fault tolerance, (3) system reliability dominated by errors due to flaws in the system definition, and (4) elaborate analytical modeling techniques whose precision outputs are quite sensitive to errors of approximation in their input data. The technique of emulation is described, indicating how its input is a simple description of the logical structure of a system and its output is the consequent behavior. The use of emulation techniques is discussed for pseudo-testing systems to evaluate bounds on the parameter values needed for the analytical techniques.

Migneault, G. E.↗

Emulation applied to reliability analysis of reconfigurable, highly reliable, fault-tolerant computing systems

Emulation techniques applied to the analysis of the reliability of highly reliable computer systems for future commercial aircraft are described. The lack of credible precision in reliability estimates obtained by analytical modeling techniques is first established. The difficulty is shown to be an unavoidable consequence of: (1) a high reliability requirement so demanding as to make system evaluation by use testing infeasible; (2) a complex system design technique, fault tolerance; (3) system reliability dominated by errors due to flaws in the system definition; and (4) elaborate analytical modeling techniques whose precision outputs are quite sensitive to errors of approximation in their input data. Next, the technique of emulation is described, indicating how its input is a simple description of the logical structure of a system and its output is the consequent behavior. Use of emulation techniques is discussed for pseudo-testing systems to evaluate bounds on the parameter values needed for the analytical techniques. Finally an illustrative example is presented to demonstrate from actual use the promise of the proposed application of emulation.

Migneault, G. E.↗

The application of emulation techniques in the analysis of highly reliable, guidance and control computer systems

Emulation techniques can be a solution to a difficulty that arises in the analysis of the reliability of guidance and control computer systems for future commercial aircraft. Described here is the difficulty, the lack of credibility of reliability estimates obtained by analytical modeling techniques. The difficulty is an unavoidable consequence of the following: (1) a reliability requirement so demanding as to make system evaluation by use testing infeasible; (2) a complex system design technique, fault tolerance; (3) system reliability dominated by errors due to flaws in the system definition; and (4) elaborate analytical modeling techniques whose precision outputs are quite sensitive to errors of approximation in their input data. Use of emulation techniques for pseudo-testing systems to evaluate bounds on the parameter values needed for the analytical techniques is then discussed. Finally several examples of the application of emulation techniques are described.

Migneault, Gerard E.↗

ENVIRONMENTAL REACTIVITY OF SOLID-STATE HYDROGEN SYSTEMS: FUNDAMENTAL TESTING AND EVALUATION

In order to enable the commercial acceptance of solid state hydrogen storage materials and systems it is important to understand the risks associated with the environmental exposure of various materials. In some instances, these materials are sensitive to the environment surrounding the material and the behavior is unique and independent to each material. The development of testing procedures to evaluate a material's behavior with different environmental exposures is a critical need. In some cases material modifications may be needed in order to reduce the risk of environmental exposure. We have redesigned two standardized UN tests for clarity and exactness; the burn rate and self-heating tests. The results of these and other UN tests are shown for ammonia borane, NH{sub 3}BH{sub 3}, and alane, AlH{sub 3}. The burn rate test showed a strong dependence on the preparation method of aluminum hydride as the particle size and trace amounts of solvent greatly influence the test results. The self-heating test for ammonia borane showed a failed test as low as 70 C in a modified cylindrical form. Finally, gas phase calorimetry was performed and resulted in an exothermic behavior within an air and 30%RH environment.

08 HYDROGEN↗

Evaluation of cryogenic system test options for the OTV on-orbit propellant depot

Future space missions to geosynchronous, lunar, and planetary orbits will require an orbital depot for stockpiling propellants. This depot will provide long-term storage of cryogens, requiring new technologies for fluid management in microgravity and further development of thermal management technologies for minimization of cryogen boiloff. Preliminary evaluations have been made to define a test program approach for reducing technical risk through verifying performance models and building a base of engineering data for depot design. A number of testing options were defined and evaluated, leading to selection of ground testing combined with an orbital systems test. Ground testing is inadequate because of critical microgravity concerns; extending testing aboard the Space Station was eliminated because the data would not be available soon enough to benefit the propellant depot design. The orbital test would either be a short-term test carried out in the cargo bay of the Space Shuttle Orbiter using a nonhazardous cryogen or a longer term test carried out with hydrogen aboard a free-flying experiment orbited with an expendable launch vehicle.

Schuster, John R.↗

Review of Wave Energy Converter Power Take-Off Systems, Testing Practices, and Evaluation Metrics: Preprint

While the field of wave energy has been the subject of numerical simulation, scale model testing, and precommercial project testing for decades, wave energy technologies remain in the early stages of development and must continuing proving themselves as a promising modern renewable energy field. One of the difficulties that wave energy systems have been struggling to overcome is the design of highly efficient energy conversion systems that can convert the mechanical power, derived from the oscillation of wave activated bodies, into another useful product. Often the power take-off (PTO) is defined as the single unit responsible for converting mechanical power into another usable form such as electricity, pressurized fluid, compressed air, and others. The PTO, and the entire power conversion chain (PCC), is of great importance as it affects not only how efficient wave power is converted into electricity, but also contributes to the mass, size, structural dynamics, and levelized cost of energy (LCOE) of the wave energy converter (WEC). Unlike wind and solar, there is no industrial standard device, or devices, for wave energy conversion and this diversity is transferred to the PTO system. The majority of current WEC PTO systems incorporate a mechanical or hydraulic drive train, power generator, and an electrical control system. The challenge of WEC PTO designs is designing a mechanical-to-electrical component that can efficiently convert irregular, bi-directional, low frequency and low alternating velocity wave motions. While gross average power levels can be predicted in advance, the variable wave elevation input has to be converted into smooth electrical output and hence usually necessitates some type of energy storage system, such as battery storage, accumulator super capacitors, etc., or other means of compensation such as an array of devices. One of the primary challenges for wave energy converter systems is the fluctuating nature of wave resources, which require WEC components to be designed to handle loads (i.e. torques, forces, and powers) that are many times greater than the average load. This approach requires a much greater PTO capacity than the average power output and indicates a higher cost. In addition, supporting mechanical coupling and or gearing can be added to the PCC to help alleviate the difficulties with transmission and control of fluctuating large loads with low frequencies (indicative of wave forcing) into smaller loads at higher frequencies (optimum for conventional electrical machine design) can quickly increase the complexity of the PCC which could result in a greater number of failure modes and increased maintenance costs. All of the previous points demonstrate how the PTO influences WEC dynamics, reliability, performance and cost which are critical design factors. This paper further explores these topics by providing a review of the state-of-the-art PTO systems currently under development, how these novel PTO systems are tested and derisked prior to precommercial deployment, and the evaluation metrics historically used to differentiate between PTO designs and how they can be improved to support control co-design focused development of wave energy systems.

laboratory testing↗

A NOVEL TOOL FOR IMPROVING THE DATA COLLECTION PROCESS DURING CONTROL ROOM MODERNIZATION HUMAN-SYSTEM INTERFACE TESTING AND EVALUATION ACTIVITIES

The U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program is developing a scientific basis to extend the existing U.S. nuclear power plant (NPP) operating life beyond the current 60-year licensing period and to ensure NPP long-term reliability, productivity, safety, and security. Under the Advanced Instrumentation, Information, and Control (II&C) Systems Technologies pathway, the LWRS Program conducts targeted research and development to address long-term aging and obsolescence of existing instrumentation and control technologies, as well as to implement and test new digital II&C technology that enables broad innovation and business improvement in the NPP operating model. To this end, the role of human factors engineering (HFE) is important as part of ensuring that the control room modernization (CRM) activities account for the operator’s cognitive and physiological characteristics. Part of the HFE process for CRM entails conducting tests and evaluations (T&Es) that will inform the design of new human-system interface (HSI) technologies. Qualitative data that captures operator feedback and important human-system interactions are an important part of T&E. However, the analyzing this qualitative data in order to provide actionable design recommendations can be time-consuming and error prone. This paper presents an early version of a tool that can be used to analyze this qualitative data to (1) sift through freeform text for improved design issue identification, (2) maintain consistency in theme coding conventions, and (3) readily provide actionable design guidance that can be traceable to state-of-the-art HFE guidelines such as NUREG-0700. A description of this tool is discussed next.

99 GENERAL AND MISCELLANEOUS↗

A novel tool for improving the data collection process during control room modernization human-system interface testing and evaluation activities - 271

The U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program is developing a scientific basis to extend the existing U.S. nuclear power plant (NPP) operating life beyond the current 60-year licensing period and to ensure NPP long-term reliability, productivity, safety, and security. Under the Advanced Instrumentation, Information, and Control (II and C) Systems Technologies pathway, the LWRS Program conducts targeted research and development to address long-term aging and obsolescence of existing instrumentation and control technologies, as well as to implement and test new digital II and C technology that enables broad innovation and business improvement in the NPP operating model. To this end, the role of human factors engineering (HFE) is important as part of ensuring that the control room modernization (CRM) activities account for the operator's cognitive and physiological characteristics. Part of the HFE process for CRM entails conducting tests and evaluations (T and Es) that will inform the design of new human-system interface (HSI) technologies. Qualitative data that captures operator feedback and important human-system interactions are an important part of T and E. However, the analyzing this qualitative data in order to provide actionable design recommendations can be time-consuming and error prone. This paper presents an early version of a tool, which uses a combination of natural language processing and supervised machine learning to support the analysis of qualitative data. The objectives of this tool is to improve the process of (1) sifting through freeform text for improved design issue identification, (2) maintaining consistency in theme coding conventions, and (3) readily providing actionable design guidance that can be traceable to state-of-the-art HFE guidelines such as NUREG-0700. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The pseudonoise test set: Communication system's performance evaluation based upon RMS error testing

A pseudonoise (PN) test set was built to provide a relatively easy means of accurately determining the end-to-end rms error introduced by a communication system when subjected to wideband data. It utilizes a filtered pseudorandom sequence generator as a wideband data source, providing a convenient means for digitally delaying the input reference signal for comparison with the distorted test communication system output. In addition to providing a means to measure the end-to-end rms error and the average delay of a communication system, the PN test set also provides a means to determine the tested system's impulse response and correlation function. The theory of PN testing is discussed in detail along with the most difficult aspects of implementation, the building of matched filter pairs. Both analytical and empirical results are reported which support the contentions that this is an accurate and practical way to acquire figures of merit for complete communication systems.

Wallace, G. R.↗

Test and Evaluation of Systems with Embedded Machine Learning Components

As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.

97 MATHEMATICS AND COMPUTING↗

Analysis and testing of stability augmentation systems

Testing and evaluation of stability augmentation systems for aircraft flight control were conducted. The flutter suppression system analysis of a scale supersonic transport wing model is described. Mechanization of the flutter suppression system is reported. The ride control synthesis for the B-52 aeroelastic model is discussed. Model analyses were conducted using equations of motion generated from generalized mass and stiffness data.

Sevart, F. D.↗