Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52

Subjective loudness response to simulated sonic booms

A series of laboratory studies were conducted at LaRC to: (1) quantify the effects of sonic boom signature shaping on subjective loudness; (2) evaluate candidate loudness metrics; (3) quantify the effects of signature asymmetry on loudness; and (4) document sonic boom acceptability within the laboratory. A total of 212 test subjects evaluated a wide range of signatures using the NASA Langley Research Center's sonic boom simulator. Results indicated that signature shaping via front-shock minimization was particularly effective in reducing subjective loudness without requiring reductions in peak overpressure. Metric evaluations showed that A-weighted sound exposure level, Perceived Level (Stevens Mark 7), and Zwicker's Loudness level were effective descriptors of the loudness of symmetrical shaped signatures. The asymmetrical signatures were generally rated as being quieter than symmetrical signatures of equal calculated metric level. The magnitude of the loudness reductions were observed to increase as the degree of asymmetry increased and to be greatest when the rear half of the signature was loudest. This effect was not accounted for by the loudness metrics. Sonic boom acceptability criteria were determined within the laboratory. These agreed well with results previously obtained in more realistic situations.

Leatherwood, Jack D.↗

Performance and Health Test Procedure for Grid Energy Storage Systems: Preprint

A test procedure to evaluate the performance and health of field installations of grid-connected battery energy storage systems (BESS) is described. Performance and health metrics captured in the procedures are: Round-trip efficiency, Standby losses, Response time/accuracy, and Useable Energy/ State of Charge at different discharge/charge rates over the system's lifetime. The procedures are divided into Reference Performance Tests, which require the system to be put in a test mode and are to be conducted in intervals, and Real-time Monitoring tests, which collect data during normal operation without interruption. The procedures can be applied on a wide array of BESS with little modifications and can thus support BESS operators in the management of BESS field installations with minimal interruption and expenditures.can be applied on a wide array of BESS with little modifications and can thus support BESS operators in the management of BESS field installations with minimal interruption and expenditures.

25 ENERGY STORAGE↗

Verification of BOUT++ by the method of manufactured solutions

BOUT++ is a software package designed for solving plasma fluid models. It has been used to simulate a wide range of plasma phenomena ranging from linear stability analysis to 3D plasma turbulence and is capable of simulating a wide range of drift-reduced plasma fluid and gyro-fluid models. A verification exercise has been performed as part of a EUROfusion Enabling Research project, to rigorously test the correctness of the algorithms implemented in BOUT++, by testing order-of-accuracy convergence rates using the Method of Manufactured Solutions (MMS). We present tests of individual components including time-integration and advection schemes, non-orthogonal toroidal field-aligned coordinate systems and the shifted metric procedure which is used to handle highly sheared grids. The flux coordinate independent approach to differencing along magnetic field-lines has been implemented in BOUT++ and is here verified using the MMS in a sheared slab configuration. Finally, we show tests of three complete models: 2-field Hasegawa-Wakatani in 2D slab, 3-field reduced magnetohydrodynamics (MHD) in 3D field-aligned toroidal coordinates, and 5-field reduced MHD in slab geometry.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

MO-FG-CAMPUS-TeP1-04: Pseudo-In-Vivo Dose Verification of a New Mono-Isocentric Technique for the Treatment of Multiple Brain Metastases

Purpose: To validate dose calculation and delivery accuracy of a recently introduced mono-isocentric technique for the treatment of multiple brain metastases in a realistic clinical case. Methods: Anonymized CT scans of a patient were used to model a hollow phantom that duplicates anatomy of the skull. A 3D printer was used to construct the phantom of a radiologically bone-equivalent material. The hollow phantom was subsequently filled with a polymer gel 3D dosimeter which also acted as a water-equivalent material. Irradiation plan consisted of 5 targets and was identical to the one delivered to the specific patient except for the prescription dose which was optimized to match the gel dose-response characteristics. Dose delivery was performed using a single setup isocenter dynamic conformal arcs technique. Gel dose read-out was carried out by a 1.5 T MRI scanner. All steps of the corresponding patient’s treatment protocol were strictly followed providing an end-to-end quality assurance test. Pseudo-in-vivo measured 3D dose distribution and calculated one were compared in terms of spatial agreement, dose profiles, 3D gamma indices (5%/2mm, 20% dose threshold), DVHs and DVH metrics. Results: MR-identified polymerized areas and calculated high dose regions were found to agree within 1.5 mm for all targets, taking into account all sources of spatial uncertainties involved (i.e., set-up errors, MR-related geometric distortions and registration inaccuracies). Good dosimetric agreement was observed in the vast majority of the examined profiles. 3D gamma index passing rate reached 91%. DVH and corresponding metrics comparison resulted in a satisfying agreement between measured and calculated datasets within targets and selected organs-at-risk. Conclusion: A novel, pseudo-in-vivo QA test was implemented to validate spatial and dosimetric accuracy in treatment of multiple metastases. End-to-end testing demonstrated that our gel dosimetry phantom is suited for such QA procedures, allowing for 3D analysis of both targeting placement and dose.

60 APPLIED LIFE SCIENCES↗

Aeroelastic Stability Assessment Methodology and Application to Slat Noise Treatments on the High-Lift Common Research Model

This work supports a larger effort at NASA to reduce airframe noise and, thus, environmental noise around airports. The leading-edge slat of conventional high-lift systems is a prominent source of airframe noise, and two technologies were previously identified as promising for noise reduction without significant aerodynamic or weight penalty: the slat cove filler (SCF) and the slat gap filler (SGF). NASA fabricated a 10%-scale semispan aircraft model for wind tunnel testing based upon the High-Lift Common Research Model (CRM-HL). NASA embarked on a test campaign with the 10% CRM-HL to determine the noise reduction effectiveness and other performance metrics of 3D SCF and SGF treatments in flow conditions representative of flight. The highly flexible SCF and SGF treatments posed a risk to the model and aeroelastic instability would compromise research integrity, so aeroelastic stability assessment was required. This work briefly describes the 10% CRM-HL, design of the SCF and SGF treatments to the model, and supporting fluid structure interaction (FSI) research. Relations for static and dynamic aeroelastic similitude are then developed for convenience of reference, and simplifications are introduced of particular utility to the cases considered in this study. The similitude relations are employed to assess the stability of the treatments under wind tunnel flow conditions based upon known behaviors from the supporting research. It is shown that the treatments are safe for test under the anticipated wind tunnel flow conditions with significant margins/factors of safety. It is furthermore demonstrated that the SCF is more susceptible to aeroelastic instability than the SGF, which has implications for flight feasibility.

Similitude↗

Active learning for SNAP interatomic potentials via Bayesian predictive uncertainty

Bayesian inference with a simple Gaussian error model is used to efficiently compute prediction variances for energies, forces, and stresses in the linear SNAP interatomic potential. Here, the prediction variance is shown to have a strong correlation with the absolute error over approximately 24 orders of magnitude. Using this prediction variance, an active learning algorithm is constructed to iteratively train a potential by selecting the structures with the most uncertain properties from a pool of candidate structures. The relative importance of the energy, force, and stress errors in the objective function is shown to have a strong impact upon the trajectory of their respective net error metrics when running the active learning algorithm. Batched training of different batch sizes is also tested against singular structure updates, and it is found that batches can be used to significantly reduce the number of retraining steps required with only minor impact on the active learning trajectory.

97 MATHEMATICS AND COMPUTING↗

Software stewardship and advancement of a high-performance computing scientific application: QMCPACK

Here, we provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions own runners, and NVIDIA and AMD GPUs used in pre-exascale systems, (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Docker containers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the stewardship and advancement of community HPC codes to enable scientific discovery at scale.

97 MATHEMATICS AND COMPUTING↗

Gravitomagnetic effects in Kerr-de Sitter space-time

We explicitly worked out the orbital effects induced on the trajectory of a test particle by the the weak-field approximation of the Kerr-de Sitter metric. It results that the node, the pericentre and the mean anomaly undergo secular precessions proportional to k. We used such theoretical predictions and the latest observational determinations of the non-standard precessions of the perihelia of the inner planets of the Solar System to put a bound on k getting k {<=} 10{sup -29} m{sup -2}. The node rate of the LAGEOS Earth's satellite yields k {<=} 10{sup -26} m{sup -2}. The periastron precession of the double pulsar PSR J0737-3039A/B allows to obtain k {<=} 3 Multiplication-Sign 10{sup -21} m{sup -2}. Interpreting k as a cosmological constant {Lambda}, it turns out that such constraints are weaker than those obtained from the Schwarzschild-de Sitter metric.

79 ASTRONOMY AND ASTROPHYSICS↗

Assessment of Experimental Uncertainty for a Floating Wind Semisubmersible Under Hydrodynamic Loading

The objective of this paper is to assess the sources of experimental uncertainty in an offshore wind validation campaign focused on better understanding the nonlinear hydrodynamic response behavior of a floating semisubmersible. The test specimen and conditions were simplified compared to other floating wind test campaigns to reduce potential sources of uncertainties and better focus on the hydrodynamic load attributes. Repeat tests were used to understand the repeatability of the test conditions and to assess the level of random uncertainty in the measurements. Attention was also given to understanding bias in all components of the test. The end goal of this work is to set uncertainty bounds on the response metrics of interest, which will be used in future work to evaluate the success of modeling tools in accurately calculating hydrodynamic loads and the associated motion responses of the system.

17 WIND ENERGY↗

Transport Test Problems for Radiation Detection Scenarios

Modeling forms an important component of radiation detection development, allowing for testing of new detector designs, evaluation of existing equipment against a wide variety of potential threat sources, and assessing operation performance of radiation detection systems. This can, however, result in large and complex scenarios which are time consuming to model. A variety of approaches to radiation transport modeling exist with complementary strengths and weaknesses for different problems. This variety of approaches, and the development of promising new tools (such as ORNL’s ADVANTG) which combine benefits of multiple approaches, illustrates the need for a means of evaluating or comparing different techniques for radiation detection problems. This report presents a set of 9 test problems for comparing different types of radiation transport calculations, identifying appropriate tools for classes of problems, and testing and guiding the development of new methods. Cases are drawn primarily from existing or previous calculations with a preference for cases which include experimental data, or otherwise have results with a high level of confidence, are non-sensitive, and represent problem sets of interest to NA-22. From a technical perspective, cases are chosen to span a range of difficulty and to include gamma transport, neutron transport, or both and represent different important physical processes and a range of sensitivity to angular or energy fidelity. Following test case identification, existing information about geometry, measurements, and previous calculations were assembled. Monte Carlo results (MCNP decks) were reviewed or created and re-run in order to attain accurate computational times and to verify agreement with experimental data, when present. Test case information was then conveyed to ORNL in order to guide testing and development of hybrid calculations. The results of those ADVANTG calculations were then sent to PNNL for compilation. This is a report describing the details of the selected test cases and results from available transport codes. Additional results from other transport codes will be added during the next year, along with additional cases if needed for software testing. Future work will extend the test problems to include more extensive cross-code comparisons as well as more refined metrics for evaluating the tradeoffs between operator time, computational time, and solution accuracy, to assemble a formalized benchmark set.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The Low-Mass Limit: Dark Matter Detectors with eV-Scale Energy Resolution

The SuperCDMS SNOLAB experiment will be a 20-kg scale Si and Ge direct dark matter detection experiment designed to probe down to 300 MeV in dark matter (DM) mass through DM-nucleus scattering and 500 keV in DM electron scattering. In order to reach these low masses with appreciable sensitivity to dark matter, it needs to achieve very low energy resolution ($\le 10$ ev) for nuclear recoils in both detector materials, which will be achieved using a new detector design and operating mode, CDMS HV. This detector is designed to operate at a bias of 100V to convert charges liberated in our detector targets to into phonon energy in order to resolve individual electron-hole pairs. This has never before been achieved in a kg-scale detector. In this thesis, I cover three elements of the design of the CDMS HV detectors. I discuss the detector physics controlling how charges and phonons are generated in our detector crystals, comparing theory to results of recent experiments carried out at Stanford. I move on to describe the operating principles of our phonon-mediated charge readout, as well as the design of the CDMS HV detector. I then describe the performance tests of early CDMS HV prototypes in conjunction with the SuperCDMS SNOLAB electronics, and discuss the path towards achieving single electron-hole pair resolving detectors at the kg-scale given the performance obtained thus far. As a result of these tests, we were able to refine our noise and sensor dynamics models, and develop new metrics for diagnosing non-ideal sources of noise to aid in reducing coupling of the external environment to our detectors. In order to study the microphysics of phonon and charge production in our target crystals, we fabricated a number of gram-scale devices with various sensor designs in order to separate sensor and environmental effects from intrinsic crystal properties. These devices provided the first successful demonstrating of using voltage to amplify charge energ y by production of phonons (the Neganov-Trofimov-Luke effect! ) in order to resolve electron-hole pairs, and opened up a new regime of dark matter and photon science at the gram-scale that we are just beginning to explore. A first dark matter search was carried out with one of these gram-scale devices, producing world-leading limits on electron-recoiling dark matter between 0.5 and 5 MeV in dark matter mass for multiple form factors. This device achieved a phonon resolution of 10~eV, allowing a single gram-day of exposure to rival kg-days of exposure in the competing liquid-noble based electron-recoil search.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

YOLO Test Software v1.4

This is a user manual of the software being developed at Brookhaven National Laboratory to test deep learnings algorithms. (YOLO) You Only Look Once. V1.4 added new features, including Display metric average recall, early termination option and display filename/image.

97 MATHEMATICS AND COMPUTING↗

Heavy-Duty Nonroad Material Handler Electrification Part 2: Energy Efficiency and Performance Evaluation

Decarbonization efforts achieved through electrification in nonroad mobile machinery can realize a reduction in fuel consumption of more than 20%, thanks to concepts familiar to light-duty passenger vehicles. This case study compares the results of a hybrid-electric material handler to its conventional counterpart, utilizing machine-specific drive cycles presented in part one of this paper series. The hybrid prototype features an extended-range electric vehicle (EREV) powertrain that demonstrated substantial energy efficiency improvements. Specifically, there was a reduction in equivalent fuel consumption of 75% when operating in electric-only mode, and 33% when maintaining the battery by charging with an on-board generator. Together, the efficiency improvements can be extrapolated over a low-intensity, 8-h shift characterized by significant idle time and highly dynamic engine load for a 47% reduction in net energy consumption. Key technologies that led to this improvement included engine downsizing and decoupling, regenerative braking, and an electrohydraulic pump unit with advanced controls. Finally, his study explains details of the powertrain architecture and subsystems that were implemented on a demonstration vehicle, control strategies used to meet project goals, and an analysis of energy consumption from testing on a closed course. Also included in this study is a discourse on comparison metrics that can be used for quantifying the energy consumption differences between hybrid-electric and conventional diesel powertrains in nonroad mobile machinery.

25 ENERGY STORAGE↗

Supersonic axial-force characteristics of a rectangular-box cavity with various length-to-depth ratios in a flat plate

A wind-tunnel investigation has been conducted at Mach numbers of 1.50, 2.16, and 2.86 to obtain axial-force data on a metric rectangular-box cavity with various length-to-depth ratios. The model was tested at angles of attack from -4 deg to -2 deg. The results are summarized to show variations in cavity axial-force coefficient for deep- and shallow-cavity configurations with detached and attached cavity flow fields, respectively. The results of the investigation indicate that for a wide range of cavity lengths and depths, good correlations of the cavity axial-force coefficients (based on cavity rear-face area) are obtained when these coefficients are plotted as a function of cavity length-to-depth ratio. Abrupt increases in the cavity axial-force coefficients at an angle of attack of 0 deg. reflect the transition from an open (detached) cavity flow field to a closed (attached) cavity flow field. Cavity length-to-depth ratio is the dominant factor affecting the switching of the cavity flow field from one type to the other. The type of cavity flow field (open or closed) is not dependent on the test angles of attack except near the critical value of length-to-depth ratio.

Blair, A. B., Jr.↗

Toward higher order tests of the gravitational interaction

Analyses and interpretations of experiments which test post-Newtonian gravity are usually done under the assumption that gravity is a metric field phenomenon - a manifestation of space-time geometry. This, however, is unnecessary and one can start at a more primitive level - that there simply exists a phenomenological, gravitational, many-body equation of motion which must be determined by a package of observations. In fact, over the last couple decades, a diverse collection of solar system interbody tracking observations, supplemented by data from the binary pulsar system PSR 1913 + 16, has completely mapped out the first post-Newtonian order. After the fact, using empirically determined equations of motion, along with some observed properties of nongravitational clocks and rulers and conservation laws for energy, momentum and angular momentum, a post-Newtonian Lagrangian can be constructed, a geometrical space-time metric field conceptual interpretation can be developed, Lorentz invariance of the equations of motion can be shown, and the equations of motion are found to agree with the predictions of Einstein's gravitational theory, General Relativity, within experimental accuracy.

Nordtvedt, Ken↗

Analyzing the flared landing task with pitch-rate flight control systems

A closed-loop handling qualities methodology is applied to an analysis of the flared landing task with pitch-rate flight control systems. A model of pilot behavior throughout approach and flare is developed which postulates the manner in which the pilot may move from pitch attitude to flight path angle control. Twenty-five configurations flight tested on the NC-131H Total In-flight Simulator aircraft are analyzed using a structural pilot model ad a handling qualities methodology previously reported in the literature. Closed-loop simulation of the simplified landing task is undertaken using the structural model. The pilot ratings from flight test extended the data base supporting the utility of a model-based handling qualities metric. A handling qualities sensitivity function is introduced which may have potential as a design tool.

Hess, Ronald A.↗

Mobile Computing for Aerospace Applications

The use of commercial computer technology in specific aerospace mission applications can reduce the cost and project cycle time required for the development of special-purpose computer systems. Additionally, the pace of technological innovation in the commercial market has made new computer capabilities available for demonstrations and flight tests. Three areas of research and development being explored by the Portable Computer Technology Project at NASA Ames Research Center are the application of commercial client/server network computing solutions to crew support and payload operations, the analysis of requirements for portable computing devices, and testing of wireless data communication links as extensions to the wired network. This paper will present computer architectural solutions to portable workstation design including the use of standard interfaces, advanced flat-panel displays and network configurations incorporating both wired and wireless transmission media. It will describe the design tradeoffs used in selecting high-performance processors and memories, interfaces for communication and peripheral control, and high resolution displays. The packaging issues for safe and reliable operation aboard spacecraft and aircraft are presented. The current status of wireless data links for portable computers is discussed from a system design perspective. An end-to-end data flow model for payload science operations from the experiment flight rack to the principal investigator is analyzed using capabilities provided by the new generation of computer products. A future flight experiment on-board the Russian MIR space station will be described in detail including system configuration and function, the characteristics of the spacecraft operating environment, the flight qualification measures needed for safety review, and the specifications of the computing devices to be used in the experiment. The software architecture chosen shall be presented. An analysis of the performance characteristics of wireless data links in the spacecraft environment will be discussed. Network performance and operation will be modeled and preliminary test results presented. A crew support application will be demonstrated in conjunction with the network metrics experiment.

Alena, Richard↗

Benchmarking Diagnostic Algorithms on an Electrical Power System Testbed

Diagnostic algorithms (DAs) are key to enabling automated health management. These algorithms are designed to detect and isolate anomalies of either a component or the whole system based on observations received from sensors. In recent years a wide range of algorithms, both model-based and data-driven, have been developed to increase autonomy and improve system reliability and affordability. However, the lack of support to perform systematic benchmarking of these algorithms continues to create barriers for effective development and deployment of diagnostic technologies. In this paper, we present our efforts to benchmark a set of DAs on a common platform using a framework that was developed to evaluate and compare various performance metrics for diagnostic technologies. The diagnosed system is an electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The paper presents the fundamentals of the benchmarking framework, the ADAPT system, description of faults and data sets, the metrics used for evaluation, and an in-depth analysis of benchmarking results obtained from testing ten diagnostic algorithms on the ADAPT electrical power system testbed.

Kurtoglu, Tolga↗