Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Objectively Quantifying Radiation Esophagitis With Novel Computed Tomography–Based Metrics

Purpose: To study radiation-induced esophageal expansion as an objective measure of radiation esophagitis in patients with non-small cell lung cancer (NSCLC) treated with intensity modulated radiation therapy. Methods and Materials: Eighty-five patients had weekly intra-treatment CT imaging and esophagitis scoring according to Common Terminlogy Criteria for Adverse Events 4.0, (24 Grade 0, 45 Grade 2, and 16 Grade 3). Nineteen esophageal expansion metrics based on mean, maximum, spatial length, and volume of expansion were calculated as voxel-based relative volume change, using the Jacobian determinant from deformable image registration between the planning and weekly CTs. An anatomic variability correction method was validated and applied to these metrics to reduce uncertainty. An analysis of expansion metrics and radiation esophagitis grade was conducted using normal tissue complication probability from univariate logistic regression and Spearman rank for grade 2 and grade 3 esophagitis endpoints, as well as the timing of expansion and esophagitis grade. Metrics' performance in classifying esophagitis was tested with receiver operating characteristic analysis. Results: Expansion increased with esophagitis grade. Thirteen of 19 expansion metrics had receiver operating characteristic area under the curve values >0.80 for both grade 2 and grade 3 esophagitis endpoints, with the highest performance from maximum axial expansion (MaxExp1) and esophageal length with axial expansion ≥30% (LenExp30%) with area under the curve values of 0.93 and 0.91 for grade 2, 0.90 and 0.90 for grade 3 esophagitis, respectively. Conclusions: Esophageal expansion may be a suitable objective measure of esophagitis, particularly maximum axial esophageal expansion and esophageal length with axial expansion ≥30%, with 2.1 Jacobian value and 98.6 mm as the metric value for 50% probability of grade 3 esophagitis. The uncertainty in esophageal Jacobian calculations can be reduced with anatomic correction methods.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A Summary of the Department of Energy’s Advanced Demonstration and Test Reactor Options Study

An assessment of advanced reactor technology options was conducted to provide a sound comparative technical context for future decisions by the U.S. Department of Energy (DOE) concerning these technologies. Strategic objectives were established that span a wide variety of important missions, and advanced reactor technology needs were identified based on recent DOE and international studies. A broad team of stakeholders from industry, academia, and government was assembled to develop a comprehensive set of goals, criteria, and metrics to evaluate advanced irradiation test and demonstration reactor concepts. Point designs of a select number of concepts were commissioned to provide a deeper technical basis for evaluation. The technology options were compared on the bases of technical readiness and the ability to meet the different strategic objectives. Using the study's evaluation criteria and metrics, an independent group of experts from industry, universities, and national laboratories scored each of the point designs. Pathways to deployment for concepts of varying technical maturities were estimated for the different demonstration systems with regard to cost, schedule, and possible licensing approaches. This study also presents the trade-offs that exist among the different irradiation test reactor options in terms of the ability to conduct irradiations in support of advanced reactor research and development and to serve potential secondary missions. The main findings of the study indicate the following: (1) for industrial process heat supply, a high-temperature gas-cooled reactor is the best choice because of the high outlet temperature of the reactor and its strong passive and inherent safety characteristics; (2) for resource utilization and waste management, a sodium-cooled fast reactor (SFR) is best because of the use of a fast flux to destroy actinides; (3) to realize the advantages of a promising but less-mature technology, a fluoride salt-cooled high-temperature reactor and a lead-cooled fast reactor fare about the same; (4) for fulfilling the needs of a materials test reactor, a SFR is considered best because of its ability to produce high fast flux, incorporate test loops, and provide additional large volumes for testing.

Advanced reactors↗

A Summary of the Department of Energy's Advanced Demonstration and Test Reactor Options Study

An assessment of advanced reactor technology options was conducted to provide a sound comparative technical context for future decisions by the Department of Energy (DOE) concerning these technologies. Strategic objectives were established that span a wide variety of important missions and advanced reactor technology needs were identified based on recent Department of Energy and international studies. A broad team of stakeholders from industry, academia and government was assembled to develop a comprehensive set of goals, criteria and metrics to evaluate advanced irradiation test and demonstration reactor concepts. Point designs of a select number of concepts were commissioned to provide a deeper technical basis for evaluation. The technology options were compared on the bases of technical readiness and the ability to meet the different strategic objectives. Using the study’s evaluation criteria and metrics, an independent group of exerts from industry, universities and national labs scored each of the point designs. Pathways to deployment for concepts of varying technical maturity were estimated for the different demonstration systems with regard to cost, schedule and possible licensing approaches. Furthermore, this study also presents the tradeoffs that exist among the different irradiation test reactor options in terms of the ability to conduct irradiations in support of advanced reactor research and development (R&D) and to serve potential secondary missions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Wind Turbine Drivetrain Reliability Research - Gearbox Bearing Axial Cracking Failure Mode Example

The U.S. Department of Energy's National Renewable Energy Laboratory and Argonne National Laboratory have been conducting wind turbine drivetrain (formerly gearbox) reliability research for many years. Although the drivetrain focus has not changed, detailed projects are adjusted every few years based on dynamic needs seen in the field across the wind industry. This webinar will walk through the research methodology by using wind turbine gearbox bearing axial cracking failure mode as an example. The detailed steps include: 1) top failure mode identification based on actual failure data collected from project partners, 2) bench-top testing to identify possible contributing factors and formulate a damage metric, 3) physics domain modeling and validation through testing, 4) reliability assessment and prognosis based on the physics domain model and data domain inputs, and further enhancement through machine learning algorithms, using actual wind plant operational and failure event data. Hopefully, the presented work is of interest to the IISE community, and some members can apply their expertise to wind turbine and plant applications, helping enhance wind power generation technology advancement and its broader deployment.

axial cracking↗

A study of a proposed modified torsional agility metric

A new candidate lateral agility metric, the modified torsional agility parameter, is proposed and tested through generic, nonlinear, non-real-time flight simulation programs of the F-18 and F-5A. The metric is aimed at quantifying high subsonic loaded roll capabilities which might be useful in modern air combat. The metric is considered to be straightforward for testing and measuring based on nonreal-time unmanned flight simulation. The metric is found to be sensitive to pilot input errors of less than full lateral stick to capture bank angle, when tested using unmanned flight simulations. It is suggested that, for redesigned configurations of both aircraft with improved lateral agility, the major benefit would be provided by fast and highly effective rudders, and a high level of pitch, roll, and yaw damping at moderate to high normal load factor levels.

Valasek, John↗

WE-H-207A-04: Impact of Lesion Location On the Repeatability of 18F-NaF PET/CT

Purpose: Quantifying the repeatability of imaging biomarkers is critical for assessing therapeutic response. While {sup 18}F-NaF PET/CT has shown to be a repeatable imaging method, research has not shown which factors may influence its repeatability. The purpose of this study was to evaluate whether the location of the lesion impacts the repeatability of quantitative {sup 18}F-NaF PET-derived SUV metrics. Methods: Metastatic castrate-resistant prostate cancer patients with multiple bone lesions received whole-body test-retest NaF PET/CT scans. Malignant bone lesions of PET-defined volume greater than 1.5 cm{sup 3} were identified by a nuclear medicine physician and automatically delineated using a SUV>15 threshold. The maximum (SUVmax), average (SUVmean), and total (SUVtotal) SUV were extracted from each lesion. Atlas-based segmentation was used to divide each patient skeleton into 25 skeletal regions. Test-retest repeatability of each SUV metric was assessed with coefficient of variation (CV). Results: A total of 265 malignant bone lesions from 18 patients were identified by nuclear medicine physician. The largest proportion of bone lesions were localized to the spine (41%), with 41% of those lesions localized to the thoracic spine. One-way ANOVA showed that measurement differences differed significantly for all three metrics across locations (p<0.01 for each metrics). Overall, CV was smallest for SUVmean at 5.3%, followed by SUVmax at 11.5% and SUVtotal at 20.4%. Lesions in the pubis were consistently the most repeatable (CV(SUVmax)= 5.6%, CV(SUVmean)= 0.6%, CV(SUVtotal)= 2.9%). According to SUVmean, repeatability was poorest in the cervical spine (CV = 6.2%), whereas according to SUVmax and SUVtotal, repeatability was poorest in the ribs (CV(SUVmax)= 15.0%, CV(SUVtotal)= 29.8%). Conclusion: Location of the lesion affects the repeatability of {sup 18}F-NaF PET/CT, with the ribs and cervical spine having the lowest repeatability and the pubis having the highest repeatability. These results can be used to establish location-specific response criteria for NaF PET-based treatment response assessment.

60 APPLIED LIFE SCIENCES↗

Small Punch Testing of Molybdenum-99 Targets

Northstar Medical Radioisotopes is developing an accelerator-based method to produce 99 Mo, which is a parent isotope of the commonly used 99m Tc medical isotope. The Mo targets being designed for the accelerator will be produced from enriched 100 Mo, also known as aMo . Pressed and sintered powder feedstock is used to fabricate aMo targets, producing 29 mm disk-shaped targets. The targets are subjected to 1–6 days in line of an electron beam with subsequent dissolution of the disk to retain the 99 Mo, which decays to 99m Tc at radio-pharmacies. The press and sinter method is advantageous because the inherent porosity produced by this method enables increased surface area and therefore increased flow of dissolution media, decreasing the dissolution time and reducing the need for a highly acidic media. Although porosity aids in dissolution, it reduces the mechanical strength and ductility. Targets require good mechanical integrity when subjected to the conditions in the accelerator. Therefore, Northstar is seeking methods to rapidly test disk samples after fabrication to assure mechanical performance metrics are achieved. This report details the design and testing of a small punch test (SPT) that accommodates the 29 mm disk. Initial data were used to relate the SPT data to tensile properties, such as the yield strength (YS), ultimate tensile strength (UTS), and total elongation to failure. Although the SPT has been established as a somewhat reliable method for testing metallic materials, few studies have applied the SPT to refractory materials such as Mo. Based on tensile testing performed at Oak Ridge National Laboratory on different Mo samples, correlation between the Mo tensile and SPT properties could be performed, establishing a standard calibration that could be applied to other Mo samples. To test the efficacy of the SPT with Mo, multiple different disk batches were fabricated under different conditions (e.g., pressure, lubricant) with commercially available pure Mo powder. Generally, only a UTS could be well defined because the press and sinter disks failed under brittle fracture, making it difficult to determine the YS and elongation. Compared with disks fabricated with aMo powder, the aMo samples underperformed their pure nat Mo counterparts. This report summarizes the current status of the SPT, but further evaluation is needed before it can be applied as a reliable quality assurance tool.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A menu of self-administered microcomputer-based neurotoxicology tests

This study examined the feasibility of repeated self-administration of a newly developed battery of mental acuity tests. Researchers developed this battery to be used to screen the fitness for duty of persons in at-risk occupations (astronauts, race car drivers), or those who may be exposed to environmental stress, toxic agents, or disease. The menu under study contained cognitive and motor tests implemented on a portable microcomputer including: a five-test core battery, lasting six minutes, which had demonstrable reliabilities and stability from several previous repeated-measures studies, and also 13 new tests, lasting 42 minutes, which had appeared in other batteries but had not yet been evaluated for repeated-measures implementation in this medium. Sixteen subjects self-administered the battery over 10 repeated sessions. The hardware performed well throughout the study and the tests appeared to be easily self-administered. Stabilities and reliabilities of the test from the core battery were comparable to those obtained previously under more controlled experimental conditions. Analyses of metric properties of the remaining 13 tests produced eight additional tests with satisfactory properties. Although the average retest reliability was high, cross-correlations between tests were low, indicating factorial richness. The menu can be used to form batteries of flexible total testing time which are likely to tap different mental processes and functions.

Kennedy, Robert S.↗

Proceedings of the Second NASA Formal Methods Symposium

This publication contains the proceedings of the Second NASA Formal Methods Symposium sponsored by the National Aeronautics and Space Administration and held in Washington D.C. April 13-15, 2010. Topics covered include: Decision Engines for Software Analysis using Satisfiability Modulo Theories Solvers; Verification and Validation of Flight-Critical Systems; Formal Methods at Intel -- An Overview; Automatic Review of Abstract State Machines by Meta Property Verification; Hardware-independent Proofs of Numerical Programs; Slice-based Formal Specification Measures -- Mapping Coupling and Cohesion Measures to Formal Z; How Formal Methods Impels Discovery: A Short History of an Air Traffic Management Project; A Machine-Checked Proof of A State-Space Construction Algorithm; Automated Assume-Guarantee Reasoning for Omega-Regular Systems and Specifications; Modeling Regular Replacement for String Constraint Solving; Using Integer Clocks to Verify the Timing-Sync Sensor Network Protocol; Can Regulatory Bodies Expect Efficient Help from Formal Methods?; Synthesis of Greedy Algorithms Using Dominance Relations; A New Method for Incremental Testing of Finite State Machines; Verification of Faulty Message Passing Systems with Continuous State Space in PVS; Phase Two Feasibility Study for Software Safety Requirements Analysis Using Model Checking; A Prototype Embedding of Bluespec System Verilog in the PVS Theorem Prover; SimCheck: An Expressive Type System for Simulink; Coverage Metrics for Requirements-Based Testing: Evaluation of Effectiveness; Software Model Checking of ARINC-653 Flight Code with MCP; Evaluation of a Guideline by Formal Modelling of Cruise Control System in Event-B; Formal Verification of Large Software Systems; Symbolic Computation of Strongly Connected Components Using Saturation; Towards the Formal Verification of a Distributed Real-Time Automotive System; Slicing AADL Specifications for Model Checking; Model Checking with Edge-valued Decision Diagrams; and Data-flow based Model Analysis.

Munoz, Cesar↗

Development of a Ground Test and Analysis Protocol to Support NASA's NextSTEP Phase 2 Habitation Concepts

The NASA Next Space Technologies for Exploration Partnerships (NextSTEP) program is a public-private partnership model that seeks commercial development of deep space exploration capabilities to support extensive human spaceflight missions around and beyond cislunar space. NASA first issued the Phase 1 NextSTEP Broad Agency Announcement to U.S. industries in 2014, which called for innovative cislunar habitation concepts that leveraged commercialization plans for low Earth orbit. These habitats will be part of the Deep Space Gateway (DSG), the cislunar space station planned by NASA for construction in the 2020s. In 2016, Phase 2 of the NextSTEP program selected five commercial partners to develop ground prototypes. A team of NASA research engineers and subject matter experts have been tasked with developing the ground test protocol that will serve as the primary means by which these Phase 2 prototype habitats will be evaluated. Since 2008, this core test team has successfully conducted multiple spaceflight analog mission evaluations utilizing a consistent set of operational products, tools, methods, and metrics to enable the iterative development, testing, analysis, and validation of evolving exploration architectures, operations concepts, and vehicle designs. The purpose of implementing a similar evaluation process for the NextSTEP Phase 2 Habitation Concepts is to consistently evaluate the different commercial partner ground prototypes to provide data-driven, actionable recommendations for Phase 3.

Beaton, Kara H.↗

Investigation of similarity metrics for simulation based scaling analysis

Scaling analysis is a technique for characterizing degree of similarity between processes to occur in a real prototypic system and those to occur in its reduced scale (experimental) analog. In the context of nuclear reactor thermal-hydraulics and safety, scaling analysis is instrumental in assessing the applicability of data obtained from a scaled test facility for calibration of models and validation of simulation codes. Quantification of scale-related distortions is the key element in scaling analysis. In this work, we examine an approach for scaling analysis using computer simulation codes. Similarity analysis using information-theoretic approach is performed by comparing performance of experiment and application to variation of system conditions. Three different information theoretic measures namely mutual information, Kullback-Leibler divergence and overlap coefficient are selected for their potential use as metric for similarity analysis. We test the metrics by comparing them against physics-based scaling ratios, which are well known in fundamental thermal-hydraulic processes. In this exploratory analysis, a case of pressure drop of a steady-state flow through a channel is employed to test the proposed approach. It was found that in the absence of model uncertainties, mutual information serve as an effective measure of the scale effects present in the experiment. However, model uncertainty can significantly impact the quality of similarity analysis. Our analysis indicates that model uncertainties can have significant impact on the information theoretic metrics used for scaling analysis. Furthermore, if the model uncertainties exceeds beyond a critical value, the scale effect present in the experiment cannot be adequately evaluated. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Drawbar Pull (DP) Procedures for Off-Road Vehicle Testing

As NASA strives to explore the surface of the Moon and Mars, there is a continued need for improved tire and vehicle development. When tires or vehicles are being designed for off-road conditions where significant thrust generation is required, such as climbing out of craters on the Moon, it is important to use a standard test method for evaluating their tractive performance. The drawbar pull (DP) test is a way of measuring the net thrust generated by tires or a vehicle with respect to performance metrics such as travel reduction, sinkage, or power efficiency. DP testing may be done using a single tire on a traction rig, or with a set of tires on a vehicle; this report focuses on vehicle DP tests. Though vehicle DP tests have been used for decades, there are no standard procedures that apply to exploration vehicles. This report summarizes previous methods employed, shows the sensitivity of certain test parameters, and provides a body of knowledge for developing standard testing procedures. The focus of this work is on lunar applications, but these test methods can be applied to terrestrial and planetary conditions as well. Section 1.0 of this report discusses the utility of DP testing for off-road vehicle evaluation and the metrics used. Section 2.0 focuses on test-terrain preparation, using the example case of lunar terrain. There is a review of lunar terrain analogs implemented in the past and a discussion on the lunar terrain conditions created at the NASA Glenn Research Center, including methods of evaluating the terrain strength variation and consistency from test to test. Section 3.0 provides details of the vehicle test procedures. These consist of a review of past methods, a comprehensive study on the sensitivity of test parameters, and a summary of the procedures used for DP testing at Glenn.

wheel↗

Using Dispersed Modes During Model Correlation

The model correlation process for the modal characteristics of a launch vehicle is well established. After a test, parameters within the nominal model are adjusted to reflect structural dynamics revealed during testing. However, a full model correlation process for a complex structure can take months of man-hours and many computational resources. If the analyst only has weeks, or even days, of time in which to correlate the nominal model to the experimental results, then the traditional correlation process is not suitable. This paper describes using model dispersions to assist the model correlation process and decrease the overall cost of the process. The process creates thousands of model dispersions from the nominal model prior to the test and then compares each of them to the test data. Using mode shape and frequency error metrics, one dispersion is selected as the best match to the test data. This dispersion is further improved by using a commercial model correlation software. In the three examples shown in this paper, this dispersion based model correlation process performs well when compared to models correlated using traditional techniques and saves time in the post-test analysis.

Stewart, Eric C.↗

Spatial patterns of historical crop yields reveal soil health attributes in US Midwest fields

Abstract Attaining high crop yields and increasing carbon storage in agricultural soils, while avoiding negative environmental impacts on water quality, soil erosion, and biodiversity, requires accurate and precise management of crop inputs and management practices. The long-term analysis of spatial and temporal patterns of crop yields provides insights on how yields vary in a field, with parts of field constantly producing either high yields or low yields and other parts that fluctuate from one year to the next. The concept of yield stability has shown to be informative on how plants translate the effects of environmental conditions (e.g., soil, climate, topography) across the field and over the years in the final yield, and as a valuable layer in developing prescription maps of variable fertilizer rate inputs. Using known relationships between soil health and crop yields, we hypothesize that areas with measured constantly low yield will return low carbon to the soil affecting its heath. On this premises, yield stability zones (YSZ) provide an effective and practical integrative measure of the small-scale variability of soil health on a field relative basis. We tested this hypothesis by measuring various metrics of soil health from commercial farmers’ fields in the north central Midwest of the USA in samples replicated across YSZ, using a soil test suite commonly used by producers and stakeholders active in agricultural carbon credits markets. We found that the use of YSZ allowed us to successfully partition field-relative soil organic carbon (SOC) and soil health metrics into statistically distinct regions. Low and stable (LS) yield zones were statistically lower in normalized SOC when compared to high and stable (HS) and unstable (US) yield zones. The drivers of the yield differences within a field are a series of factors ranging from climate, topography and soil. LS zones occur in areas of compacted soil layers or shallow soils (edge of the field) on steeper slopes. The US zones occurring with high water flow accumulation, were more dependent on topography and rainfall. The differences in the components of the overall soil health score (SHS) between these YSZ increased with sample depth suggesting a deeper topsoil in the US and HS zones, driven by the accumulation of water, nutrients, and carbon downslope. Comparison of the field management provided initial evidence that zero tillage reduces the magnitude of the variance in SOC and soil health metrics between the YSZ.

Fowler, Ames↗

pnnl/i2x

Interconnection Innovation e-Xchange (i2x) Test Systems allows for scripted customizations of social behavior, recovery operations, investment decisions, and metrics in a simplified Python module. Public test systems allow for replication of results by other stakeholders

McDermott, Tom↗

Comparison of Test Data Using Different Tools Available in W-13 [Slides]

Here, we show progress on recent work performed in our team to answer system engineer’s questions; solicit feedback on metrics and provide suggestions on other test comparison examples; and use quasi “tool-time” to share two tools that fellow engineers in our group have developed and are using.

42 ENGINEERING↗

Advanced Diagnostic and Prognostic Testbed (ADAPT) Testability Analysis Report

As system designs become more complex, determining the best locations to add sensors and test points for the purpose of testing and monitoring these designs becomes more difficult. Not only must the designer take into consideration all real and potential faults of the system, he or she must also find efficient ways of detecting and isolating those faults. Because sensors and cabling take up valuable space and weight on a system, and given constraints on bandwidth and power, it is even more difficult to add sensors into these complex designs after the design has been completed. As a result, a number of software tools have been developed to assist the system designer in proper placement of these sensors during the system design phase of a project. One of the key functions provided by many of these software programs is a testability analysis of the system essentially an evaluation of how observable the system behavior is using available tests. During the design phase, testability metrics can help guide the designer in improving the inherent testability of the design. This may include adding, removing, or modifying tests; breaking up feedback loops, or changing the system to reduce fault propagation. Given a set of test requirements, the analysis can also help to verify that the system will meet those requirements. Of course, a testability analysis requires that a software model of the physical system is available. For the analysis to be most effective in guiding system design, this model should ideally be constructed in parallel with these efforts. The purpose of this paper is to present the final testability results of the Advanced Diagnostic and Prognostic Testbed (ADAPT) after the system model was completed. The tool chosen to build the model and to perform the testability analysis with is the Testability Engineering and Maintenance System Designer (TEAMS-Designer). The TEAMS toolset is intended to be a solution to span all phases of the system, from design and development through health management and maintenance. TEAMS-Designer is the model-building and testability analysis software in that suite.

Ossenfort, John↗

Landsat 9 TIRS-2 Performance Results Based on Subsystem-Level Testing

Landsat 9 is the next in the series of Landsat satellites and has a complement of two pushbroom imagers: Operational Land Imager-2 (OLI-2) that samples the solar reflective spectrum with nine channels and Thermal Infrared Sensor-2 (TIRS-2) samples the thermal infrared spectrum with two channels. The first builds of these sensors, OLI and TIRS, were launched on Landsat 8 in 2013 and Landsat 9 is expected to launch in December 2020. TIRS-2 is designed and built to continue the Landsat data record and satisfy the needs of the remote sensing community. There are two sets of requirements considered for planning the component, subsystem and instrument level tests for TIRS-2: performance requirements and Special Calibration Test Requirements (SCTR). The performance requirements specify key spectral, spatial, radiometric, and operational parameters of TIRS-2 while the SCTRs specify parameters of how the instrument is tested. Several requirements can only be verified at the instrument level, but many performance metrics can be assessed earlier in prelaunch testing at the subsystem level. A test program called TIRS Imaging Performance and Cryoshell Evaluation (TIPCE) was developed to characterize TIRS-2 spectral, spatial, and scattered-light rejection performance at the telescope and detector subsystem level. There were three thermal vacuum campaigns in TIPCE that occurred from November 2017 to March 2018. This work shows results of TIPCE data analysis which provide confidence that key requirements will be met at instrument level with a few minor waivers. A full complement of performance testing will be done at the TIRS-2 instrument level for final verification in late 2018 through Spring 2019.

scatter↗