Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57

Open Call LDRD: Physically Informed Autoencoders for Galactic Redshift Regression

Physical constraints have been suggested to make neural network models more generalizable, act scientifically plausible, and be more data-efficient over unconstrained baselines. In this report, we present preliminary work on evaluating the effects of adding soft physical constraints to computer vision neural networks trained to estimate the conditional density of redshift on input galaxy images for the Sloan Digital Sky Survey. We introduce physically motivated soft constraint terms that are not implemented with differential or integral operators. We frame this work as a simple ablation study where the effect of including soft physical constraints is compared to an unconstrained baseline. We compare networks using standard point estimate metrics for photometric redshift estimation, as well as metrics to evaluate how faithful our conditional density estimate represents the probability over the ensemble of our test dataset. We find no evidence that the implemented soft physical constraints are more effective regularizers than augmentation.

97 MATHEMATICS AND COMPUTING↗

The Event Horizon General Relativistic Magnetohydrodynamic Code Comparison Project

Recent developments in compact object astrophysics, especially the discovery of merging neutron stars by LIGO, the imaging of the black hole in M87 by the Event Horizon Telescope, and high- precision astrometry of the Galactic Center at close to the event horizon scale by the GRAVITY experiment motivate the development of numerical source models that solve the equations of general relativistic magnetohydrodynamics (GRMHD). In this work we compare GRMHD solutions for the evolution of a magnetized accretion flow where turbulence is promoted by the magnetorotational instability from a set of nine GRMHD codes: Athena++, BHAC, Cosmos++, ECHO, H-AMR, iharm3D, HARM-Noble, IllinoisGRMHD, and KORAL. Agreement among the codes improves as resolution increases, as measured by a consistently applied, specially developed set of code performance metrics. Thus, we decide that the community of GRMHD codes is mature, capable, and consistent on these test problems.

79 ASTRONOMY AND ASTROPHYSICS↗

Strain-Gage Loads Calibration Parametric Study

This paper documents a parametric study of various aircraft wing-load test features that affect the quality of the resultant derived shear, bending-moment, and torque strain-gage load equations. The effect of the following on derived strain-gage equation accuracy are compared: single-point loading compared with distributed loading, variation in applied test load magnitude, number of applied load cases, and wing-box-only compared with control-surface loading. The subject of this study is an extensive wing-load calibration test of the Active Aeroelastic Wing F/A-18 airplane. Selected subsets of the available test data were used to derive load equations using the linear regression method. Results show the benefit of distributed loading and the diminishing-return benefits of test load magnitudes and number of load cases. The use of independent check cases as a quality metric for the derived load equations is shown to overcome blind extrapolating beyond the load data used to derive the load equations.

Lokos, William A.↗

NASA's Ares I and Ares V Launch Vehicles -- Effective Space Operations Through Efficient Ground Operations

The United States (U.S.) plans to return to the Moon by 2020, with the development of a new human-rated space transportation system to replace the Space Shuttle, which is due for retirement in 2010 after it completes its missions of building the International Space Station and servicing the Hubble Space Telescope. Powering the future of space-based scientific exploration will be the Ares I Crew Launch Vehicle, which will transport the Orion Crew Exploration Vehicle to orbit where it will rendezvous with the Lunar Lander. which will be delivered by the Ares V Cargo Launch Vehicle. This new transportation infrastructure, developed by the National Aeronautics and Space Administration (NASA), will allow astronauts to leave low-Earth orbit for extended lunar exploration and preparation for the first footprint on Mars. All space-based operations begin and are controlled from Earth. NASA's philosophy is to deliver safe, reliable, and cost-effective solutions to sustain a multi-billion-dollar program across several decades. Leveraging 50 years of lessons learned, NASA is partnering with private industry, while building on proven hardware experience. This paper will discuss how the Engineering Directorate at NASA's Marshall Space Flight Center is working with the Ares Projects Office to streamline ground operations concepts and reduce costs. Currently, NASA's budget is around $17 billion, which is less than 1 percent of the U.S. Federal budget. Of this amount, NASA invests approximately $4.5 billion each year in Space Shuttle operations, regardless of whether the spacecraft is flying or not. The affordability requirement is for the Ares I to reduce this expense by 50 percent, in order to allow NASA to invest more in space-based scientific operations. Focusing on this metric, the Engineering Directorate provides several solutions-oriented approaches, including Lean/Six Sigma practices and streamlined hardware testing and integration, such as assembling major hardware elements before shipping to the Kennedy Space Center for launch operations. This paper provides top-level details for several cost saving initiatives, including both process and product improvements that will result in space transportation systems that are designed with operations efficiencies in mind. The Engineering Directorate provides both the intellectual capital embodied in an experienced workforce and unique facilities in which to validate the information technology tools that allow a nationwide team to collaboratively connect across miles that separate them and the engineering disciplines that integrate various piece parts into a whole system. As NASA transforms ground-based operations, it also is transitioning its workforce from an era of intense hands-on labor to a new one of mechanized conveniences and robust hardware with simpler interfaces. Ensuring that space exploration is on sound footing requires that operations efficiencies be designed into the transportation system and implemented in the development stage. Applying experience gained through decades of ground and space op'erations, while using value-added processes and modern business and engineering tools, is the philosophy upon which a new era of exploration will be built to solve some of the most pressing exploration challenges today -- namely, safety, reliability, and affordability.

Dumbacher, Daniel L.↗

Tracking Data Certification for the Lunar Reconnaissance Orbiter

This paper details the National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) Flight Dynamics Facility (FDF) tracking data certification effort of the Lunar Reconnaissance Orbiter (LRO) Space Communications Network (SCN) complement of tracking stations consisting of the NASA White Sands 1 antenna (WS1), and the commercial provider Universal Space Network (USN) antennas at South Point, Hawaii; Dongara Australia; Weilheim, Germany; and Kiruna, Sweden. Certification assessment required the cooperation and coordination of parties not under the control of either the LRO project or ground stations as uplinks on cooperating spacecraft were necessary. The LRO range-tracking requirement of 10m 1 sigma could be satisfactorily demonstrated using any typical spacecraft capable of range tracking. Though typical Low Earth Orbiting (LEO) or Geosynchronous Earth Orbiting (GEO) spacecraft may be adequate for range certification, their measurement dynamics and noise would be unacceptable for proper Doppler certification of 1-3mm/sec 1 sigma. As LRO will orbit the Moon, it was imperative that a suitable target spacecraft be utilized which can closely mimic the expected lunar orbital Doppler dynamics of +/-1.6km/sec and +/-1.5m/sq sec to +/-0.15m/sq sec, is in view of the ground stations, supports coherent S-Band Doppler tracking measurements, and can be modeled by the FDF. In order to meet the LRO metric tracking data specifications, the SCN ground stations employed previously uncertified numerically controlled tracking receivers. Initial certification testing revealed certain characteristics of the units that required resolution before being granted certification.

Morinelli, Patrick J.↗

Evaluation of Gear Condition Indicator Performance on Rotorcraft Fleet

The U.S. Army is currently expanding its fleet of Health Usage Monitoring Systems (HUMS) equipped aircraft at significant rates, to now include over 1,000 rotorcraft. Two different on-board HUMS, the Honeywell Modern Signal Processing Unit (MSPU) and the Goodrich Integrated Vehicle Health Management System (IVHMS), are collecting vibration health data on aircraft that include the Apache, Blackhawk, Chinook, and Kiowa Warrior. The objective of this paper is to recommend the most effective gear condition indicators for fleet use based on both a theoretical foundation and field data. Gear diagnostics with better performance will be recommended based on both a theoretical foundation and results of in-fleet use. In order to evaluate the gear condition indicator performance on rotorcraft fleets, results of more than five years of health monitoring for gear faults in the entire HUMS equipped Army helicopter fleet will be presented. More than ten examples of gear faults indicated by the gear CI have been compiled and each reviewed for accuracy. False alarms indications will also be discussed. Performance data from test rigs and seeded fault tests will also be presented. The results of the fleet analysis will be discussed, and a performance metric assigned to each of the competing algorithms. Gear fault diagnostic algorithms that are compliant with ADS-79A will be recommended for future use and development. The performance of gear algorithms used in the commercial units and the effectiveness of the gear CI as a fault identifier will be assessed using the criteria outlined in the standards in ADS-79A-HDBK, an Army handbook that outlines the conversion from Reliability Centered Maintenance to the On-Condition status of Condition Based Maintenance.

Antolick, Lance J.↗

CCSDS Advanced Orbiting Systems Virtual Channel Access Service for QoS MACHETE Model

To support various communications requirements imposed by different missions, interplanetary communication protocols need to be designed, validated, and evaluated carefully. Multimission Advanced Communications Hybrid Environment for Test and Evaluation (MACHETE), described in "Simulator of Space Communication Networks" (NPO-41373), NASA Tech Briefs, Vol. 29, No. 8 (August 2005), p. 44, combines various tools for simulation and performance analysis of space networks. The MACHETE environment supports orbital analysis, link budget analysis, communications network simulations, and hardware-in-the-loop testing. By building abstract behavioral models of network protocols, one can validate performance after identifying the appropriate metrics of interest. The innovators have extended the MACHETE model library to include a generic link-layer Virtual Channel (VC) model supporting quality-of-service (QoS) controls based on IP streams. The main purpose of this generic Virtual Channel model addition was to interface fine-grain flow-based QoS (quality of service) between the network and MAC layers of the QualNet simulator, a commercial component of MACHETE. This software model adds the capability of mapping IP streams, based on header fields, to virtual channel numbers, allowing extended QoS handling at link layer. This feature further refines the QoS v existing at the network layer. QoS at the network layer (e.g. diffserv) supports few QoS classes, so data from one class will be aggregated together; differentiating between flows internal to a class/priority is not supported. By adding QoS classification capability between network and MAC layers through VC, one maps multiple VCs onto the same physical link. Users then specify different VC weights, and different queuing and scheduling policies at the link layer. This VC model supports system performance analysis of various virtual channel link-layer QoS queuing schemes independent of the network-layer QoS systems.

Jennings, Esther H.↗

Techniques for Liquid Rocket Combustion Spontaneous Stability and Rough Combustion Assessments

This work presents techniques for liquid rocket engine combustion stability assessments with respect to spontaneous stability and rough combustion. Techniques covering empirical parameter extraction, which were established in prior works, are applied for three additional programs: the F-1 Gas Generator (F1GG) component test program, the RS-84 preburner component test program, and the Marshall Integrated Test Rig (MITR) program. Stability assessment parameters from these programs are compared against prior established spontaneous stability metrics and updates are identified. Also, a procedure for comparing measured with predicted mode shapes is presented, based on an extension of the Modal Assurance Criterion (MAC).

Kenny, R. J.↗

NASA's Space Launch System: Enabling Exploration and Discovery

As NASA's new Space Launch System (SLS) launch vehicle continues to mature toward its first flight and beyond, so too do the agency's plans for utilization of the rocket. Substantial progress has been made toward the production of the vehicle for the first flight of SLS - an initial "Block 1" configuration capable of delivering more than 70 metric tons (t) to Low Earth Orbit (LEO). That vehicle will be used for an uncrewed integrated test flight, propelling NASA's Orion spacecraft into lunar orbit before it returns safely to Earth. Flight hardware for that launch is being manufactured at facilities around the United States, and, in the case of Orion's service module, beyond. At the same time, production has already begun on the vehicle for the second SLS flight, a more powerful Block 1B configuration capable of delivering more than 105 t to LEO. This configuration will be used for crewed launches of Orion, sending astronauts farther into space than anyone has previously ventured. The 1B configuration will introduce an Exploration Upper Stage, capable of both ascent and in-space propulsion, as well as a Universal Stage Adapter - a payload bay allowing the flight of exploration hardware with Orion - and unprecedentedly large payload fairings that will enable currently impossible spacecraft and mission profiles on uncrewed launches. The Block 1B vehicle will also expand on the initial configuration's ability to deploy CubeSat secondary payloads, creating new opportunities for low-cost access to deep space. Development work is also underway on future upgrades to SLS, which will culminate in about a decade in the Block 2 configuration, capable of delivering 130 t to LEO via the addition of advanced boosters. As the first SLS draws closer to launch, NASA continues to refine plans for the human deep-space exploration it will enable. Planning currently focuses on use of the vehicle to assemble a Deep Space Gateway, which would comprise a habitat in the lunar vicinity allowing astronauts to gain experience living and working in deep space, a testbed for new systems and capabilities needed for exploration beyond, and a departure point for NASA and partners to send missions to other destinations. Assembly of the Gateway would be followed by a Deep Space Transport, which would be a vehicle capable of carrying astronauts farther into our solar system and eventually to Mars. This paper will give an overview of SLS' current status and its capabilities, and discuss current utilization planning.

Schorr, Andrew↗

Objective Measurement Assessment of Departure Advisories for Ramp Controllers from a Human-In-The-Loop Simulation

NASA has developed and demonstrated an integrated arrival, departure, and surface concept and technology for efficient air traffic management in busy terminal environments under the Airspace Technology Demonstration 2 (ATD-2) subproject. In April/May 2019, NASA conducted a human-in-the-loop (HITL) simulation with ramp and tower controllers having experience at Dallas/Fort Worth International Airport (DFW). The purpose of this HITL was to evaluate the impacts of various surface metering goals in ramp operations at DFW and test new features of the ATD-2 ramp controller decision support tools. This paper evaluates the quantitative metrics from the simulation results related to airport performance and surface metering given various departure scheduling advisories for ramp controllers. The objective measurements compared include the controller’s compliance with the target times for pushback and spot arrival, the number of metered flights, aircraft gate hold and taxi time, runway throughput, and the number of aircraft on the surface. The simulation results show that there were no statistically significant benefits from surface metering at DFW for airport performance due to several simulation artifacts notably different from real operations. Considering controller workload and situation awareness, following either gate pushback or spot arrival advisory could be a better metering option than utilizing both advisories when surface metering is applied at DFW.

airport ramp traffic control decision support↗

Mars Commercial-Compatible Initial Human-Precursor Surveying and Instrument-Deployment Concepts

Water is a critical resource need for future human exploration and habitation of Mars and other planetary environments. An integral scientific question in the search for life outside of Earth is: What are the constraints on life in planetary environments? One of the most critical features of habitable environments is the availability of water - both as an environment for prebiotic chemistry, abiogenesis, and life; and as a vehicle for generating geo- and electro- chemical systems that can support metabolism. Seeking evidence of life prior to human arrival is important both to preserve signs of any existing or past evidence of Martian life, reducing the risks of unintentional forward contamination of Martian environments, and to potentially protect the health of any human crews. Looking forward towards Mars, future human exploration will require a rapid, but thorough, initial survey of any local “area of operations” on Mars to determine the spatial distribution, quantity, and quality (e.g., salt/mineral/organic content) of any local water, as well as to evaluate any signs of current or extinct life. We will discuss several candidate field survey methodologies, involving humans, a teamed instrument platform and a surveying/sampling rover. These are proposed to be validated in a given terrestrial Mars analog environment exhibiting varying distributions of water (ice, permafrost, rock-bound) to evaluate the constraints on life. Commercial space exploration brings the future possibility of unprecedented mass, power and in situ instrument capability, potentially enabling thorough exploration of the geology, hydrology and astrobiology of a km-radius study-zone around a future Mars human/commercial space landing site. We propose to study several representative planetary surface precursor-survey architectures and mission concepts (that are commercial-Mars-compatible), and to test and compare them in field application. We will examine some of the performance issues, using given metrics and controls, of spacesuited humans using handheld and standalone instruments, vs. a robotic human-tended rover/drill, robot arm and instruments. The latter rover-based architecture can in turn be viewed in terms of returned-to-base sample in-situ analyses vs sample analysis performed on a standalone instrument platform deployed away from the primary area of operations.

astrobiology↗

Initial Assessment of Lost Command and Control Link Procedures

This paper presents an initial assessment of lost command and control (LC2L) procedures for large Uncrewed Aircraft Systems (UAS) in a simulated representative airspace environment using real airspace procedures. The experiment matrix consists of nine different flight routes: four nominal, four following current LC2L procedures, and one that routes the UAS more conservatively through less-busy airspace. For each route, eleven simulated UAS flights – in ten-minute increments – were flown into Fort Worth Alliance Airport following a real Instrument Approach Procedure. The simulated UAS flew amongst real recorded tracks of approximately 4,700 flights on January 18, 2022. The analysis focused primarily on the number of aircraft with which each UAS lost separation and where the losses occurred. This work presents a significant increase in testing capability and provides the foundation for further verification and validation of LC2L procedures using additional analysis metrics.

Uncrewed aircraft↗

Initial Assessment of Lost Command and Control Link Procedures

This paper presents an initial assessment of lost command and control (LC2L) procedures for large Uncrewed Aircraft Systems (UAS) in a simulated representative airspace environment using real airspace procedures. The experiment matrix consists of nine different flight routes: four nominal, four following current LC2L procedures, and one that routes the UAS more conservatively through less-busy airspace. For each route, eleven simulated UAS flights – in ten-minute increments – were flown into Fort Worth Alliance Airport following a real Instrument Approach Procedure. The simulated UAS flew amongst real recorded tracks of approximately 4,700 flights on January 18, 2022. The analysis focused primarily on the number of aircraft with which each UAS lost separation and where the losses occurred. This work presents a significant increase in testing capability and provides the foundation for further verification and validation of LC2L procedures using additional analysis metrics.

uncrewed aircraft↗

Leveraging Paraphrase Labels to Extract Synonyms from Twitter

We present an approach for automatically learning synonyms from a paraphrase corpus of tweets. This work shows improvement on the task of paraphrase detection when we substitute our extracted synonyms into the training set. The synonyms are learned by using chunks from a shallow parse to create candidate synonyms and their context windows, and the synonyms are incorporated into a paraphrase detection system that uses machine translation metrics as features for a classifier. We demonstrate a 2.29% improvement in F1 when we train and test on the paraphrase training set, providing better coverage than previous systems, which shows the potential power of synonyms that are representative of a specific topic.

Antoniak, Maria A.↗

Sandia PUF Analysis Tool

This program is a graphical user interface for measuring and performing inter-active analysis of physical unclonable functions (PUFs). It is intended for demonstration and education purposes. See license.txt for license details. The program features a PUF visualization that demonstrates how signatures differ between PUFs and how they exhibit noise over repeated measurements. A similarity scoreboard shows the user how close the current measurement is to the closest chip signatures in the database. Other metrics such as average noise and inter-chip Hamming distances are presented to the user. Randomness tests published in NIST SP 800-22 can be computed and displayed. Noise and inter-chip histograms for the sample of PUFs and repeated PUF measurements can be drawn.

Source record↗

Spent Nuclear Fuel Exploratory Roadmaps - 20054

The U.S. Department of Energy manages nearly 2,500 metric tons of heavy metal (MTHM) of spent nuclear fuel (SNF) resulting from several decades of research, testing, and production reactors [DOE 1995]. This SNF is managed at the DOE Hanford Site in Washington State (Hanford), Idaho National Laboratory (INL), and Savannah River Site in South Carolina (SRS). From 1995 to 2004, the Department of Energy (DOE) made several key programmatic decisions, supported by appropriate documentation in accordance with the National Environmental Policy Act (NEPA) for the management of SNF [DOE 1995]. These decisions have provided an overarching framework for SNF management, as well as site-specific and SNF-related management decisions for the past two decades. In the years since these decisions were made, with the notable exception of the successful drying and packaging of the production reactor fuel at Hanford, a majority of the decisions have been largely unimplemented. Also, since these decisions were made, a number of changes that bear on considerations relative to the path forward have occurred. A SNF Exploratory Road map activity, identified reasonable alternate pathways for DoE's inventory of SNF. Three fundamental pathways were identified in this activity for the long-term management and disposition of DOE SNF: Direct Disposal Pathway, Existing Processing Pathway and Alternate Processing Pathway. Further actions would be required (e.g., evaluation of NEPA analysis, technology maturity evaluations, refinement of cost and risks, benefits and advantages relative to other alternate paths) to define and inform many key decisions that will result in the selection of the disposition pathways that are most beneficial to the US Government in dispositioning DoE's SNF inventory. DOE is currently managing all SNF safely; however, the age of DOE-owned SNF and facilities for storing and processing SNF, coupled with the uncertainty of the storage duration, necessitates decisions and actions to ensure that the infrastructure will be in place to ensure continued safe and effective long-term management and eventual disposal. The identification of reasonable alternate pathways took these considerations into account in an effort to proactively manage conditions that could challenge the safety of storing and managing DOE SNF over the time periods now contemplated and also to ensure that flexibility is preserved to ensure that the DOE SNF remains compatible with final disposition pathways when they become available. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Preliminary Analysis of Facility Design for Pyro-processing Safeguardability

The deployment of advanced nuclear energy systems can assure sustainable energy use worldwide. For some of these systems, nuclear fuel will be fabricated in remotely-handled facilities in which batch-type processing will occur in heavily-shielded hot cells. Because special nuclear material is in both different physical and chemical form than that of the contemporary fuel cycle, new challenges have arisen in terms of safeguards, safety, and security, integrated with facility design; i.e., safeguardability. There is a need to develop models that can quantitatively assess safeguardability from a design-driven perspective. Previously, the first build of a material throughput model was established using discrete event simulation (DES) for the fuel fabrication process. Results showed that the DES model potentially can detect diversion attempts. This paper continues the study by further developing the the model. We test the use of the probability of the false alarm; i.e., the Type I error, as a metric for safeguardability assessment. Initiating events leading to a false alarm arise from aspects of safeguards, safety, and security. Different conceptual facility designs could exhibit different frequencies for the Type I error, and this could provide a measure of the safeguardability of each design. A quantitative assessment for safeguardable facility designs has not been developed in this manner. Much of the research into safeguardability has been qualitative. Engineering design approaches for a safeguardable facility are now needed. As there are not accepted metrics for safeguardability assessment, our motivation is to develop this design facet of safeguardability further.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

TEAMER: Twin Ocean Power Wave Energy Converter Comprehensive Overview

These files collectively provide a comprehensive overview of the testing process, data analysis, and validation for the Twin Ocean Power device tested at the O.H. Hinsdale Wave Research Laboratory, supported by TEAMER funding. This resource includes an overview of power results for a series of 7 trials. The files included in this comprehensive overview include a comprehensive log sheet for each trial, a summary of all trials, and processing scripts for the raw data. It includes all raw data in .tsv and MATLAB compatible formats, an average power chart, angular velocity charts for each trial, trial metrics, and power output files. This resource includes images of the Twin Ocean Power Wave Energy Converter device components and movement during testing and video recordings of each trial.

16 TIDAL AND WAVE POWER↗