Search NASA⌕ Search

SEARCH · Search NASA

Results for “test frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2003

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

We report on a study to determine the maturity of different verification and validation technologies (V&V) on a representative example of NASA flight software. The study consisted of a controlled experiment where three technologies (static analysis, runtime analysis and model checking) were compared to traditional testing with respect to their ability to find seeded errors in a prototype Mars Rover. What makes this study unique is that it is the first (to the best of our knowledge) to do a controlled experiment to compare formal methods based tools to testing on a realistic industrial-size example where the emphasis was on collecting as much data on the performance of the tools and the participants as possible. The paper includes a description of the Rover code that was analyzed, the tools used as well as a detailed description of the experimental setup and the results. Due to the complexity of setting up the experiment, our results can not be generalized, but we believe it can still serve as a valuable point of reference for future studies of this kind. It did confirm the belief we had that advanced tools can outperform testing when trying to locate concurrency errors. Furthermore the results of the experiment inspired a novel framework for testing the next generation of the Rover.

Brat, Guillaume↗
NASA NTRS2016

Safeguarding the Health of the NASA Astronaut Community: the Need for Expanded Medical Monitoring for Former NASA Astronauts Under the Astronaut Occupational Health Program

The astronaut community is unique, and may be disproportionately exposed to occupational hazards not commonly seen in other communities. The extent to which the demands of the astronaut occupation and exposure to spaceflight-related hazards affect the health of the astronaut population over the life course is not completely known. Provision of health screening services to active and former astronauts ensures individual, mission, and community health and safety. Currently, the NASA Johnson Space Center (JSC) Flight Medicine Clinic (FMC) provides extensive medical monitoring to active astronauts throughout their careers. Upon retirement, astronauts may voluntarily return to the JSC FMC for an annual preventive exam. However, current retiree monitoring includes only selected screening tests, representing an opportunity for augmentation. The potential latent health effects of spaceflight demand an expanded framework of testing for former astronauts. The need is two-fold: screening tests widely recommended for other aging communities are necessary for astronauts to rule out conditions resulting from the natural aging process (e.g., colonoscopy, mammography), as opposed to conditions resulting directly from the astronaut occupation; and increased breadth of monitoring services will improve the understanding of occupational health risks and longitudinal health of the astronaut community, past, present, and future. To meet this need, NASA has begun an extensive exploration of the overall approach, cost, and policy implications of expanding existing medical monitoring under the Astronaut Occupational Health program for former NASA astronauts.

Rossi, Meredith↗
NASA NTRS2017

Protecting the Health of Astronauts: Enhancing Occupational Health Monitoring and Surveillance for Former NASA Astronauts to Understand Long-Term Outcomes of Spaceflight-Related Exposures

The astronaut community is unique, and may be disproportionately exposed to occupational hazards not commonly seen in other communities. The extent to which the demands of the astronaut occupation and exposure to spaceflight-related hazards affect the health of the astronaut population over the life course is not completely known. A better understanding of the individual, population, and mission impacts of astronaut occupational exposures is critical to providing clinical care, targeting occupational surveillance efforts, and planning for future space exploration. The ability to characterize the risk of latent health conditions is a significant component of this understanding. Provision of health screening services to active and former astronauts ensures individual, mission, and community health and safety. Currently, the NASA-Johnson Space Center (JSC) Flight Medicine Clinic (FMC) provides extensive medical monitoring to active astronauts throughout their careers. Upon retirement, astronauts may voluntarily return to the JSC FMC for an annual preventive exam. However, current retiree monitoring includes only selected screening tests, representing an opportunity for augmentation. The potential long-term health effects of spaceflight demand an expanded framework of testing for former astronauts. The need is two-fold: screening tests widely recommended for other aging populations are necessary to rule out conditions resulting from the natural aging process (e.g., colonoscopy, mammography); and expanded monitoring will increase NASA's ability to better characterize conditions resulting from astronaut occupational exposures. To meet this need, NASA has begun an extensive exploration of the overall approach, cost, and policy implications of e an Astronaut Occupational Health program to include expanded medical monitoring of former NASA astronauts. Increasing the breadth of monitoring services will ultimately enrich the existing evidence base of occupational health risks to astronauts. Such an expansion would therefore improve the understanding of the health of the astronaut population as a whole, and the ability to identify, mitigate, and manage such risks in preparation for deep space exploration missions.

Rossi, Meredith↗
NASA NTRS2001

Using XML and Java Technologies for Astronomical Instrument Control

Traditionally, instrument command and control systems have been highly specialized, consisting mostly of custom code that is difficult to develop, maintain, and extend. Such solutions are initially very costly and are inflexible to subsequent engineering change requests, increasing software maintenance costs. Instrument description is too tightly coupled with details of implementation. NASA Goddard Space Flight Center, under the Instrument Remote Control (IRC) project, is developing a general and highly extensible framework that applies to any kind of instrument that can be controlled by a computer. The software architecture combines the platform independent processing capabilities of Java with the power of the Extensible Markup Language (XML), a human readable and machine understandable way to describe structured data. A key aspect of the object-oriented architecture is that the software is driven by an instrument description, written using the Instrument Markup Language (IML), a dialect of XML. IML is used to describe the command sets and command formats of the instrument, communication mechanisms, format of the data coming from the instrument, and characteristics of the graphical user interface to control and monitor the instrument. The IRC framework allows the users to define a data analysis pipeline which converts data coming out of the instrument. The data can be used in visualizations in order for the user to assess the data in real-time, if necessary. The data analysis pipeline algorithms can be supplied by the user in a variety of forms or programming languages. Although the current integration effort is targeted for the High-resolution Airborne Wideband Camera (HAWC) and the Submillimeter and Far Infrared Experiment (SAFIRE), first-light instruments of the Stratospheric Observatory for Infrared Astronomy (SOFIA), the framework is designed to be generic and extensible so that it can be applied to any instrument. Plans are underway to test the framework with other types of instruments, such as remote sensing earth science instruments.

Ames, Troy↗
NASA NTRS1972

The theoretical tools of experimental gravitation

Theoretical frameworks for testing relativistic gravity are presented in terms of a system for analyzing theories of gravity invented as alternatives to Einstein. The parametrized post-Newtonian (PPN) formalism, based on the Dicke framework and the Eotvos-Dicke-Braginsky experiment, is discussed in detail. The metric theories of gravity, and their post-Newtonian limits are reviewed, and PPN equations of motion are derived. These equations are used to analyze specific effects and experimental tests in the solar system.

Will, C. M.↗
NASA NTRS2022

Reservoir Assessment Tool 2.0: Stakeholder driven improvements to satellite remote sensing based reservoir monitoring

In light of the rapidly increasing regulation of rivers due to planned and constructed reservoirs, monitoring reservoir operations has become very crucial. The Reservoir Assessment Tool (RAT) framework was developed to monitor reservoir operations globally, using hydrological modeling and satellite observations. With feedback from stakeholders, improvements in the RAT framework are demonstrated in this study using the Mekong River Basin as an example. A novel multi-sensor reservoir area mapping technique was developed using complementary strengths of optical and SAR sensors at a 1-5 day temporal resolution, allowing the quantification of sub-weekly reservoir operations. Additionally, the skill of radar altimeters in the RAT framework was tested using the Jason-3 altimeter. Using in-situ data from three Thai reservoirs in the Mekong Basin, consistent improvements were observed as compared to the original RAT framework. A functionality for visualizing forecasted outflow was also added using historically inferred reservoir operations or stakeholder-driven target storage expectations.

Reservoirs↗
NASA NTRS2016

A Framework Based on 2-D Taylor Expansion for Quantifying the Impacts of Subpixel Reflectance Variance and Covariance on Cloud Optical Thickness and Effective Radius Retrievals Based on the Bispectral Method

The bispectral method retrieves cloud optical thickness (τ) and cloud droplet effective radius (re) simultaneously from a pair of cloud reflectance observations, one in a visible or near-infrared (VIS/NIR) band and the other in a shortwave infrared (SWIR) band. A cloudy pixel is usually assumed to be horizontally homogeneous in the retrieval. Ignoring subpixel variations of cloud reflectances can lead to a significant bias in the retrieved τ and re. In the literature, the retrievals of τ and re are often assumed to be independent and considered separately when investigating the impact of subpixel cloud reflectance variations on the bispectral method. As a result, the impact on τ is contributed only by the subpixel variation of VIS/NIR band reflectance and the impact on re only by the subpixel variation of SWIR band reflectance. In our new framework, we use the Taylor expansion of a two-variable function to understand and quantify the impacts of subpixel variances of VIS/NIR and SWIR cloud reflectances and their covariance on the τ and re retrievals. This framework takes into account the fact that the retrievals are determined by both VIS/NIR and SWIR band observations in a mutually dependent way. In comparison with previous studies, it provides a more comprehensive understanding of how subpixel cloud reflectance variations impact the τ and re retrievals based on the bispectral method. In particular, our framework provides a mathematical explanation of how the subpixel variation in VIS/NIR band influences the re retrieval and why it can sometimes outweigh the influence of variations in the SWIR band and dominate the error in re retrievals, leading to a potential contribution of positive bias to the re retrieval. We test our framework using synthetic cloud fields from a large-eddy simulation and real observations from Moderate Resolution Imaging Spectroradiometer. The predicted results based on our framework agree very well with the numerical simulations. Our framework can be used to estimate the retrieval uncertainty from subpixel reflectance variations in operational satellite cloud products and to help understand the differences in τ and re retrievals between two instruments.

reflectance↗
NASA NTRS2017

Development of a High-Fidelity Simulation Environment for Shadow-Mode Assessments of Air Traffic Concepts

This paper describes the Shadow-Mode Assessment Using Realistic Technologies for the National Airspace System (SMART-NAS) Test Bed. The SMART-NAS Test Bed is an air traffic simulation platform being developed by the National Aeronautics and Space Administration (NASA). The SMART-NAS Test Bed's core purpose is to conduct high-fidelity, real-time, human-in-the-loop and automation-in-the-loop simulations of current and proposed future air traffic concepts for the United States' Next Generation Air Transportation System called NextGen. The setup, configuration, coordination, and execution of realtime, human-in-the-loop air traffic management simulations are complex, tedious, time intensive, and expensive. The SMART-NAS Test Bed framework is an alternative to the current approach and will provide services throughout the simulation workflow pipeline to help alleviate these shortcomings. The principle concepts to be simulated include advanced gate-to-gate, trajectory-based operations, widespread integration of novel aircraft such as unmanned vehicles, and real-time safety assurance technologies to enable autonomous operations. To make this possible, SNTB will utilize Web-based technologies, cloud resources, and real-time, scalable, communication middleware. This paper describes the SMART-NAS Test Bed's vision, purpose, its concept of use, and the potential benefits, key capabilities, high-level requirements, architecture, software design, and usage.

human-in-the-loop↗
NASA NTRSDate not supplied

Validation of Image-Based Neural Network Controllersthrough Adaptive Stress Testing

Neural networks have become state-of-the-art for computer vision problems because of their ability to efficiently model complex functions from large amounts of data. While neural networks can be shown to perform well empirically fora variety of tasks, their performance is difficult to guarantee.Neural network verification tools have been developed that can certify robustness with respect to a given input image; however,for neural network systems used in closed-loop controllers,robustness with respect to individual images does not address multi-step properties of the neural network controller and itsenvironment. Furthermore, neural network systems interacting in the physical world and using natural images are operating in a black-box environment, making formal verification in-tractable. This work combines the adaptive stress testing (AST)framework with neural network verification tools to search for the most likely sequence of image disturbances that cause the neural network controlled system to reach a failure. Anautonomous aircraft taxi application is presented, and results show that the AST method finds failures with more likely image disturbances than baseline methods. Further analysis of AST results revealed an explainable cause of the failure, giving insight into the problematic scenarios that should be addressed.

Adaptive Stress Testing, Marabou, Deep Neural Netw↗
NASA NTRS1983

Rastall's and related theories are conservative gravitational theories although physically inequivalent to general relativity

The proper framework for testing Rastall's theory and its generalizations is in the case of non-negligible (i.e. discernible) gravitational effects such as gravity gradients. These theories have conserved integral four-momentum and angular momentum. The Nordtvedt effect then provides limits on the parameters which arise as the result of the non-zero divergence of the energy-momentum tensor.

Smalley, L. L.↗
NASA NTRS1992

Atmospheric effects on ejecta emplacement and crater formation on Venus from Magellan

Surface signatures of energy partitioning are used as a framework for testing extrapolations from laboratory experiments and other planetary settings to assess the effects of both gravity and an atmosphere on impact crater formation on Venus. The dense lower atmosphere of Venus is found to assume the role of a low-density target for bodies smaller than about 4 km in diameter. Air blasts created by cratering in the atmosphere produce distinctive surface signatures that allow the derivation of an independent assessment of impactor energy at the limit of break up. Dynamic pressures during entry of larger bodies will exceed their strength limit but may not prevent penetration of the atmosphere due to aerodynamic reshaping that minimizes the drag coefficient. This process may account for the formation of unusually small craters (1-3 km). The dense atmosphere of Venus preserves signatures of early time cratering processes on the surface that are typically lost on atmosphere-free surfaces.

Schultz, Peter H.↗
NASA NTRS2003

Earth's earliest biosphere-a proposal to develop a collection of curated archean geologic reference materials

The discovery of evidence indicative of life in a Martian meteorite has led to an increase in interest in astrobiology. As a result of this discovery, and the ensuing controversy, it has become apparent that our knowledge of the early development of life on Earth is limited. Archean stratigraphic successions containing evidence of Earth's early biosphere are well preserved in the Pilbara Craton of Western Australia. The craton includes part of a protocontinent consisting of granitoid complexes that were emplaced into, and overlain by, a 3.51-2.94 Ga volcanigenic carapace - the Pilbara Supergroup. The craton is overlain by younger supracrustal basins that form a time series recording Earth history from approximately 2.8 Ga to approximately 1.9 Ga. It is proposed that a well-documented suite of these ancient rocks be collected as reference material for Archean and astrobiological research. All samples would be collected in a well-defined geological context in order to build a framework to test models for the early evolution of life on Earth and to develop protocols for the search for life on other planets.

Archaea↗
NASA NTRS2008

Geological Implications of a Physical Libration on Enceladus

Given the non-spherical shape of Enceladus (Thomas et al., 2007), the satellite will experience gravitational torques that will cause it to physically librate as it orbits Saturn. Physical libration would produce a diurnal oscillation in the longitude of Enceladus tidal bulge which, could have a profound effect on the diurnal stresses experienced by the surface of the satellite. Although Cassini ISS has placed an observational upper limit on Enceladus libration amplitude of F < 1.5deg (Porco et al., 2006), smaller amplitudes can still have geologically significant consequences. Here we present the first detailed description of how physical libration affects tidal stresses and how those stresses then might affect geological processes including crack formation and propagation, south polar eruption activity, and tidal heating. Our goal is to provide a framework for testing the hypothesis that geologic features on Enceladus are produced by tidal stresses from diurnal physical and optical librations of the satellite.

Hurford, T. A.↗
NASA NTRS2011

An Overview of the Total Lightning Jump Algorithm: Past, Present and Future Work

Rapid increases in total lightning prior to the onset of severe and hazardous weather have been observed for several decades. These rapid increases are known as lightning jumps and can precede the occurrence of severe weather by tens of minutes. Over the past decade, a significant effort has been made to quantify lightning jump behavior in relation to its utility as a predictor of severe and hazardous weather. Based on a study of 34 thunderstorms that occurred in the Tennessee Valley, early work conducted in our group at Huntsville determined that it was indeed possible to create a reasonable operational lightning jump algorithm (LJA) based on a statistical framework relying on the variance behavior of the lightning trending signal. We the expanded this framework and tested several variance-related LJA configurations on a much larger sample of 87 severe and non severe thunderstorms. This study determined that a configuration named the "2(sigma)" algorithm had the most promise in development of the operational LJA with a probability of detection (POD) of 87%, a false alarm rate (FAR) of 33%, a Heidke Skill Score (HSS) of 0.75. The 2(sigma) algorithm was then tested on an even larger sample of 711 thunderstorms of all types from four regions of the country where total lightning measurement capability existed. The result was very encouraging.Despite the larger number of storms and the inclusion of different regions of the country, the POD remained high (79%), the FAR was low (36%) and HSS was solid (0.71). Average lead time from jump to severe weather occurrence was 20.65 minutes, with a standard deviation of +/- 15 minutes. Also, trends in total lightning were compared to cloud to ground (CG) lightning trends, and it was determined that total lightning trends had a higher POD (79% vs 66%), lower FAR (36% vs 54 %) and a better HSS (0.71 vs 0.55). From the 711-storm case study it was determined that a majority of missed events were due to severe weather producing thunderstorms in low flashing environments. The latest efforts have been geared toward examining these low flashing storms in order to adjust the algorithm for such storms, thus enhancing the capability of the LJA. Future work will test the algorithm in real time using current satellite and radar based cell tracking methods, as well as, comparing total lightning jump occurrence to both satellite based and ground base observations of thunderstorms to create correlations between lightning jumps and the observed structures within thunderstorms. Finally this algorithm will need to be tested using Geostationary Lightning Mapper proxy data to transition the algorithm from VHF ground based lightning measurements to lower frequency space-based lightning measurements.

Schultz, Christopher J.↗
NASA NTRS2013

SILHIL Replication of Electric Aircraft Powertrain Dynamics and Inner-Loop Control for V&V of System Health Management Routines

Software-in-the-loop and Hardware-in-the-loop testing of failure prognostics and decision making tools for aircraft systems will facilitate more comprehensive and cost-effective testing than what is practical to conduct with flight tests. A framework is described for the offline recreation of dynamic loads on simulated or physical aircraft powertrain components based on a real-time simulation of airframe dynamics running on a flight simulator, an inner-loop flight control policy executed by either an autopilot routine or a human pilot, and a supervisory fault management control policy. The creation of an offline framework for verifying and validating supervisory failure prognostics and decision making routines is described for the example of battery charge depletion failure scenarios onboard a prototype electric unmanned aerial vehicle.

Verification and Validation↗
NASA NTRS2014

Antarctic Analog for Dilational Bands on Europa

Europa's surface shows signs of extension, which is revealed as lithospheric dilation expressed along ridges, dilational bands and ridged bands. Ridges, the most common tectonic feature on Europa, comprise a central crack flanked by two raised banks a few hundred meters high on each side. Together these three classes may represent a continuum of formation. In Tufts' Dilational Model ridge formation is dominated by daily tidal cycling of a crack, which can be superimposed with regional secular dilation. The two sources of dilation can combine to form the various band morphologies observed. New GPS data along a rift on the Ross Ice Shelf, Antarctica is a suitable Earth analog to test the framework of Tufts' Dilational Model. As predicted by Tufts' Dilational Model, tensile failures in the Ross Ice Shelf exhibit secular dilation, upon which a tidal signal can be seen. From this analog we conclude that Tufts' Dilational Model for Europan ridges and bands may be credible and that the secular dilation is most likely from a regional source and not tidally driven.

Tectonics↗
NASA NTRS2017

All-Sky Microwave Imager Data Assimilation at NASA GMAO

Efforts in all-sky satellite data assimilation at the Global Modeling and Assimilation Office (GMAO) at NASA Goddard Space Flight Center have been focused on the development of GSI configurations to assimilate all-sky data from microwave imagers such as the GPM Microwave Imager (GMI) and Global Change Observation Mission-Water (GCOM-W) Advanced Microwave Scanning Radiometer 2 (AMSR-2). Electromagnetic characteristics associated with their wavelengths allow microwave imager data to be relatively transparent to atmospheric gases and thin ice clouds, and highly sensitive to precipitation. Therefore, GMAOs all-sky data assimilation efforts are primarily focused on utilizing these data in precipitating regions. The all-sky framework being tested at GMAO employs the GSI in a hybrid 4D-EnVar configuration of the Goddard Earth Observing System (GEOS) data assimilation system, which will be included in the next formal update of GEOS. This article provides an overview of the development of all-sky radiance assimilation in GEOS, including some performance metrics. In addition, various projects underway at GMAO designed to enhance the all-sky implementation will be introduced.

NWP↗
NASA NTRSDate not supplied

Assessing the Viability of Using GEOS-Forecast Product for Landslides Forecasting: A Step Toward Early Warning System

Landslides across the globe are mostly triggered by extreme rainfall events affecting infrastructure, transportation and livelihoods. The risks are rarely quantified due to lack of data, analytical skills and limited modeling techniques. Knowledge of local to global scale landslide risks provides communities and national agencies the ability to adapt disaster management practices to mitigate and recover from these hazards. In order to minimize the risks and improve characterization of community resilience to landslides, it is vital to have reliable information about the factors triggering landslides such as rainfall, well ahead in time. Forecasting potential landslide activity and impacts can be achieved through reliable precipitation forecast models. However, it is challenging because of the temporal and spatial variability of precipitation, an important factor in triggering landslides. Evaluation of the precipitation field, associated errors, and sampling uncertainties is integral for development of efficient and reliable landslide forecasting and early warning system. This study develops a methodology to assess the viability of using a precipitation field provided by a global model and its potential integration in the landslide forecasting system. The study focuses on the comparison between the IMERG (Integrated Multi-satellitE Retrievals for Global Precipitation Mission) and GEOS (NASA Goddard Earth Observing System)-Forecast product over contiguous United States (CONUS). GEOS model assimilates new observations every 6 hours, at 00, 06, 12, and 18 UTC. The framework is tested on the GEOS-Forecast Model initialized at 00 UTC using daily IMERG early product as reference using both categorical and continuous statistics. The categorical statistics includes the probability of detection (POD), success ratio (SR), critical success index (CSI), and the hit bias. Continuous statistics such as correlation, normalized standard deviation, and root-mean-square error are also evaluated. Overall, GEOS-Forecast precipitation field over the analysis period (~1 year) show underestimation with respect to IMERG early for the daily accumulated rainfall. However, the probability distribution function and cumulative distribution function of both show similar patterns. In terms of correlations, POD, SR, CSI, hit bias, the performance varies with respect to the rainfall threshold used.

Sana Khan↗