Search NASA⌕ Search

SEARCH · Search NASA

Results for “Evaluation Model Development and Assessment Process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

A Framework for Preliminary Design of Aircraft Structures Based on Process Information

This report discusses the general framework and development of a computational tool for preliminary design of aircraft structures based on process information. The described methodology is suitable for multidisciplinary design optimization (MDO) activities associated with integrated product and process development (IPPD). The framework consists of three parts: (1) product and process definitions; (2) engineering synthesis, and (3) optimization. The product and process definitions are part of input information provided by the design team. The backbone of the system is its ability to analyze a given structural design for performance as well as manufacturability and cost assessment. The system uses a database on material systems and manufacturing processes. Based on the identified set of design variables and an objective function, the system is capable of performing optimization subject to manufacturability, cost, and performance constraints. The accuracy of the manufacturability measures and cost models discussed here depend largely on the available data on specific methods of manufacture and assembly and associated labor requirements. As such, our focus in this research has been on the methodology itself and not so much on its accurate implementation in an industrial setting. A three-tier approach is presented for an IPPD-MDO based design of aircraft structures. The variable-complexity cost estimation methodology and an approach for integrating manufacturing cost assessment into design process are also discussed. This report is presented in two parts. In the first part, the design methodology is presented, and the computational design tool is described. In the second part, a prototype model of the preliminary design Tool for Aircraft Structures based on Process Information (TASPI) is described. Part two also contains an example problem that applies the methodology described here for evaluation of six different design concepts for a wing spar.

Rais-Rohani, Masoud↗

Experimentally-Derived Bidirectional Reflectance Distribution Function Data in Support of the Orbital Debris Program Office

The NASA Orbital Debris Program Office (ODPO) has used various optical assets to acquire photometric data of Earth-orbiting objects to define the orbital debris environment. To better characterize and model optical data acquired from ground-based telescopes, the Optical Measurements Center (OMC) at NASA Johnson Space Center emulates illumination conditions seen in space by using equipment and techniques that parallel telescopic observations and source-target-sensor orientations. One of the OMC goals is to improve the size calculation used for optical data by developing an optical-based Size Estimation Model. The current size estimation requires applying a Lambertian phase function, a set albedo value, and range to the observed magnitude. The first step to improving the sampled brightness of laboratory targets is to remove aspect-angle dependencies. Then, the volume of possible object viewing angles is sampled at 21 combinations of azimuth and zenith angles for each solar phase angle. Finally, the acquired images are input into an image processing program that generates approximations for the object’s Bidirectional Reflectance Distribution Function (BRDF) and phase function. The BRDF is a radiometric concept that identifies an object’s material composition by matching a BRDF approximated with photometric data collected by ground-based telescopes with a BRDF generated experimentally from a known object in the laboratory. This paper discusses the validation of experimental BRDF and phase function approximations produced in the OMC and how the findings will be incorporated into ODPO models. A Lambertian sphere is imaged and the subsequent experimental functions are scrutinized to confirm that they correspond to an object that has an isotropic luminance. With the image processing algorithm validated, test objects with varying optical properties are then imaged to confirm that the produced photometric functions are both unique and repeatable. Once the validation is complete, the OMC will be used to evaluate a subset of fragments from a hypervelocity impact test of a mock-up satellite and assess the appropriate phase function and size estimates using BRDF measurements for a large volume of targets composed of various shapes, sizes, and materials.

Orbital Debris Program Office (ODPO)↗

Use of Raman Spectroscopy and Delta Volume Growth from Void Collapse to Assess Overwrap Stress Gradients Compromising the Reliability of Large Kevlar/Epoxy COPVs

Composite Overwrapped Pressure Vessels (COPVs) are frequently used for storing pressurized gases aboard spacecraft and aircraft when weight saving is desirable compared to all-metal versions. Failure mechanisms in fibrous COPVs and variability in lifetime can be very different from their metallic counterparts; in the former, catastrophic stress-rupture can occur with virtually no warning, whereas in latter, a leak before burst design philosophy can be implemented. Qualification and certification typically requires only one burst test on a production sample (possibly after several pressure cycles) and the vessel need only meet a design burst strength (the maximum operating pressure divided by a knockdown factor). Typically there is no requirement to assess variability in burst strength or lifetime, much less determine production and materials processing parameters important to control of such variability. Characterizing such variability and its source is crucial to models for calculating required reliability over a given lifetime (e.g. R = 0.9999 for 15 years). In this paper we present a case study of how lack of control of certain process parameters in COPV manufacturing can result in variations among vessels and between production runs that can greatly increase uncertainty and reduce reliability. The vessels considered are 40-inch ( NASA Glenn Research center, Cleveland, OH, 44135 29,500 in3 ) spherical COPVs with a 0.74 in. thick Kevlar49/epoxy overwrap and with a titanium liner of which 34 were originally produced. Two burst tests were eventually performed that unexpectedly differed by almost 5%, and were 10% lower than anticipated from burst tests on 26-inch sister vessels similar in every detail. A major observation from measurements made during proof testing (autofrettage) of the 40-inch vessels was that permanent volume growth from liner yielding varied by a factor of more than two (150 in3 to 360 in3 ), which suggests large differences in the residual stress gradient through their overwraps. This resulted in large uncertainty in true fiber stress ratio (fiber stress at operating pressure divided by fiber stress at burst) which governs lifetime. The vessels were originally designed with tight safety margins, so it became crucial to develop a non-destructive evaluation (NDE) technique to directly measure the overwrap residual stress state of each vessel, and to identify those vessels at highest risk of having poor reliability. This paper describes a Raman Spectroscopy technique for measuring certain patterns of fluctuation in fiber elastic strains over the outside vessel surface (where all but one wrap is exposed at certain locations) that are shown to directly correlate to increased fiber stress ratios and reduced reliability.

Kezirian, Michael T.↗

Autonomous Navigation with Ground-to-Space Doppler Measurements Referenced to a Temperature-Compensated Crystal Oscillator

The National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) has spent several years developing operational Onboard Navigation Systems (ONS's) to provide real-time autonomous, high-accuracy navigation products for spacecraft using NASA's space and ground communication systems. The highly successful Tracking and Data Relay Satellite System (TDRSS) ONS (TONS) experiment on the Explorer Platform/Extreme Ultraviolet Explorer (EP/EUVE) spacecraft, launched June 7, 1992, flight-demonstrated the ONS for high-accuracy navigation using TDRSS forward- link communications services, In late 1994, the ground station ONS (GONS) experiment, using the same EP/EUVE flight hardware, flight-demonstrated the feasibility of high-accuracy autonomous navigation using ground station forward-link communication services, with an ultrastable oscillator (USO) as the frequency reference (1 part in 10(exp 10) over 24 hours). This paper provides a follow-on analysis of GONS performance to assess the navigation accuracy achievable if GONS uses the significantly less stable (5 parts in 10(exp 8) over 24 hours, compared with a specification of 1 part in 10(exp 6)) Temperature-Compensated Crystal Oscillator (TCXO), which is integral to the transponder, as a frequency reference rather than an external USO. The GONS TCXO experiment results from a 20-day period are used to project the expected performance of an operational system. The GONS processes Doppler data derived from nominally scheduled ground station forward-link communication services using a sequential estimation algorithm enhanced by a sophisticated process noise model to provide onboard orbit and frequency determination. To evaluate the navigation accuracies achievable if a TCXO were used, actual experiment data (which used the USO as the frequency reference) were corrupted with errors from real TDRSS one-way return tracking measurements taken from EP/EUVE's TCXO. Analysis of the GONS TCXO experiment performance indicates that real-time onboard position accuracies of better than 300 meters (1 sigma) are achievable with as few as two tracking contacts per day for the EP/EUVE 525-kilometer altitude, 28.5-degree inclination orbit, provided the TCXO is as stable as that in EP/EUVE's TDRSS transponder. GONS using a TCXO frequency reference provides a viable option for many upcoming spacecraft missions with moderate position accuracy requirements and an interest in reducing their operational costs with autonomous navigation.

Radomski, M. S.↗

Tracking Critical Thermal Metrics throughout the Life Cycle of a Large Observatory Thermal Model

Observatory thermal models for large, complex missions, such as the Wide Field InfraRed Survey Telescope (WFIRST) mission, produce an immense amount of data to be processed. Configuration management of the model throughout the project life cycle has mainly focused on which versions of the subsystem models form the current observatory level configuration. However, the results produced by the model are not nearly as well tracked and traceable. Given the various states of design maturity for each of the components in the WFIRST design, an updated component model is nearly ready to be integrated at the next higher level of assembly about every month or two. With each subsystem model delivery, the observatory model needs to remove the old component, integrate the new one, execute the model, and inspect the results. Usually, this inspection focuses primarily on the newly integrated component. Recently, a Metric Tracking Spreadsheet was developed to help provide a “big picture” view of the entire observatory highlighting key parameters critical to mission performance. This spreadsheet helps track impacts on subsystems by updates of other subsystems that were not intuitively obvious. Metrics tracked include: absorbed environmental loading (to determine effectiveness of sunshield), temperatures of critical avionics, electrical dissipations, heater power predictions, stability of critical optics, parasitic heat leaks in cryogenic region, high level heat flows between elements, and model run time. Producing this data for the same operational configuration with each model update has helped produce a trail of data to evaluate the impact of model updates. While the metrics selected are specific for WFIRST, other large, complex observatories could be well served to establish their own metrics early in the project life cycle to track to quickly assess the impact of any subsystem on other subsystems or the overall system itself.

Thermal Desktop↗

Evaluation of Aerosol Data Assimilation and Forecasts in the NASA GEOS Model during the ASIA-AQ Campaign

Fine particulate matter (PM2.5) poses significant risks to human health and the environment by penetrating the lungs and causing respiratory and cardiovascular diseases, making it crucial to understand its sources and behavior for effective air quality management. The Goddard Earth Observing System (GEOS) Forward Processing (FP) system model, operated by the Global Modeling and Assimilation Office (GMAO) at NASA's Goddard Space Flight Center, provides real-time weather and aerosol analyses and forecasts. In addition to meteorological data assimilation, the GEOS-FP system also assimilates aerosol using Moderate Resolution Imaging Spectroradiometer (MODIS) Aerosol Optical Depth (AOD) and Aerosol Robotic Network (AERONET) AOD data. In this study, the aerosol data assimilation and forecasts performance of the GEOS-FP model were evaluated for predicting PM2.5 in Korea using observations from the Airborne and Satellite Investigation of Asian Air Quality (ASIA-AQ) campaign. The ASIA-AQ campaign, an international collaborative field study initiative, aims to enhance understanding of local air quality issues and address common challenges in interpreting satellite data and air quality modeling. Conducted in South Korea from February 15 to March 13, 2024, during the high PM2.5 concentration winter season, this campaign provided extensive airborne and ground observations for intensive analysis of PM2.5 model simulations. We demonstrate how the assimilation runs and the forecasting performance of PM2.5 at 24-hour and 48-hour intervals vary. Additionally, we analyzed the differences and characteristics of PM2.5 composition in cases of long-range transport and local emissions. Using ASIA-AQ airborne data, we also examined the vertical profile of fine particulate matter. Through the intensive observations of this campaign, the GEOS model was assessed over South Korea using both in situ and airborne measurements to establish a baseline and identify priorities for future development.

Seunghee Lee↗

Fine particulate concentrations over East Asia derived from aerosols measured by the Advanced Himawari Imager using machine learning

Fine particulate matter with a diameter below 2.5 μm (PM 2.5 ) is deleterious to the cardiovascular and respiratory systems. It is often difficult to assess the effects of PM 2.5 on human health over regions with limited ground monitoring sites, especially in East Asia. As an alternative, we estimated near-surface PM 2.5 concentrations by analyzing Advanced Himawari Imager (AHI) Yonsei Aerosol Retrieval (YAER) products. This study incorporates daytime data for East Asia covering the Korean Peninsula, China, Japan, Southeast Asia, and southern Mongolia. We collocated AHI YAER product pixels with meteorological, land-cover, and other ancillary data for the period from March 2018 to February 2019. To estimate PM 2.5 concentrations over wide areas spanning many countries displaying various relationships between aerosol optical depth and PM 2.5 , monthly models were developed by considering both the spatial and temporal characteristics of ground-based PM 2.5 measurements. Random forest machine learning model estimated ground-level mass concentrations of PM 2.5 ; subsequent 10-fold cross validation (CV) yielded a CV R 2 value of 0.81 and a CV root mean squared error (RMSE) of 12.3 μg m -3 . We investigated the spatial pattern of PM 2.5 concentrations over multiple countries and seasonal variation in PM 2.5 concentrations. Diurnal variation of a severe PM 2.5 event in the Korean Peninsula was investigated as a case study. The model captured the extremely heterogeneous spatial distribution of PM 2.5 concentrations peaked around local noon. To measure the capability of the developed model to estimate PM 2.5 concentrations in areas with few in-situ data, its predictive performance was evaluated using a dataset independent of the training process with an R 2 of 0.60 and RMSE of 8.18 μg m −3 . This study demonstrates the potential for satellite-based PM 2.5 estimation for areas with insufficient measuring stations.

Pm2.5↗

Aggregation Tool to Create Curated Data albums to Support Disaster Recovery and Response

Despite advances in science and technology of prediction and simulation of natural hazards, losses incurred due to natural disasters keep growing every year. Natural disasters cause more economic losses as compared to anthropogenic disasters. Economic losses due to natural hazards are estimated to be around $6-$10 billion dollars annually for the U.S. and this number keeps increasing every year. This increase has been attributed to population growth and migration to more hazard prone locations such as coasts. As this trend continues, in concert with shifts in weather patterns caused by climate change, it is anticipated that losses associated with natural disasters will keep growing substantially. One of challenges disaster response and recovery analysts face is to quickly find, access and utilize a vast variety of relevant geospatial data collected by different federal agencies such as DoD, NASA, NOAA, EPA, USGS etc. Some examples of these data sets include high spatio-temporal resolution multi/hyperspectral satellite imagery, model prediction outputs from weather models, latest radar scans, measurements from an array of sensor networks such as Integrated Ocean Observing System etc. More often analysts may be familiar with limited, but specific datasets and are often unaware of or unfamiliar with a large quantity of other useful resources. Finding airborne or satellite data useful to a natural disaster event often requires a time consuming search through web pages and data archives. Additional information related to damages, deaths, and injuries requires extensive online searches for news reports and official report summaries. An analyst must also sift through vast amounts of potentially useful digital information captured by the general public such as geo-tagged photos, videos and real time damage updates within twitter feeds. Collecting and aggregating these information fragments can provide useful information in assessing damage in real time and help direct recovery efforts. The search process for the analyst could be made much more efficient and productive if a tool could go beyond a typical search engine and provide not just links to web sites but actual links to specific data relevant to the natural disaster, parse unstructured reports for useful information nuggets, as well as gather other related reports, summaries, news stories, and images. This presentation will describe a semantic aggregation tool developed to address similar problem for Earth Science researchers. This tool provides automated curation, and creates "Data Albums" to support case studies. The generated "Data Albums" are compiled collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; information about the event contained in news reports, and images or videos to supplement research analysis. An ontology-based relevancy-ranking algorithm drives the curation of relevant data sets for a given event. This tool is now being used to generate a catalog of Hurricane Case Studies at Global Hydrology Resource Center (GHRC), one of NASA's Distribute Active Archive Centers. Another instance of the Data Albums tool is currently being created in collaboration with NASA/MSFC's SPoRT Center, which conducts research on unique NASA products and capabilities that can be transitioned to the operational community to solve forecast problems. This new instance focuses on severe weather to support SPoRT researchers in their model evaluation studies

Ramachandran, Rahul↗

Development of Standardized Lunar Regolith Simulant Materials

Lunar exploration requires scientific and engineering studies using standardized testing procedures that ultimately support flight certification of technologies and hardware. It is necessary to anticipate the range of source materials and environmental constraints that are expected on the Moon and Mars, and to evaluate in-situ resource utilization (ISRU) coupled with testing and development. We describe here the development of standardized lunar regolith simulant (SLRS) materials that are traceable inter-laboratory standards for testing and technology development. These SLRS materials must simulate the lunar regolith in terms of physical, chemical, and mineralogical properties. A summary of these issues is contained in the 2005 Workshop on Lunar Regolith Simulant Materials [l]. Lunar mare basalt simulants MLS-1 and JSC-1 were developed in the late 1980s. MLS-1 approximates an Apollo 11 high-Ti basalt, and was produced by milling of a holocrystalline, coarse-grained intrusive gabbro (Fig. 1). JSC-1 approximates an Apollo 14 basalt with a relatively low-Ti content, and was obtained from a glassy volcanic ash (Fig. 2). Supplies of MLS-1 and JSC-1 have been exhausted and these materials are no longer available. No highland anorthosite simulant was previously developed. Upcoming lunar polar missions thus require the identification, assessment, and development of both mare and highland simulants. A lunar regolith simulant is manufactured from terrestrial components for the purpose of simulating the physical and chemical properties of the lunar regolith. Significant challenges exist in the identification of appropriate terrestrial source materials. Lunar materials formed under comparatively reducing conditions in the absence of water, and were modified by meteorite impact events. Terrestrial materials formed under more oxidizing conditions with significantly greater access to water, and were modified by a wide range of weathering processes. The composition space of lunar materials can be modeled by mixing programs utilizing a low-Ti basalt, ilmenite, KREEP component, high-Ca anorthosite, and meteoritic components. This approach has been used for genetic studies of lunar samples via chemical and modal analysis. A reduced composition space may be appropriate for simulant development, but it is necessary to determine the controlling properties that affect the physical, chemical and mineralogical components of the simulant.

Carpenter, P.↗

Spectral Analysis of Integrated Pressures on Patches with Unsteady Pressure-Sensitive Paint Measurements

This paper describes the spectral analysis of integrated pressures on patches of the scale model of the Space Launch System (SLS) Block 1B crew vehicle with the Unsteady Pressure-Sensitive Paint (uPSP) measurements, which were collected in the ascent transient aerodynamics tests with the Unitary Plan Wind Tunnel 11-by-11-foot Transonic Wind Tunnel in September 2019 at NASA Ames Research Center. Recent research has demonstrated that uPSP can be an essential tool for the assessment of the unsteady, aerodynamic phenomena. The work described in this paper is a part of NASA’s development of a new state-of-the-art uPSP capability in production wind tunnels. In this paper, 108 patches are defined with x station values and azimuth angles of the scale model. For each patch, the polygons are determined from the surface cells of the grid of the model, clipped with the edges of the patch, and each of the polygons is divided into triangles. The inputs of the pressure integration algorithm are the time series of pressure coefficients on the vertices of the grid of the model, generated by the uPSP data processing program from the videos taken with 4 Phantom high-speed cameras during the wind tunnel tests. The integrated pressure of the patch is determined as the ratio of the sum of the forces on the triangles over the sum of the areas of the triangles. For each of the test cases, the Cross Power Spectral Density (CPSD) and Magnitude-Squared Coherence (msCohere) are computed for frequencies over 1/3 octave bands from the time series of the integrated pressures on the patches. The spectral analysis outputs of different cases of the wind tunnel tests are analyzed and the coherence of patches is investigated over frequencies, x station values, azimuth angles, Mach numbers and SLS vehicle attitudes. The pressure integration and spectral analysis described in this paper were executed on the NASA Pleiades supercomputer. Funding for this research was provided by the NASA Aeroscience Evaluation and Test Capabilities (AETC) Project.

Pressure-Sensitive Paint↗

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval. We present two methods for comparing sequences of this sort, one based on computing the time-varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions.We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of activities, each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates.We have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Haslbeck, Gontar Schubert, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

attention↗

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval.We present two methods for comparing sequences of this sort, one based on computing the time varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions.We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of activities, each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates. Using these criteria, we have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Gontar Hoermann, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

Mulligan, Jeffrey B.↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval. We present two methods for comparing sequences of this sort, one based on computing the time-varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions. We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of "activities," each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates. We have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Haslbeck, Gontar & Schubert, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

attention↗

Material Property Estimation in Thin Battery Components Using Guided Wave Measurement, Experimental Dispersion Curve Extraction and Finite Element Modeling

At NASA, we are investigating nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques to detect precursors of thermal runaway failure in lithium metal based lithium ion batteries (LIB). The approach is centered on computational simulation models to guide inspection and aid in interpretation of results. Since lithium metal LIBs have combinations of solid and fluid-filled porous materials, obtaining accurate material properties is both challenging and critical for successful simulation of battery inspection. To this end, we investigated a multi-verification approach for material characterization of thin battery components. First, a laser Doppler vibrometer (LDV) was used to measure guided wave fields in thin (microns-thick) battery components subject to broadband excitation. The time-space wavefield data was converted to frequency-wave number data to extract guided wave dispersion curves. A data visualization and post processing graphics user interface (GUI) was developed at NASA to aid the data exploration and analysis. Due to the thinness of the samples, low frequency-thickness-product plate wave approximations allowed for the calculation of closed-form solutions for material elastic property estimation. These approximations were then verified by calculating the Lamb wave solutions using the previously obtained material properties. Finally, the estimated elastic material properties were implemented in a COMSOL simulation model, and dispersion curves were extracted from simulation results. The dispersion curves and derived material properties were then compared to the analysis results from the experimental data. These comparisons informed on the accuracy of the measured material properties and helped demonstrate the accuracy of the finite element analysis (FEA) computational models. This assessment will prove vital when we start simulating more complex multi-layer components and poroelastic media. This paper gives a brief background of the problem space, outlines the workflow for data analysis and verification, shows results from the workflow, and gives an overview of future plans for simulation of lithium metal LIB inspection.

Peter Juarez↗

The SMART-NAS Testbed

The SMART-NAS Testbed for Safe Trajectory Based Operations Project will deliver an evaluation capability, critical to the ATM community, allowing full NextGen and beyond-NextGen concepts to be assessed and developed. To meet this objective a strong focus will be placed on concept integration and validation to enable a gate-to-gate trajectory-based system capability that satisfies a full vision for NextGen. The SMART-NAS for Safe TBO Project consists of six sub-projects. Three of the sub-projects are focused on exploring and developing technologies, concepts and models for evolving and transforming air traffic management operations in the ATM+2 time horizon, while the remaining three sub-projects are focused on developing the tools and capabilities needed for testing these advanced concepts. Function Allocation, Networked Air Traffic Management and Trajectory Based Operations are developing concepts and models. SMART-NAS Test-bed, System Assurance Technologies and Real-time Safety Modeling are developing the tools and capabilities to test these concepts. Simulation and modeling capabilities will include the ability to assess multiple operational scenarios of the national airspace system, accept data feeds, allowing shadowing of actual operations in either real-time, fast-time and/or hybrid modes of operations in distributed environments, and enable integrated examinations of concepts, algorithms, technologies, and NAS architectures. An important focus within this project is to enable the development of a real-time, system-wide safety assurance system. The basis of such a system is a continuum of information acquisition, analysis, and assessment that enables awareness and corrective action to detect and mitigate potential threats to continuous system-wide safety at all levels. This process, which currently can only be done post operations, will be driven towards "real-time" assessments in the 2035 time frame.

Trajectory Based Operations↗

The NASA Soil Moisture Active Passive (SMAP) Mission - Algorithm and Cal/Val Activities and Synergies with SMOS and Other L-Band Missions

NASA's Soil Moisture Active Passive (SMAP) mission, planned for launch in late 2014, has as its key measurement objective the frequent, global mapping of near-surface soil moisture and its freeze-thaw state. SMAP soil moisture and freeze/thaw measurements at 10 km and 3 km resolutions respectively, would enable significantly improved estimates of water, energy and carbon transfers between the land and atmosphere. Soil moisture control of these fluxes is a key factor in the performance of atmospheric models used for weather forecasts and climate projections Soil moisture measurements are also of great importance in assessing floods and for monitoring drought. In addition, observations of soil moisture and freeze/thaw timing over the boreal latitudes can help reduce uncertainties in quantifying the global carbon balance. The SMAP measurement concept utilizes an L-band radar and radiometer sharing a rotating 6-meter mesh reflector antenna. The SMAP radiometer and radar flight hardware and ground processing designs are incorporating approaches to identify and mitigate potential terrestrial radio frequency interference (RFI). The radar and radiometer instruments are planned to operate in a 680 km polar orbit, viewing the surface at a constant 40-degree incidence angle with a 1000-km swath width, providing 3-day global coverage. Data from the instruments would yield global maps of soil moisture and freeze/thaw state to be provided at 10 km and 3 km resolutions respectively, every two to three days. Plans are to provide also a radiometer-only soil moisture product at 40-km spatial resolution. This product and the underlying brightness temperatures have characteristics similar to those provided by the Soil Moisture and Ocean Salinity (SMOS) mission. As a result, there are unique opportunities for common data product development and continuity between the two missions. SMAP also has commonalities with other satellite missions having L-band radiometer and/or radar sensors applicable to soil moisture measurement, such as Aquarius, SAO COM, and ALOS-2. The algorithms and data products for SMAP are being developed in the SMAP Science Data System (SDS) Testbed. The algorithms are developed and evaluated in the SDS Testbed using simulated SMAP observations as well as observational data from current airborne and spaceborne L-band sensors including SMOS. The SMAP project is developing a Calibration and Validation (Cal/Val) Plan that is designed to support algorithm development (pre-launch) and data product validation (post-launch). A key component of the Cal/Val Plan is the identification, characterization, and instrumentation of sites that can be used to calibrate and validate the sensor data (Level I) and derived geophysical products (Level 2 and higher). In this presentation we report on the development status of the SMAP data product algorithms, and the planning and implementation of the SMAP Cal/Val program. Several components of the SMAP algorithm development and Cal/Val plans have commonality with those of SMOS, and for this reason there are shared activities and resources that can be utilized between the missions, including in situ networks, ancillary data sets, and long-term monitoring sites.

Njoku, Eni↗