Search NASA⌕ Search

SEARCH · Search NASA

Results for “ROC analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Advanced Signal Processing Methods Applied to Digital Mammography

The work reported here is on the extension of the earlier proposal of the same title, August 1994-June 1996. The report for that work is also being submitted. The work reported there forms the foundation for this work from January 1997 to September 1997. After the earlier work was completed there were a few items that needed to be completed prior to submission of a new and more comprehensive proposal for further research. Those tasks have been completed and two new proposals have been submitted, one to NASA, and one to Health & Human Services WS). The main purpose of this extension was to refine some of the techniques that lead to automatic large scale evaluation of full mammograms. Progress on each of the proposed tasks follows. Task 1: A multiresolution segmentation of background from breast has been developed and tested. The method is based on the different noise characteristics of the two different fields. The breast field has more power in the lower octaves and the off-breast field behaves similar to a wideband process, where more power is in the high frequency octaves. After the two fields are separated by lowpass filtering, a region labeling routine is used to find the largest contiguous region, the breast. Task 2: A wavelet expansion that can decompose the image without zero padding has been developed. The method preserves all properties of the power-of-two wavelet transform and does not add appreciably to computation time or storage. This work is essential for analysis of the full mammogram, as opposed to selecting sections from the full mammogram. Task 3: A clustering method has been developed based on a simple counting mechanism. No ROC analysis has been performed (and was not proposed), so we cannot finally evaluate this work without further support. Task 4: Further testing of the filter reveals that different wavelet bases do yield slightly different qualitative results. We cannot provide quantitative conclusions about this for all possible bases without further support. Task 5: Better modeling does indeed make an improvement in the detection output. After the proposal ended, we came up with some new theoretical explanations that helps in understanding when the D4 filter should be better. This work is currently in the review process. Task 6: N/A. This no longer applies in view of Tasks 4-5. Task 7: Comprehensive plans for further work have been completed. These plans are the subject of two proposals, one to NASA and one to HHS. These proposals represent plans for a complete evaluation of the methods for identifying normal mammograms, augmented with significant further theoretical work.

Stauduhar, Richard P.↗

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval. We present two methods for comparing sequences of this sort, one based on computing the time-varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions.We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of activities, each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates.We have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Haslbeck, Gontar Schubert, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

attention↗

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval. We present two methods for comparing sequences of this sort, one based on computing the time-varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions. We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of "activities," each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates. We have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Haslbeck, Gontar & Schubert, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

attention↗

Measuring and Modeling Shared Visual Attention

Multi-person teams are sometimes responsible for critical tasks, such as flying an airliner. Here we present a method using gaze tracking data to assess shared visual attention, a term we use to describe the situation where team members are attending to a common set of elements in the environment. Gaze data are quantized with respect to a set of N areas of interest (AOIs); these are then used to construct a time series of N dimensional vectors, with each vector component representing one of the AOIs, all set to 0 except for the component corresponding to the currently fixated AOI, which is set to 1. The resulting sequence of vectors can be averaged in time, with the result that each vector component represents the proportion of time that the corresponding AOI was fixated within the given time interval.We present two methods for comparing sequences of this sort, one based on computing the time varying correlation of the averaged vectors, and another based on a chi-square test testing the hypothesis that the observed gaze proportions are drawn from identical probability distributions.We have evaluated the method using synthetic data sets, in which the behavior was modeled as a series of activities, each of which was modeled as a first-order Markov process. By tabulating distributions for pairs of identical and disparate activities, we are able to perform a receiver operating characteristic (ROC) analysis, allowing us to choose appropriate criteria and estimate error rates. Using these criteria, we have applied the methods to data from airline crews, collected in a high-fidelity flight simulator (Gontar Hoermann, 2014). We conclude by considering the problem of automatic (blind) discovery of activities, using methods developed for text analysis.

Mulligan, Jeffrey B.↗

FIESTA ROC: A new finite element analysis program for solar cell simulation

The Finite Element Semiconductor Three-dimensional Analyzer by Ralph O. Clark (FIESTA ROC) is a computational tool for investigating in detail the performance of arbitrary solar cell structures. As its name indicates, it uses the finite element technique to solve the fundamental semiconductor equations in the cell. It may be used for predicting the performance (thereby dictating the design parameters) of a proposed cell or for investigating the limiting factors in an established design.

Clark, Ralph O.↗

Psychophysical Models for Signal Detection with Time Varying Uncertainty

Psychophysical models for the behavior of the human operator in detection tasks which include change in detectability, correlation between observations and deferred decisions are developed. Classical Signal Detection Theory (SDT) is discussed and its emphasis on the sensory processes is contrasted to decision strategies. The analysis of decision strategies utilizes detection tasks with time varying signal strength. The classical theory is modified to include such tasks and several optimal decision strategies are explored. Two methods of classifying strategies are suggested. The first method is similar to the analysis of ROC curves, while the second is based on the relation between the criterion level (CL) and the detectability. Experiments to verify the analysis of tasks with changes of signal strength are designed. The results show that subjects are aware of changes in detectability and tend to use strategies that involve changes in the CL's.

Gai, E.↗

Signal Detection Theory Applied to Helicopter Transmission Diagnostic Thresholds

Helicopter Health Usage Monitoring Systems (HUMS) have potential for providing data to support increasing the service life of a dynamic mechanical component in the transmission of a helicopter. Data collected can demonstrate the HUMS condition indicator responds to a specific component fault with appropriate alert limits and minimal false alarms. Defining thresholds for specific faults requires a tradeoff between the sensitivity of the condition indicator (CI) limit to indicate damage and the number of false alarms. A method using Receiver Operating Characteristic (ROC) curves to assess CI performance was demonstrated using CI data collected from accelerometers installed on several UH60 Black Hawk and AH64 Apache helicopters and an AH64 helicopter component test stand. Results of the analysis indicate ROC curves can be used to reliably assess the performance of commercial HUMS condition indicators to detect damaged gears and bearings in a helicopter transmission.

Dempsey, Paula J.↗

Signal Detection Theory Applied to Helicopter Transmission Diagnostic Thresholds

Helicopter Health Usage Monitoring Systems (HUMS) have potential for providing data to support increasing the service life of a dynamic mechanical component in the transmission of a helicopter. Data collected can demonstrate the HUMS condition indicator responds to a specific component fault with appropriate alert limits and minimal false alarms. Defining thresholds for specific faults requires a tradeoff between the sensitivity of the condition indicator (CI) limit to indicate damage and the number of false alarms. A method using Receiver Operating Characteristic (ROC) curves to assess CI performance was demonstrated using CI data collected from accelerometers installed on several UH60 Black Hawk and AH64 Apache helicopters and an AH64 helicopter component test stand. Results of the analysis indicate ROC curves can be used to reliably assess the performance of commercial HUMS condition indicators to detect damaged gears and bearings in a helicopter transmission.

Dempsey, Paula J.↗

A Perceptual Repetition Blindness Effect

Before concluding Repetition Blindness is a perceptual phenomenon, alternative explanations based on memory retrieval problems and report bias must be rejected. Memory problems were minimized by requiring a judgment about only a single briefly displayed field. Bias and sensitivity effects were empirically measured with an ROC-curve analysis method based on confidence ratings. Results from five experiments support the hypothesis that Repetition Blindness can be a perceptual phenomenon.

Hochhaus, Larry↗

An Investigation of State-Space Model Fidelity for SSME Data

In previous studies, a variety of unsupervised anomaly detection techniques for anomaly detection were applied to SSME (Space Shuttle Main Engine) data. The observed results indicated that the identification of certain anomalies were specific to the algorithmic method under consideration. This is the reason why one of the follow-on goals of these previous investigations was to build an architecture to support the best capabilities of all algorithms. We appeal to that goal here by investigating a cascade, serial architecture for the best performing and most suitable candidates from previous studies. As a precursor to a formal ROC (Receiver Operating Characteristic) curve analysis for validation of resulting anomaly detection algorithms, our primary focus here is to investigate the model fidelity as measured by variants of the AIC (Akaike Information Criterion) for state-space based models. We show that placing constraints on a state-space model during or after the training of the model introduces a modest level of suboptimality. Furthermore, we compare the fidelity of all candidate models including those embodying the cascade, serial architecture. We make recommendations on the most suitable candidates for application to subsequent anomaly detection studies as measured by AIC-based criteria.

Martin, Rodney Alexander↗

Evaluating Alerting and Guidance Performance of a UAS Detect-And-Avoid System

A key challenge to the routine, safe operation of unmanned aircraft systems (UAS) is the development of detect-and-avoid (DAA) systems to aid the UAS pilot in remaining "well clear" of nearby aircraft. The goal of this study is to investigate the effect of alerting criteria and pilot response delay on the safety and performance of UAS DAA systems in the context of routine civil UAS operations in the National Airspace System (NAS). A NAS-wide fast-time simulation study was conducted to assess UAS DAA system performance with a large number of encounters and a broad set of DAA alerting and guidance system parameters. Three attributes of the DAA system were controlled as independent variables in the study to conduct trade-off analyses: UAS trajectory prediction method (dead-reckoning vs. intent-based), alerting time threshold (related to predicted time to LoWC), and alerting distance threshold (related to predicted Horizontal Miss Distance, or HMD). A set of metrics, such as the percentage of true positive, false positive, and missed alerts, based on signal detection theory and analysis methods utilizing the Receiver Operating Characteristic (ROC) curves were proposed to evaluate the safety and performance of DAA alerting and guidance systems and aid development of DAA system performance standards. The effect of pilot response delay on the performance of DAA systems was evaluated using a DAA alerting and guidance model and a pilot model developed to support this study. A total of 18 fast-time simulations were conducted with nine different DAA alerting threshold settings and two different trajectory prediction methods, using recorded radar traffic from current Visual Flight Rules (VFR) operations, and supplemented with DAA-equipped UAS traffic based on mission profiles modeling future UAS operations. Results indicate DAA alerting distance threshold has a greater effect on DAA system performance than DAA alerting time threshold or ownship trajectory prediction method. Further analysis on the alert lead time (time in advance of predicted loss of well clear at which a DAA alert is first issued) indicated a strong positive correlation between alert lead time and DAA system performance (i.e. the ability of the UAS pilot to maneuver the unmanned aircraft to remain well clear). While bigger distance thresholds had beneficial effects on alert lead time and missed alert rate, it also generated a higher rate of false alerts. In the design and development of DAA alerting and guidance systems, therefore, the positive and negative effects of false alerts and missed alerts should be carefully considered to achieve acceptable alerting system performance by balancing false and missed alerts. The results and methodology presented in this study are expected to help stakeholders, policymakers and standards committees define the appropriate setting of DAA system parameter thresholds for UAS that ensure safety while minimizing operational impacts to the NAS and equipage requirements for its users before DAA operational performance standards can be finalized.

UAS Detect-and-Avoid (DAA) System↗

Refinement of a Method for Identifying Probable Archaeological Sites from Remotely Sensed Data

To facilitate locating archaeological sites before they are compromised or destroyed, we are developing approaches for generating maps of probable archaeological sites, through detecting subtle anomalies in vegetative cover, soil chemistry, and soil moisture by analyzing remotely sensed data from multiple sources. We previously reported some success in this effort with a statistical analysis of slope, radar, and Ikonos data (including tasseled cap and NDVI transforms) with Student's t-test. We report here on new developments in our work, performing an analysis of 8-band multispectral Worldview-2 data. The Worldview-2 analysis begins by computing medians and median absolute deviations for the pixels in various annuli around each site of interest on the 28 band difference ratios. We then use principle components analysis followed by linear discriminant analysis to train a classifier which assigns a posterior probability that a location is an archaeological site. We tested the procedure using leave-one-out cross validation with a second leave-one-out step to choose parameters on a 9,859x23,000 subset of the WorldView-2 data over the western portion of Ft. Irwin, CA, USA. We used 100 known non-sites and trained one classifier for lithic sites (n=33) and one classifier for habitation sites (n=16). We then analyzed convex combinations of scores from the Archaeological Predictive Model (APM) and our scores. We found that that the combined scores had a higher area under the ROC curve than either individual method, indicating that including WorldView-2 data in analysis improved the predictive power of the provided APM.

Tilton, James C.↗

Structural Analysis of Pressurized Small Diameter Lines in a Random Vibration Environment

The pressurization and propellant feed lines for the Ares 1 Upper Stage Reaction and Roll Control Systems (ReCS and RoCS) were required to be in a high g-load random vibration flight environment. The lines connected the system components and were filled with both liquid hydrazine and gaseous helium. They are considered small and varied between one fourth to one inch in diameter. The random vibration of the lines was considered to be base excitation through the mating components and mounting hardware. It was found that reducing the amount of support structure for the lines added flexibility to the system and improved the line stresses from random vibration, but caused higher stresses from the static g-loads. The locations and number of brackets were optimized by analyzing the mode shapes of the lines causing high stresses. The use of brackets that only constrain motion in the direction of concern further reduced the stresses in the lines. Finite element analysis was used to perform the analysis. The lines were pre-stressed by temperature and internal pressure with fluid and insulation included as non-structural mass. Base excitation was added to the model using Power Spectral Density (PSD) data for the expected flight loads. The random vibration and static g-load cases were combined to obtain the total stress in the lines. This approach advances the state of the art in line analysis by using FEA to predict the stresses in the lines and to optimize the entire system based on the expected flight environment. Adding flexibility to lines has been used in piping system for temperature loads, but in flight environments flexibility has been limited for the static stresses. Adding flexibility to the system in a flight environment by reducing brackets has the benefit of reducing stresses and weight

Davis, Mark↗

Ares I-X Ground Diagnostic Prototype

Automating prelaunch diagnostics for launch vehicles offers three potential benefits. First, it potentially improves safety by detecting faults that might otherwise have been missed so that they can be corrected before launch. Second, it potentially reduces launch delays by more quickly diagnosing the cause of anomalies that occur during prelaunch processing. Reducing launch delays will be critical to the success of NASA's planned future missions that require in-orbit rendezvous. Third, it potentially reduces costs by reducing both launch delays and the number of people needed to monitor the prelaunch process. NASA is currently developing the Ares I launch vehicle to bring the Orion capsule and its crew of four astronauts to low-earth orbit on their way to the moon. Ares I-X will be the first unmanned test flight of Ares I. It is scheduled to launch on October 27, 2009. The Ares I-X Ground Diagnostic Prototype is a prototype ground diagnostic system that will provide anomaly detection, fault detection, fault isolation, and diagnostics for the Ares I-X first-stage thrust vector control (TVC) and for the associated ground hydraulics while it is in the Vehicle Assembly Building (VAB) at John F. Kennedy Space Center (KSC) and on the launch pad. It will serve as a prototype for a future operational ground diagnostic system for Ares I. The prototype combines three existing diagnostic tools. The first tool, TEAMS (Testability Engineering and Maintenance System), is a model-based tool that is commercially produced by Qualtech Systems, Inc. It uses a qualitative model of failure propagation to perform fault isolation and diagnostics. We adapted an existing TEAMS model of the TVC to use for diagnostics and developed a TEAMS model of the ground hydraulics. The second tool, Spacecraft Health Inference Engine (SHINE), is a rule-based expert system developed at the NASA Jet Propulsion Laboratory. We developed SHINE rules for fault detection and mode identification. The prototype uses the outputs of SHINE as inputs to TEAMS. The third tool, the Inductive Monitoring System (IMS), is an anomaly detection tool developed at NASA Ames Research Center and is currently used to monitor the International Space Station Control Moment Gyroscopes. IMS automatically "learns" a model of historical nominal data in the form of a set of clusters and signals an alarm when new data fails to match this model. IMS offers the potential to detect faults that have not been modeled. The three tools have been integrated and deployed to Hangar AE at KSC where they interface with live data from the Ares I-X vehicle and from the ground hydraulics. The outputs of the tools are displayed on a console in Hangar AE, one of the locations from which the Ares I-X launch will be monitored. The full paper will describe how the prototype performed before the launch. It will include an analysis of the prototype's accuracy, including false-positive rates, false-negative rates, and receiver operating characteristics (ROC) curves. It will also include a description of the prototype's computational requirements, including CPU usage, main memory usage, and disk usage. If the prototype detects any faults during the prelaunch period then the paper will include a description of those faults. Similarly, if the prototype has any false alarms then the paper will describe them and will attempt to explain their causes.

Schwabacher, Mark↗

The Influence of Land Use and Land Cover Change on Landslide Susceptibility in the Lower Mekong River Basin

The Lower Mekong River Basin in Southeast Asia experiences frequent rainfall-triggered landslides especially during the monsoon season. In this study, the influence of land use and land cover change and other causative factors on landslide susceptibility is evaluated in the Lower Mekong Basin. Frequency ratio analysis is performed to quantify the relationship between LULC change and susceptibility. Detailed landslide inventory maps are used for analysis with yearly LULC maps. The LULC change is used as a contributing variable in a logistic regression-based susceptibility model with other variables including distance to roads, slope, aspect, forest loss, and soil properties. The Receiver Operating Characteristic (ROC) curve and Area Under the Curve are estimated for the model trained by each landslide inventory. The models show good performance, with AUC values ranging from 0.697 to 0.958 and an average AUC equal to 0.820. Both the Frequency Ratio analysis and the Logistic Regression models indicate LULC change from agricultural land to forest has a positive correlation with landslide occurrence. The most significant factors in the models are found to be distance to roads, slope, and aspect. A better understanding of the effects of LULC on landslide susceptibility can be useful for local land and disaster management and for the implementation of LULC as a factor in future susceptibility models. Using datasets that are unique to the Lower Mekong region, this study provides additional insights into the relationship between causative factors and landslide activity to better inform regional and global landslide susceptibility modeling.

River Basin↗

An Orbiting Sample Capture and Orientation System Architecture for Potential Mars Sample Return

An orbiting sample capture and orientation system architecture for a Rendezvous and Orbiting Sample Capture System (ROCS) concept was developed to enable spacecraft based, in-orbit capture, orientation, and transfer of a Mars sample container into a containment vessel as part of a potential Mars Sample Return (MSR) campaign. An analysis of the system functions, requirements, interactions, and constraints was performed. A trade study was carried out on relevant technologies, and a set of evaluation criteria was developed to help determine the most feasible concepts for implementation. The Mars Capture and Re-orientation for the potential Next Mars Orbiter (MACARONE) concept is proposed as a promising system architecture for the ROCS Capture and Orient Module (COM). The concept uses a sliding trap door for Orbiting Sample (OS) capture, a Motorized Cups Mechanism for OS orientation, and a 2 DOF Turret Arm with a paddle for transferring the OS into a containment vessel. This approach facilitates modularity, development flexibility, test-ability in a 1G environment, analyze-ability without the need to simulate or test for 0G contact dynamics, ability to encapsulate potential dust surrounding the OS, and ability to be ejected to reduce the probability of Earth exposure to non-sterilized Mars material.

Lalla, Karan↗

Impact of lossy compression on diagnostic accuracy of radiographs for periapical lesions

OBJECTIVES: The purpose of this study was to evaluate the lossy Joint Photographic Experts Group compression for endodontic pretreatment digital radiographs. STUDY DESIGN: Fifty clinical charge-coupled device-based, digital radiographs depicting periapical areas were selected. Each image was compressed at 2, 4, 8, 16, 32, 48, and 64 compression ratios. One root per image was marked for examination. Images were randomized and viewed by four clinical observers under standardized viewing conditions. Each observer read the image set three times, with at least two weeks between each reading. Three pre-selected sites per image (mesial, distal, apical) were scored on a five-scale score confidence scale. A panel of three examiners scored the uncompressed images, with a consensus score for each site. The consensus score was used as the baseline for assessing the impact of lossy compression on the diagnostic values of images. The mean absolute error between consensus and observer scores was computed for each observer, site, and reading session. RESULTS: Balanced one-way analysis of variance for all observers indicated that for compression ratios 48 and 64, there was significant difference between mean absolute error of uncompressed and compressed images (P <.05). After converting the five-scale score to two-level diagnostic values, the diagnostic accuracy was strongly correlated (R (2) = 0.91) with the compression ratio. CONCLUSION: The results of this study suggest that high compression ratios can have a severe impact on the diagnostic quality of the digital radiographs for detection of periapical lesions.

NASA Discipline Space Human Factors↗

Computational Analysis of Ares I Roll Control System Jet Interaction Effects on Rolling Moment

The computational flow solver USM3D was used to investigate the jet interaction effects from the roll control system on the rolling moment of the Ares I full protuberance configuration at wind tunnel Reynolds numbers. Solutions were computed at freestream Mach numbers from M = 0.5 to M = 5 at the angle of attack 0deg, at the angle of attack 3.5deg for a roll angle of 120deg, and at the angle of attack 7deg for roll angles of 120deg and 210deg. Results indicate that the RoCS housing provided a beneficial jet interaction effect on vehicle rolling moment for M > or = 0.9. Most of the components downstream of the roll control system housing contributed to jet interaction penalties on vehicle rolling moment.

Deere, Karen A.↗