Search NASA⌕ Search

SEARCH · Search NASA

Results for “statistical comparison”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Statistical-learning strategies generate only modestly performing predictive models for urinary symptoms following external beam radiotherapy of the prostate: A comparison of conventional and machine-learning methods

Purpose: Given the paucity of available data concerning radiotherapy-induced urinary toxicity, it is important to ensure derivation of the most robust models with superior predictive performance. This work explores multiple statistical-learning strategies for prediction of urinary symptoms following external beam radiotherapy of the prostate. Methods: The performance of logistic regression, elastic-net, support-vector machine, random forest, neural network, and multivariate adaptive regression splines (MARS) to predict urinary symptoms was analyzed using data from 754 participants accrued by TROG03.04-RADAR. Predictive features included dose-surface data, comorbidities, and medication-intake. Four symptoms were analyzed: dysuria, haematuria, incontinence, and frequency, each with three definitions (grade ≥ 1, grade ≥ 2 and longitudinal) with event rate between 2.3% and 76.1%. Repeated cross-validations producing matched models were implemented. A synthetic minority oversampling technique was utilized in endpoints with rare events. Parameter optimization was performed on the training data. Area under the receiver operating characteristic curve (AUROC) was used to compare performance using sample size to detect differences of ≥0.05 at the 95% confidence level. Results: Logistic regression, elastic-net, random forest, MARS, and support-vector machine were the highest-performing statistical-learning strategies in 3, 3, 3, 2, and 1 endpoints, respectively. Logistic regression, MARS, elastic-net, random forest, neural network, and support-vector machine were the best, or were not significantly worse than the best, in 7, 7, 5, 5, 3, and 1 endpoints. The best-performing statistical model was for dysuria grade ≥ 1 with AUROC ± standard deviation of 0.649 ± 0.074 using MARS. For longitudinal frequency and dysuria grade ≥ 1, all strategies produced AUROC>0.6 while all haematuria endpoints and longitudinal incontinence models produced AUROC<0.6. Conclusions: Logistic regression and MARS were most likely to be the best-performing strategy for the prediction of urinary symptoms with elastic-net and random forest producing competitive results. The predictive power of the models was modest and endpoint-dependent. New features, including spatial dose maps, may be necessary to achieve better models.

60 APPLIED LIFE SCIENCES↗

Signal detection amid noise with known statistics

Conventional methods of signal detection are based on the comparison of a local signal-to-noise ratio S/N to some threshold value. We present a new method that compares instead the histogram of the data with the one expected for background noise alone. Signals are detected from discrepancies between the two. We expect it to be applicable in the detection of signals from any data whose background statistics is known. To illustrate the method, we apply it to the case of photon-limited imaging data where the underlying background is Poissonian. Numerical simulations are used to check the efficiency of our method, finding that for signals with low S/N (less than 5), it detects signals with a higher degree of reliability than the conventional method.

Zepka, A. F.↗

Calibration and Data Analysis of the MC-130 Air Balance

Design, calibration, calibration analysis, and intended use of the MC-130 air balance are discussed. The MC-130 balance is an 8.0 inch diameter force balance that has two separate internal air flow systems and one external bellows system. The manual calibration of the balance consisted of a total of 1854 data points with both unpressurized and pressurized air flowing through the balance. A subset of 1160 data points was chosen for the calibration data analysis. The regression analysis of the subset was performed using two fundamentally different analysis approaches. First, the data analysis was performed using a recently developed extension of the Iterative Method. This approach fits gage outputs as a function of both applied balance loads and bellows pressures while still allowing the application of the iteration scheme that is used with the Iterative Method. Then, for comparison, the axial force was also analyzed using the Non-Iterative Method. This alternate approach directly fits loads as a function of measured gage outputs and bellows pressures and does not require a load iteration. The regression models used by both the extended Iterative and Non-Iterative Method were constructed such that they met a set of widely accepted statistical quality requirements. These requirements lead to reliable regression models and prevent overfitting of data because they ensure that no hidden near-linear dependencies between regression model terms exist and that only statistically significant terms are included. Finally, a comparison of the axial force residuals was performed. Overall, axial force estimates obtained from both methods show excellent agreement as the differences of the standard deviation of the axial force residuals are on the order of 0.001 % of the axial force capacity.

Booth, Dennis↗

Yearly variations of rain-rate statistics at Wallops Island and their impact on modeled slant path attenuation distributions

Rain gauge measurements at Wallops Island, VA over a five-year period have been reduced and cumulative rain-date distributions have been determined for yearly, successively combined years, and overall average cases. Yearly variations of the predicted slant path attenuation statistics for two International Radio Consultative Committee (CCIR) models (referred to as modified method I and method II) and the global model at 28.56 GHz were calculated using the yearly measured rain rates at Wallops Island. This frequency was selected as it coincided with the beacon frequency of te COMSTAR geostationary satellite which was also simultaneously monitored at Wallops Island for a period of three years enabling a comparison with the measured rain-fade statistics. The year to year variations of the predicted fades as well as the percent deviations relative to the five-year average case were examined.

Goldhirsh, J.↗

Psychophysical rating of image compression techniques

Image compression schemes abound with little work which compares their bit-rate performance based on subjective fidelity measures. Statistical measures of image fidelity, such as squared error measures, do not necessarily correspond to subjective measures of image fidelity. Most previous comparisons of compression techniques have been based on these statistical measures. A psychophysical method has been used to estimate, for a number of compression techniques, a threshold bit-rate yielding a criterion level of performance in discriminating original and compressed images. The compression techniques studied include block truncation, Laplacian pyramid, block discrete cosine transform, with and without a human visual system scaling, and cortex transform coders.

Stein, Charles S.↗

A statistical approach for detection of anatomical change - Paper 115

The purpose of this study was to develop a novel statistics-based method for automated detection of anatomical changes using cone-beam CT (CBCT) images of head and neck (H and N) patients. Vendor supplied CBCT software provides the user with the basic tools that allow for the visual assessment of the co-registration of the reference CT data and daily CBCT images. Currently there is no commercially available CBCT software that is capable of detecting anatomical changes and giving the clinician an early alert for the treatment plan adaptation. An algorithm was developed that can detect statistically significant changes at a given confidence level for H and N patients. This tool provides a reliable and automated early warning system that enables a 'just-in-time' adaptation of the treatment plan. A method was developed for automated detection of anatomical changes in H and N patients under radiation therapy treatment. The method utilized imaging CT data along with daily CBCT images to quantitatively compare the patient anatomy against the reference CT imaging data. The comparison algorithm was developed using a statistics-based approach as opposed to threshold-based methods. The algorithm allows for reliable detection of statistically significant changes in the observed patient anatomy at a given level of confidence. The method can be employed as an integral part of the adaptive radiotherapy framework. (authors)

61 RADIATION PROTECTION AND DOSIMETRY↗

Virtual tool mark generation for efficient striation analysis in forensic science

In 2009, a National Academy of Sciences report called for investigation into the scienti c basis behind tool mark comparisons (National Academy of Sciences, 2009). Answering this call, Chumbley et al. (2010) attempted to prove or disprove the hypothesis that tool marks are unique to a single tool. They developed a statistical algorithm that could, in most cases, discern matching and non-matching tool marks made at di erent angles by sequentially numbered screwdriver tips. Moreover, in the cases where the algorithm misinterpreted a pair of marks, an experienced forensics examiner could discern the correct outcome. While this research served to con rm the basic assumptions behind tool mark analysis, it also suggested that statistical analysis software could help to reduce the examiner&#x27;s workload. This led to a new tool mark analysis approach, introduced in this thesis, that relies on 3D scans of screwdriver tip and marked plate surfaces at the micrometer scale from an optical microscope. These scans are carefully cleaned to remove noise from the data acquisition process and assigned a coordinate system that mathematically de nes angles and twists in a natural way. The marking process is then simulated by using a 3D graphics software package to impart rotations to the tip and take the projection of the tip&#x27;s geometry in the direction of tool travel. The edge of this projection, retrieved from the 3D graphics software, becomes a virtual tool mark. Using this method, virtual marks are made at increments of 5 and compared to a scan of the evidence mark. The previously developed statistical package from Chumbley et al. (2010) performs the comparison, comparing the similarity of the geometry of both marks to the similarity that would occur due to random chance. The resulting statistical measure of the likelihood of the match informs the examiner of the angle of the best matching virtual mark, allowing the examiner to focus his/her mark analysis on a smaller range of angles. Preliminary results are quite promising. In a study with both sides of 6 screwdriver tips and 34 corresponding marks, the method distinguished known matches from known non-matches with zero false positive matches and only two matches mistaken for non-matches. For matches, it could predict the correct marking angle within 5-10 . Moreover, on a standard desktop computer, the virtual marking software is capable of cleaning 3D tip and plate scans in minutes and producing a virtual mark and comparing it to a real mark in seconds. These results support several of the professional conclusions of the tool mark analysis com- munity, including the idea that marks produced by the same tool only match if they are made at similar angles. The method also displays the potential to automate part of the comparison process, freeing the examiner to focus on other tasks, which is important in busy, backlogged crime labs. Finally, the method o ers the unique chance to directly link an evidence mark to the tool that produced it while reducing potential damage to the evidence.

42 ENGINEERING↗

Statistics of Storm Updraft Velocities from TWP-ICE Including Verification with Profiling Measurements

Comparisons between direct measurements and modeled values of vertical air motions in precipitating systems are complicated by differences in temporal and spatial scales. On one hand, vertically profiling radars more directly measure the vertical air motion but do not adequately capture full storm dynamics. On the other hand, vertical air motions retrieved from two or more scanning Doppler radars capture the full storm dynamics but require model constraints that may not capture all updraft features because of inadequate sampling, resolution, numerical constraints, and the fact that the storm is evolving as it is scanned by the radars. To investigate the veracity of radar-based retrievals, which can be used to verify numerically modeled vertical air motions, this article presents several case studies from storm events around Darwin, Northern Territory, Australia, in which measurements from a dual-frequency radar profiler system and volumetric radar-based wind retrievals are compared. While a direct comparison was not possible because of instrumentation location, an indirect comparison shows promising results, with volume retrievals comparing well to those obtained from the profiling system. This prompted a statistical analysis of an extended period of an active monsoon period during the Tropical Warm Pool International Cloud Experiment (TWP-ICE). Results show less vigorous deep convective cores with maximum updraft velocities occurring at lower heights than some cloudresolving modeling studies suggest. 1. Introduction The regionalization of global climate models has been a driver of demand for more complex convective parameterization schemes. A key readjustment of the modeled atmosphere

Collis, Scott↗

Feature extraction for structural dynamics model validation

This study focuses on defining and comparing response features that can be used for structural dynamics model validation studies. Features extracted from dynamic responses obtained analytically or experimentally, such as basic signal statistics, frequency spectra, and estimated time-series models, can be used to compare characteristics of structural system dynamics. By comparing those response features extracted from experimental data and numerical outputs, validation and uncertainty quantification of numerical model containing uncertain parameters can be realized. In this study, the applicability of some response features to model validation is first discussed using measured data from a simple test-bed structure and the associated numerical simulations of these experiments. issues that must be considered were sensitivity, dimensionality, type of response, and presence or absence of measurement noise in the response. Furthermore, we illustrate a comparison method of multivariate feature vectors for statistical model validation. Results show that the outlier detection technique using the Mahalanobis distance metric can be used as an effective and quantifiable technique for selecting appropriate model parameters. However, in this process, one must not only consider the sensitivity of the features being used, but also correlation of the parameters being compared.

42 ENGINEERING↗

In Situ Mg/Ca Measurements on Foraminifera: Comparison Between Laser Ablation Inductively Coupled Plasma Mass Spectrometry and Wavelength-Dispersive X-Ray Spectroscopy by Electron Probe Microanalyzer

We present a comparison of two different techniques: Laser Ablation Inductively Coupled Plasma Mass Spectrometry (LA-ICP-MS) and wavelength-dispersive X-Ray Spectroscopy by electron probe microanalyzer (EPMA) for obtaining Mg/Ca ratios in individual foraminifera shells. The goal is to assess the use of EPMA as an alternative technique for Mg/Ca analyses of single foraminiferal calcite shells. Foraminifera obtained from sediments (benthic, Uvigerina spp.) and from plankton tows (planktonic, Orbulina universa) were analyzed. All specimens were prepared in epoxy mounts and exposed in cross-section such that multiple high-resolution analyses could be completed on the shells using both techniques. We examined our data using statistical methods designed for the assessment and comparison of measurement techniques. In the case of Uvigerina, the mean difference for ratios obtained using EPMA and LA-ICP-MS is very small (−0.046 mmol mol−1) and scale independent. The Limits of Agreement (LoA, the standard deviation of the bias plus the mean bias) is [−0.315, 0.223] mmol mol−1. For samples with ratios lower than 13 mmol mol−1, we found a mean EPMA–LA-ICP-MS bias of −2.44 mmol mol−1 and a corresponding LoA of [−3.85, −1.04] mmol mol−1. For ratios higher than 13 mmol mol−1, there appears to be a scale dependent bias, meaning that the EPMA measured ratios become progressively larger than those of LA-ICP-MS as the Mg/Ca ratio increases, so the mean bias and LoA metrics are not meaningful. Results indicate that it is possible to use EPMA to collect Mg/Ca data, if the ratios are lower than ∼13 mmol mol−1.

EPMA↗

Low-level liquid cloud properties during ORACLES retrieved using airborne polarimetric measurements and a neural network algorithm

In this study we developed a neural network (NN) that can be used to retrieve cloud microphysical properties from multiangular and multispectral polarimetric remote sensing observations. This effort builds upon our previous work, which explored the sensitivity of neural network input, architecture, and other design requirements for this type of remote sensing problem. In particular this work introduces a framework for appropriately weighting total and polarized reflectances, which have vastly different magnitudes and measurement uncertainties. The NN is trained using an artificial training set and applied to research scanning polarimeter (RSP) data obtained during the ORACLES field campaign (ObseRvations of Aerosols above CLouds and their intEractionS). The polarimetric RSP observations are unique in that they observe the same cloud from a very large number of angles within a variety of spectral bands, resulting in a large dataset that can be explored rapidly with a NN approach. The usefulness of applying a NN to a dataset such as this one stems from the possibility of rapidly obtaining a retrieval that could be subsequently applied as a first guess for slower but more rigorous physical-based retrieval algorithms. This approach could be particularly advantageous for more complicated atmospheric retrievals – such as when an aerosol layer lies above clouds like in ORACLES. For RSP observations obtained during ORACLES 2016, comparisons between the NN and standard parametric polarimetric (PP) cloud retrieval give reasonable results for droplet effective radius (r(e): R=0.756, RMSE=1.74 µm) and cloud optical thickness (τ: R=0.950, RMSE=1.82). This level of statistical agreement is shown to be similar to comparisons between the two most well-established cloud retrievals, namely, the polarimetric and the bispectral total reflectance cloud retrievals. The NN retrievals from the ORACLES 2017 dataset result in retrievals of r(e) (R=0.54, RMSE=4.77 µm) and τ (R=0.785, RMSE=5.61) that behave much more poorly. In particular we found that our NN retrieval approach does not perform well for thin (τ<3), inhomogeneous, or broken clouds. We also found that correction for above-cloud atmospheric absorption improved the NN retrievals moderately – but retrievals without this correction still behaved similarly to existing cloud retrievals with a slight systematic offset.

SPECTRAL-RESOLUTION LIDAR↗

Quantification of the Impact of Nauru Island on ARM Measurements

Nauru Island at times generates low clouds that impact low-level cloud statistics and downwelling shortwave radiation measurements made at the Atmospheric Radiation Measurement Program (ARM) site. This study uses five years of Nauru data to quantify the island impact on the site measurements. The results indicate that the solar-heating-produced Nauru island effect occurs about 11% of the time during daylight hours. The island effect increases the 500-1000-m cloud base occurrence by 15%-20% when clouds occur, but because the island effect only occurs 11% of the time the overall increase in daylight low-cloud statistics is 2%, or 1% for 24-h statistics. In a similar way, the island effect produces a reduction of about 17% in the downwelling shortwave (SW) radiation across the daylight hours during the 11% of the time it occurs, an overall 2% daylight (or 1% for 24 h) average reduction. The island effect produces frequent positive downwelling SW cloud effects, in particular during the morning, which tend to somewhat mitigate the overall decrease in downwelling SW radiation that is due to clouds. This produces 17 W m22 less daylight average SW cloud effect relative to non-island-effect times, in particular for the convectively suppressed regime that typifies island-effect-producing conditions. For long-term overall statistical studies such as model and satellite comparisons, the2%daylight (or1%per 24 h) average increase in low-level cloud occurrence and decrease in downwelling SW are not of large concern as long as researchers are aware of them. For shorter-term studies, however, or those that separate data by conditions such as convectively active/suppressed regimes, the Nauru island effect can have significant impacts.

54 ENVIRONMENTAL SCIENCES↗

Direct Numerical Simulations of Acoustic Disturbances in Various Rectangular Nozzle Configurations

We perform Direct Numerical Simulations (DNS) to study the acoustic freestream disturbances radiating from the turbulent boundary layers along the contoured nozzle walls of a hypersonic wind tunnel with a rectangular test section. To begin with, the effects of the spanwise end walls are suppressed by confining the spanwise computational domain to a finite segment of the overall nozzle cross section and by imposing periodic boundary conditions across that spanwise domain. Besides providing a building-block configuration to reveal par- tial effects of the enclosed acoustic environment within the wind tunnel, these computations serve as a stepping stone toward the goal of fully-3D computations including the nozzle end walls. Building upon the earlier simulations of Deegan et al. (2018), we show that the computed acoustic characteristics in the spanwise periodic simulations are insensitive to changes in the grid resolution parameters (e.g., x+ and z+). This is relative to the previous simulations that involved a coarser resolution in the streamwise and wall-normal directions, especially up- stream of the test section. Furthermore, we outline a comparison of the pressure fluctuations induced by the turbulent boundary layers over the contoured nozzle walls and several other calculations involving boundary layers over a single flat plate at nearly the same value of the edge Mach number. We also show the impact of having only one turbulent boundary layer instead of two within the computational setup. Various flow statistics, including the first and second moments of the unsteady flow field, are computed for the different flow configurations. Good comparisons of the statistics of the nozzle-wall boundary-layer turbulence and of the freestream acoustic disturbances between the simulations of coarser and finer grids confirms DNS procedure, but also the insensitivity of noise characteristics t section to the inflow turbulence generation technique.

Hildebrand, Nathaniel↗

Direct Comparison between Bayesian and Frequentist Uncertainty Quantification for Nuclear Reactions

Until recently, uncertainty quantification in low energy nuclear theory was typically performed using frequentist approaches. However in the last few years, the field has shifted toward Bayesian statistics for evaluating confidence intervals. Although there are statistical arguments to prefer the Bayesian approach, no direct comparison is available. In this work, we compare, directly and systematically, the frequentist and Bayesian approaches to quantifying uncertainties in direct nuclear reactions. Starting from identical initial assumptions, we determine confidence intervals associated with the elastic and the transfer process for both methods, which are evaluated against data via a comparison of the empirical coverage probabilities. Expectedly, the frequentist approach is not as flexible as the Bayesian approach in exploring parameter space and often ends up in a different minimum. We also show that the two methods produce significantly different correlations. In the end, the frequentist approach produces significantly narrower uncertainties on the considered observables than the Bayesian. Lastly, our study demonstrates that the uncertainties on the reaction observables considered here within the Bayesian approach represent reality more accurately than the much narrower uncertainties obtained using the standard frequentist approach.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Evaluating Shallow Convection Parameterization Assumptions With a q t – w Quadrant Analysis

Abstract Uncertainties associated with the response of shallow clouds to global warming remain a great challenge for climate projection. Due to the small size of these clouds, parameterizations are required to represent them in both current‐ and next‐generation climate models. We present a quantitative evaluation of several important assumptions used in both mass‐flux and Assumed‐PDF Higher‐Order Closure parameterizations of shallow convection. We use large‐eddy simulations (LESs) of four different shallow convection regimes as benchmarks and apply a q t (total moisture)– w (vertical velocity) quadrant analysis technique to identify the “coherent structures” of moist and dry up/down‐drafts. The statistics of these coherent structures and the environment are then used to evaluate commonly used assumptions in mass‐flux parameterizations related to (a) the convective downdrafts and (b) the subplume variability of convective updrafts. For quantitative evaluation of the PDF closure used in AP‐HOCs, we perform offline calculations with the PDF closure used in the Cloud Layers Unified By Bi‐normals and the Simplified Higher‐Order Closure schemes by supplying the closure with statistical quantities directly calculated from the LESs. Then, the impact of parameterization assumptions embedded in the PDF closure in the same two categories is analyzed and understood through comparisons of the output from the closure with the statistics of the moist and dry up/down‐drafts from the LESs.

54 ENVIRONMENTAL SCIENCES↗

Accounting for Global Climate Model Projection Uncertainty in Modern Statistical Downscaling

Future climate change has emerged as a national and a global security threat. To carry out the needed adaptation and mitigation steps, a quantification of the expected level of climate change is needed, both at the global and the regional scale; in the end, the impact of climate change is felt at the local/regional level. An important part of such climate change assessment is uncertainty quantification. Decision and policy makers are not only interested in 'best guesses' of expected climate change, but rather probabilistic quantification (e.g., Rougier, 2007). For example, consider the following question: What is the probability that the average summer temperature will increase by at least 4 C in region R if global CO{sub 2} emission increases by P% from current levels by time T? It is a simple question, but one that remains very difficult to answer. It is answering these kind of questions that is the focus of this effort. The uncertainty associated with future climate change can be attributed to three major factors: (1) Uncertainty about future emission of green house gasses (GHG). (2) Given a future GHG emission scenario, what is its impact on the global climate? (3) Given a particular evolution of the global climate, what does it mean for a particular location/region? In what follows, we assume a particular GHG emission scenario has been selected. Given the GHG emission scenario, the current batch of the state-of-the-art global climate models (GCMs) is used to simulate future climate under this scenario, yielding an ensemble of future climate projections (which reflect, to some degree our uncertainty of being able to simulate future climate give a particular GHG scenario). Due to the coarse-resolution nature of the GCM projections, they need to be spatially downscaled for regional impact assessments. To downscale a given GCM projection, two methods have emerged: dynamical downscaling and statistical (empirical) downscaling (SDS). Dynamic downscaling involves configuring and running a regional climate model (RCM) nested within a given GCM projection (i.e., the GCM provides bounder conditions for the RCM). On the other hand, statistical downscaling aims at establishing a statistical relationship between observed local/regional climate variables of interest and synoptic (GCM-scale) climate predictors. The resulting empirical relationship is then applied to future GCM projections. A comparison of the pros and cons of dynamical versus statistical downscaling is outside the scope of this effort, but has been extensively studied and the reader is referred to Wilby et al. (1998); Murphy (1999); Wood et al. (2004); Benestad et al. (2007); Fowler et al. (2007), and references within those. The scope of this effort is to study methodology, a statistical framework, to propagate and account for GCM uncertainty in regional statistical downscaling assessment. In particular, we will explore how to leverage an ensemble of GCM projections to quantify the impact of the GCM uncertainty in such an assessment. There are three main component to this effort: (1) gather the necessary climate-related data for a regional SDS study, including multiple GCM projections, (2) carry out SDS, and (3) assess the uncertainty. The first step is carried out using tools written in the Python programming language, while analysis tools were developed in the statistical programming language R; see Figure 1.

58 GEOSCIENCES↗