Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

A Faster Parallel Algorithm and Efficient Multithreaded Implementations for Evaluating Betweenness Centrality on Massive Datasets

We present a new lock-free parallel algorithm for computing betweenness centralityof massive small-world networks. With minor changes to the data structures, ouralgorithm also achieves better spatial cache locality compared to previous approaches. Betweenness centrality is a key algorithm kernel in HPCS SSCA#2, a benchmark extensively used to evaluate the performance of emerging high-performance computing architectures for graph-theoretic computations. We design optimized implementations of betweenness centrality and the SSCA#2 benchmark for two hardware multithreaded systems: a Cray XMT system with the Threadstorm processor, and a single-socket Sun multicore server with the UltraSPARC T2 processor. For a small-world network of 134 million vertices and 1.073 billion edges, the 16-processor XMT system and the 8-core Sun Fire T5120 server achieve TEPS scores (an algorithmic performance count for the SSCA#2 benchmark) of 160 million and 90 million respectively, which corresponds to more than a 2X performance improvement over the previous parallel implementations. To better characterize the performance of these multithreaded systems, we correlate the SSCA#2 performance results with data from the memory-intensive STREAM and RandomAccess benchmarks. Finally, we demonstrate the applicability of our implementation to analyze massive real-world datasets by computing approximate betweenness centrality for a large-scale IMDb movie-actor network.

97 MATHEMATICS AND COMPUTING↗

Different long-term trends of extra-tropical cyclones and windstorms in ERA-20C and NOAA-20CR reanalyses: Extra-tropical cyclones and windstorms in 20th century reanalyses

NOAA 20th century and ERA-20C reanalysis datasets are evaluated regarding the representation of extra-tropical cyclones and windstorms over the Northern and Southern Hemisphere during the respective 6-month winter seasons. The results indicate substantial differences in low-frequency variability between the two datasets – especially in the first half of the 20th century – expressed in different signs and/or magnitudes of long-term trends. This is hampering a reliable analysis of real long-term trends of cyclone and windstorm activity. However, higher-frequency variability is in good agreement between both datasets especially for the Northern Hemisphere.

54 ENVIRONMENTAL SCIENCES↗

A Faster Parallel Algorithm and Efficient Multithreaded Implementations for Evaluating Betweenness Centrality on Massive Datasets

We present a new lock-free parallel algorithm for computing betweenness centrality of massive small-world networks. With minor changes to the data structures, our algorithm also achieves better spatial cache locality compared to previous approaches. Betweenness centrality is a key algorithm kernel in the HPCS SSCA#2 Graph Analysis benchmark, which has been extensively used to evaluate the performance of emerging high-performance computing architectures for graph-theoretic computations. We design optimized implementations of betweenness centrality and the SSCA#2 benchmark for two hardware multithreaded systems: a Cray XMT system with the ThreadStorm processor, and a single-socket Sun multicore server with the UltraSparc T2 processor. For a small-world network of 134 million vertices and 1.073 billion edges, the 16-processor XMT system and the 8-core Sun Fire T5120 server achieve TEPS scores (an algorithmic performance count for the SSCA#2 benchmark) of 160 million and 90 million respectively, which corresponds to more than a 2X performance improvement over the previous parallel implementations. To better characterize the performance of these multithreaded systems, we correlate the SSCA#2 performance results with data from the memory-intensive STREAM and RandomAccess benchmarks. Finally, we demonstrate the applicability of our implementation to analyze massive real-world datasets by computing approximate betweenness centrality for a large-scale IMDb movie-actor network.

architecture comparison↗

Evaluation of Visual Analytics Environments: The Road to the Visual Analytics Science and Technology Challenge Evaluation Methodology

The evaluation of visual analytics environments was a topic in Illuminating the Path [Thomas 2005] as a critical aspect of moving research into practice. For a thorough understanding of the utility of the systems available, evaluation not only involves assessing the visualizations, interactions or data processing algorithms themselves, but also the complex processes that a tool is meant to support (such as exploratory data analysis and reasoning, communication through visualization, or collaborative data analysis [Lam 2012; Carpendale 2007]). Researchers and practitioners in the field have long identified many of the challenges faced when planning, conducting, and executing an evaluation of a visualization tool or system [Plaisant 2004]. Evaluation is needed to verify that algorithms and software systems work correctly and that they represent improvements over the current infrastructure. Additionally to effectively transfer new software into a working environment, it is necessary to ensure that the software has utility for the end-users and that the software can be incorporated into the end-user’s infrastructure and work practices. Evaluation test beds require datasets, tasks, metrics and evaluation methodologies. As noted in [Thomas 2005] it is difficult and expensive for any one researcher to setup an evaluation test bed so in many cases evaluation is setup for communities of researchers or for various research projects or programs. Examples of successful community evaluations can be found [Chinchor 1993; Voorhees 2007; FRGC 2012]. As visual analytics environments are intended to facilitate the work of human analysts, one aspect of evaluation needs to focus on the utility of the software to the end-user. This requires representative users, representative tasks, and metrics that measure the utility to the end-user. This is even more difficult as now one aspect of the test methodology is access to representative end-users to participate in the evaluation. In many cases the sensitive nature of data and tasks and difficult access to busy analysts puts even more of a burden on researchers to complete this type of evaluation. User-centered design goes beyond evaluation and starts with the user [Beyer 1997, Shneiderman 2009]. Having some knowledge of the type of data, tasks, and work practices helps researchers and developers know the correct paths to pursue in their work. When access to the end-users is problematic at best and impossible at worst, user-centered design becomes difficult. Researchers are unlikely to go to work on the type of problems faced by inaccessible users. Commercial vendors have difficulties evaluating and improving their products when they cannot observe real users working with their products. In well-established fields such as web site design or office software design, user-interface guidelines have been developed based on the results of empirical studies or the experience of experts. Guidelines can speed up the design process and replace some of the need for observation of actual users [heuristics review references]. In 2006 when the visual analytics community was initially getting organized, no such guidelines existed. Therefore, we were faced with the problem of developing an evaluation framework for the field of visual analytics that would provide representative situations and datasets, representative tasks and utility metrics, and finally a test methodology which would include a surrogate for representative users, increase interest in conducting research in the field, and provide sufficient feedback to the researchers so that they could improve their systems.

evaluation↗

Impact of Misclassification Rates on Compression Efficiency of Red Blood Cell Images of Malaria Infection Using Deep Learning

Malaria is a severe public health problem worldwide, with some developing countries being most affected. Reliable remote diagnosis of malaria infection will benefit from efficient compression of high-resolution microscopic images. This paper addresses a lossless compression of malaria-infected red blood cell images using deep learning. Specifically, we investigate a practical approach where images are first classified before being compressed using stacked autoencoders. We provide probabilistic analysis on the impact of misclassification rates on compression performance in terms of the information-theoretic measure of entropy. We then use malaria infection image datasets to evaluate the relations between misclassification rates and actually obtainable compressed bit rates using Golomb–Rice codes. Simulation results show that the joint pattern classification/compression method provides more efficient compression than several mainstream lossless compression techniques, such as JPEG2000, JPEG-LS, CALIC, and WebP, by exploiting common features extracted by deep learning on large datasets. This study provides new insight into the interplay between classification accuracy and compression bitrates. The proposed compression method can find useful telemedicine applications where efficient storage and rapid transfer of large image datasets is desirable.

59 BASIC BIOLOGICAL SCIENCES↗

TU-H-207A-03: CT Hounsfield Unit Accuracy: Effect of Beam Hardening On Phantom and Clinical Whole-Body CT Images

Purpose: To assess the effect of beam hardening on measured CT HU values. Methods: An anthropomorphic knee phantom was scanned with the CT component of a GE Discovery 690 PET/CT scanner (120kVp, 300mAs, 40?0.625mm collimation, pitch=0.984, FOV=500mm, matrix=512?512) with four different scan setups, each of which induces different degrees of beam hardening by introducing additional attenuation media into the field of view. Homogeneous voxels representing “soft tissue” and “bone” were segmented by HU thresholding followed by a 3D morphological erosion operation which removes the non-homogenous voxels located on the interface of thresholded tissue mask. HU values of segmented “soft tissue” and “bone” were compared.Additionally, whole-body CT data with coverage from the skull apex to the end of toes were retrospectively retrieved from seven PET/CT exams to evaluate the effect of beam hardening in vivo. Homogeneous bone voxels were segmented with the same method previously described. Total In-Slice Attenuation (TISA) for each CT slice, defined as the summation of HU values over all voxels within a CT slice, was calculated for all slices of the seven whole-body CT datasets and evaluated against the mean HU values of homogeneous bone voxels within that slice. Results: HU values measured from the phantom showed that while “soft tissue” HU values were unaffected, added attenuation within the FOV caused noticeable decreases in the measured HU values of “bone” voxels. A linear relationship was observed between bone HU and TISA for slices of the torso and legs, but not of the skull. Conclusion: Beam hardening effect is not an issue of concern for voxels with HU in the soft tissue range, but should not be neglected for bone voxels. A linear relationship exists between bone HU and the associated TISA in non-skull CT slices, which can be exploited to develop a correction strategy.

60 APPLIED LIFE SCIENCES↗

SU-E-T-174: Evaluation of the Optimal Intensity Modulated Radiation Therapy Plans Done On the Maximum and Average Intensity Projection CTs

Purpose: To determine the difference in coverage between plans done on average intensity projection and maximum intensity projection CT data sets for lung patients and to establish correlations between different factors influencing the coverage. Methods: For six lung cancer patients, 10 phases of equal duration through the respiratory cycle, the maximum and average intensity projections (MIP and AIP) from their 4DCT datasets were obtained. MIP and AIP datasets had three GTVs delineated (GTVaip — delineated on AIP, GTVmip — delineated on MIP and GTVfus — delineated on each of the 10 phases and summed up). From the each GTV, planning target volumes (PTV) were then created by adding additional margins. For each of the PTVs an IMRT plan was developed on the AIP dataset. The plans were then copied to the MIP data set and were recalculated. Results: The effective depths in AIP cases were significantly smaller than in MIP (p < 0.001). The Pearson correlation coefficient of r = 0.839 indicates strong degree of positive linear relationship between the average percentage difference in effective depths and average PTV coverage on the MIP data set. The V2 0 Gy of involved lung depends on the PTV coverage. The relationship between PTVaip mean CT number difference and PTVaip coverage on MIP data set gives r = 0.830. When the plans are produced on MIP and copied to AIP, r equals −0.756. Conclusion: The correlation between the AIP and MIP data sets indicates that the selection of the data set for developing the treatment plan affects the final outcome (cases with high average percentage difference in effective depths between AIP and MIP should be calculated on AIP). The percentage of the lung volume receiving higher dose depends on how well PTV is covered, regardless of on which set plan is done.

60 APPLIED LIFE SCIENCES↗

Mesoscale evaluation of AMPS using AWARE radar observations of a wind and precipitation event over the Ross Island region of Antarctica

Surface, upper-air, and radar observations are used to assess the performance of the Antarctic Mesoscale Prediction System (AMPS) in simulating the mesoscale aspects of a wind and precipitation event over the Ross Island region of Antarctica that spanned January 16–20, 2016. The observations, collected during the Atmospheric Radiation Measurement (ARM) West Antarctic Radiation Experiment (AWARE), provide a unique dataset for evaluating AMPS, especially the radar observations that facilitate a three-dimensional depiction of winds and precipitation. Comparisons of AMPS forecast data with surface meteorology, balloon-sounding, and profiling radar observations at and above sites near McMurdo Station reveal a mixture of similarities and differences. A generally southerly flow is evident at low levels in both the AMPS simulations and observed Doppler radial velocities. AMPS winds are comparable to those observed at the surface and aloft in terms of magnitude, direction, and timing but the strongest simulated southerly flow is displaced eastward relative to the observations. AMPS-simulated reflectivity over the broader Ross Island region is more limited in areal extent and smaller in magnitude than observed by a scanning Doppler radar. Three episodes of surface precipitation are observed near McMurdo Station over the five-day event with peak rates of ~3 mm h -1 and a total accumulation of ~22 mm. However, AMPS produces no surface precipitation at that location over the five-day event due to a low-level dry bias in the forecasts. Herein, the results show the first observationally based three-dimensional understanding of meteorology in the Ross Island region.

AMPS↗

Re-evaluating probable maximum precipitation estimates: sensitivity to transposition domains and storm rotation using modern datasets

This study examines the sensitivity of Probable Maximum Precipitation (PMP) estimates to key methodological decisions embedded in the legacy approach adopted in the U.S. National Weather Service Hydrometeorological Reports No. 51 and No. 52. Although widely used for infrastructure design and risk regulation, fundamental aspects of PMP estimation—such as storm sample size, transposition domain, maximization procedures, and storm rotation—remain poorly constrained and lack formal guidance. Using the Red Rock watershed in Iowa as a case study, and leveraging the 2002–2023 NOAA Analysis of Record for Calibration (AORC) precipitation dataset, we systematically evaluate how each methodological choice, individually and in combination, influences PMP estimates. Our findings demonstrate that PMP is not a fixed physical upper bound but rather a modeling construct shaped heavily by user-defined assumptions. Notably, PMP values derived from modern gridded rainfall datasets can be substantially higher than the legacy estimate used in the original spillway design for Red Rock Dam. Decisions regarding storm sample size, domain extent, climatological window, and particularly storm rotation all contributed to higher PMP estimates. Storm rotation alone—a loosely constrained element in the current PMP practice—can amplify PMP by more than 25%. These results reveal the lack of standardized bounds in current PMP workflows and the need for systematic sensitivity and uncertainty analysis. As PMP estimation shifts toward probabilistic approaches, incorporating physically meaningful storm attributes will be key to developing more transparent, defensible methods for dam safety and climate-resilient infrastructure.

Probable maximum precipitation↗

Leveraging large language models to address data scarcity in machine learning for graphene synthesis

Machine learning in experimental materials science faces significant challenges due to the scarcity of data, which are costly and time-consuming to generate, particularly when relying on in-house experiments. Literature data mining offers a potential solution but introduces issues like mixed data quality, inconsistent formats, and non-uniform reporting of synthesis parameters, resulting in partially missing and heterogeneous features across the dataset. Here, we propose data imputation and feature engineering methods that employ pre-trained large language models (LLMs) to enhance machine learning performance on scarce, heterogeneous datasets, demonstrated on graphene CVD synthesis data and the ML-HydPARK hydrogen storage dataset. GPT models perform data imputation via tailored prompting and semantic normalization of inconsistently reported features through embeddings, for example, to harmonize the complex nomenclature of CVD substrates. Beyond yielding more diverse and richer feature representations than traditional methods such as K-nearest neighbors (KNN) and Multivariate Imputation by Chained Equations (MICE), LLM-based data imputation is evaluated against dataset characteristics and prompting strategies. We vary the level of autonomy granted to the LLM, from generic prompting that leverages pre-trained knowledge for autonomous data generation to data-informed prompting that constrains outputs using target-specific information, and demonstrate which level of autonomy yields superior imputation performance across datasets and feature types. The proposed data engineering methods markedly improve downstream performance; for example, in graphene layer number classification using a support vector machine (SVM), binary accuracy increases from 39% to 65% and ternary accuracy from 52% to 72%. Fine-tuning experiments on both datasets show that combining our proposed LLM-based data imputation and feature encoding methods with numerical machine learning predictors outperforms standalone fine-tuned LLM predictors in data-scarce settings. The proposed strategies emphasize data enhancement techniques rather than refining learning architectures or regularizing loss functions, offering a broadly applicable framework for improving machine learning performance on scarce, inhomogeneous datasets.

Chemical vapor deposition↗

Use of volumetric features for temporal comparison of mass lesions in full field digital mammograms

Purpose: Temporal comparison of lesions might improve classification between benign and malignant lesions in full-field digital mammograms (FFDM). The authors compare the use of volumetric features for lesion classification, which are computed from dense tissue thickness maps, to the use of mammographic lesion area. Use of dense tissue thickness maps for lesion characterization is advantageous, since it results in lesion features that are invariant to acquisition parameters. Methods: The dataset used in the analysis consisted of 60 temporal mammogram pairs comprising 120 mediolateral oblique or craniocaudal views with a total of 65 lesions, of which 41 were benign and 24 malignant. The authors analyzed the performance of four volumetric features, area, and four other commonly used features obtained from temporal mammogram pairs, current mammograms, and prior mammograms. The authors evaluated the individual performance of all features and of different feature sets. The authors used linear discriminant analysis with leave-one-out cross validation to classify different feature sets. Results: Volumetric features from temporal mammogram pairs achieved the best individual performance, as measured by the area under the receiver operating characteristic curve (A{sub z} value). Volume change (A{sub z} = 0.88) achieved higher A{sub z} value than projected lesion area change (A{sub z} = 0.78) in the temporal comparison of lesions. Best performance was achieved with a set that consisted of a set of features extracted from the current exam combined with four volumetric features representing changes with respect to the prior mammogram (A{sub z} = 0.90). This was significantly better (p = 0.005) than the performance obtained using features from the current exam only (A{sub z} = 0.77). Conclusions: Volumetric features from temporal mammogram pairs combined with features from the single exam significantly improve discrimination of benign and malignant lesions in FFDM mammograms compared to using only single exam features. In the comparison with prior mammograms, use of volumetric change may lead to better performance than use of lesion area change.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

High bias machine learning for antineutrino-based safeguards for small reactors

The statistical methods used for antineutrino detection will need to be improved to effectively monitor the inventory of next-generation nuclear reactors. In this sensitivity study, we evaluate machine learning models compared to previously used statistical approaches to identify diversion scenarios in a simulated Advanced Fast Reactor (AFR)-100. A chi-square goodness-of-fit technique, which individually compares the simulated antineutrino yields to the expected antineutrino yield, resulted in precise but low diversion detection probability. Various support vector machine (SVM) models were applied with diverse training datasets to evaluate the robustness of the method towards unexpected or “unseen” diversion scenarios. Furthermore, our results indicate that while the SVM models significantly improved the detection probability of near-field antineutrino-based safeguards, up to a probability of ~0.04, for the simulated small reactor, the detection system still needs improvements to reach the 0.2 detection limit established by the International Atomic Energy Agency.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Machine learning and atomic layer deposition: Predicting saturation times from reactor growth profiles using artificial neural networks

In this work, we explore the application of deep neural networks to the optimization of atomic layer deposition (ALD) processes. In particular, we focus on a one-shot optimization problem, where we try to predict the optimal dose time that leads to saturation everywhere in the reactor based on thickness values measured at different points of an ALD reactor after a single trial growth. In order to tackle this problem, we introduce a dataset designed to train neural networks to predict saturation times based on these inputs for a cross-flow ALD reactor. Here, we then explore the predictive ability of artificial neural networks of different depths and sizes using a separate testing dataset to evaluate their accuracies. The results obtained show that networks trained using stochastic gradient descent methods can accurately predict saturation times without requiring any additional information on the surface kinetics. This provides a viable approach to minimize the number of experiments required to optimize new ALD processes in a known reactor, and it highlights the way machine learning can be leveraged for thin film growth and manufacturing. While the datasets and training procedure depend on the reactor geometry, the trained neural networks provide a general surrogate model connecting thickness values and trial dose times with optimal saturation times that can be reused for different ALD processes within the same reactor.

36 MATERIALS SCIENCE↗

Low Latency Flux and Concentration Datasets in Support of Greenhouse Gas Monitoring Based on NASA's GEOS Modeling and Data Assimilation System

We present efforts to develop space-based greenhouse gas monitoring systems that can provide low latency information and traceability to independent observations. Through support from its Carbon Monitoring System program, NASA has developed the capability to assimilate XCO2 retrievals from the Orbiting Carbon Observatory, 2 (OCO-2) into the Goddard Earth Observing System (GEOS) Constituent Data Assimilation System (CoDAS) to create gap-filled, three-dimensional (3D) estimates of CO2 mixing ratio. When OCO-2 data are not available, concentration fields are further informed by a bottom-up flux package based on remotely sensed fire radiative power, nighttime lights, and vegetation reflectance combined with estimates of atmospheric growth rate based on surface in situ data. The 3D nature of this dataset supports evaluation with independent aircraft data, helping to ensure transparency of remotely sensed data products. These quasi-operational data are currently produced 2-3 months behind real time and are distributed via NASA and international dashboard services to a variety of end users. In this presentation, we provide an overview of the system as well as remaining data gaps and modeling challenges. We also highlight the application of this dataset for detecting emissions anomalies associated with COVID-19 and comparing against independent emissions estimates. Finally, we highlight a new NASA initiative called the Earth Information System (EIS), which aims to support open science and applications by leveraging emerging cloud computing capabilities to increase access to NASA’s greenhouse gas datasets, opportunities for co-development, and transparency in methods for analysis and flux attribution.

Lesley Ott↗

The SPoRT-WRF: Evaluating the Impact of NASA Datasets on Convective Forecasts

Short-term Prediction Research and Transition (SPoRT) seeks to improve short-term, regional weather forecasts using unique NASA products and capabilities SPoRT has developed a unique, real-time configuration of the NASA Unified Weather Research and Forecasting (WRF)WRF (ARW) that integrates all SPoRT modeling research data: (1) 2-km SPoRT Sea Surface Temperature (SST) Composite, (2) 3-km LIS with 1-km Greenness Vegetation Fraction (GVFs) (3) 45-km AIRS retrieved profiles. Transitioned this real-time forecast to NOAA's Hazardous Weather Testbed (HWT) as deterministic model at Experimental Forecast Program (EFP). Feedback from forecasters/participants and internal evaluation of SPoRT-WRF shows a cool, dry bias that appears to suppress convection likely related to methodology for assimilation of AIRS profiles Version 2 of the SPoRT-WRF will premier at the 2012 EFP and include NASA physics, cycling data assimilation methodology, better coverage of precipitation forcing, and new GVFs

Zavodsky, Bradley↗

Application of artificial intelligence methods in the international roughness index prediction of rigid and composite pavements: a systematic review

The International Roughness Index (IRI) is a widely adopted metric for quantifying pavement roughness, directly influencing vehicle safety, ride comfort, and overall roadway performance. In recent years, the use of Machine Learning (ML) models for IRI prediction has gained momentum, with the goal of improving the allocation of maintenance and rehabilitation resources by enabling accurate assessments of pavement conditions. Most prior reviews, however, have concentrated on flexible pavements, leaving a notable gap regarding rigid and composite pavements. To address this gap, the present study conducts a systematic review of Artificial Intelligence (AI) methods applied to IRI prediction for rigid and composite pavements. Literature published between 2004 and 2025 is synthesized to highlight prevailing trends, methodological contributions, and directions for future research. Particular attention is given to the types of models employed, the datasets used for training and validation, and the role of input variables and data-processing strategies. Across the included studies, ensemble learning methods (especially gradient boosting variants such as XGBoost), artificial neural networks, and hybrid architectures frequently achieved high predictive skill, with several models reporting test-set coefficients of determination approaching 0.9–0.96, indicating strong potential for capturing the influence of traffic, pavement structure, and climatic factors. Since these results are obtained from heterogeneous datasets and evaluation protocols, they are interpreted qualitatively rather than as strict cross-study rankings. Analysis of input variables revealed that pavement age and initial IRI were included in 91% (21 of 23) and 78% (18 of 23) of studies, respectively. Climatic variables such as the freezing index appeared in 57% (13 of 23), while traffic-related factors were considered in 65% (15 of 23). The findings underscore the importance of standardized, high-quality datasets, such as those from the Long-Term Pavement Performance (LTPP) program, along with data consistency, model interpretability, computational efficiency, and replicability in enhancing IRI prediction. Future research should focus on incorporating input variable selection techniques to identify the most influential predictors, thereby improving accuracy and robustness. Integrating these approaches with advanced non-linear data-driven models, coupled with robust hyperparameter optimization, holds considerable promise for strengthening the reliability of IRI prediction and supporting resilient pavement management strategies.

42 ENGINEERING↗

A scaling transformation for classifier output based on likelihood ratio: Applications to a CAD workstation for diagnosis of breast cancer

Purpose: The authors developed scaling methods that monotonically transform the output of one classifier to the ''scale'' of another. Such transformations affect the distribution of classifier output while leaving the ROC curve unchanged. In particular, they investigated transformations between radiologists and computer classifiers, with the goal of addressing the problem of comparing and interpreting case-specific values of output from two classifiers. Methods: Using both simulated and radiologists' rating data of breast imaging cases, the authors investigated a likelihood-ratio-scaling transformation, based on ''matching'' classifier likelihood ratios. For comparison, three other scaling transformations were investigated that were based on matching classifier true positive fraction, false positive fraction, or cumulative distribution function, respectively. The authors explored modifying the computer output to reflect the scale of the radiologist, as well as modifying the radiologist's ratings to reflect the scale of the computer. They also evaluated how dataset size affects the transformations. Results: When ROC curves of two classifiers differed substantially, the four transformations were found to be quite different. The likelihood-ratio scaling transformation was found to vary widely from radiologist to radiologist. Similar results were found for the other transformations. Our simulations explored the effect of database sizes on the accuracy of the estimation of our scaling transformations. Conclusions: The likelihood-ratio-scaling transformation that the authors have developed and evaluated was shown to be capable of transforming computer and radiologist outputs to a common scale reliably, thereby allowing the comparison of the computer and radiologist outputs on the basis of a clinically relevant statistic.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

NASA Downscaling Project: Final Report

A team of researchers from NASA Ames Research Center, Goddard Space Flight Center, the Jet Propulsion Laboratory, and Marshall Space Flight Center, along with university partners at UCLA, conducted an investigation to explore whether downscaling coarse resolution global climate model (GCM) predictions might provide valid insights into the regional impacts sought by decision makers. Since the computational cost of running global models at high spatial resolution for any useful climate scale period is prohibitive, the hope for downscaling is that a coarse resolution GCM provides sufficiently accurate synoptic scale information for a regional climate model (RCM) to accurately develop fine scale features that represent the regional impacts of a changing climate. As a proxy for a prognostic climate forecast model, and so that ground truth in the form of satellite and in-situ observations could be used for evaluation, the MERRA and MERRA - 2 reanalyses were used to drive the NU - WRF regional climate model and a GEOS - 5 replay. This was performed at various resolutions that were at factors of 2 to 10 higher than the reanalysis forcing. A number of experiments were conducted that varied resolution, model parameterizations, and intermediate scale nudging, for simulations over the continental US during the period from 2000 - 2010. The results of these experiments were compared to observational datasets to evaluate the output.

dynamical downsizing↗