Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

A novel conformity index for intensity modulated radiation therapy plan evaluation

Purpose: Intensity modulated radiation therapy (IMRT) has gained popularity in the treatment of cancers. Manual evaluation of IMRT plans for head-and-neck cancers has been especially challenging necessitating efficient and objective assessment tools. In this work, the authors address this issue by developing a personalized conformity index (CI) for comparison of IMRT plans for head-and-neck cancers and evaluating its plan quality discerning power in comparison with other widely used CIs. Methods: A two-dimensional CI with dose and distance incorporated (CI{sub DD}) was developed using the MATLAB program language, to quantify the planning target volume (PTV) coverage. Valuable information contained in the digital imaging and communication in medicine (DICOM) RT objects were harvested for computation of each of the CI{sub DD} components. Apart from the dose penalty factor, a distance-based exponential function was employed by varying the penalty weight associated with the location of cold spots within the PTV. With the goal of deriving a customized penalty factor, the distances between individual pixel and its nearest PTV boundary was found. Using the exponential function, the impact of distance penalty was substantially larger for cold spots closer to the PTV centroid but petered out quickly wherever they were situated in the vicinity of PTV border. In order to evaluate the CI{sub DD} scoring system, three CT image data sets of nasopharyngeal carcinoma (NPC) patients were collected. Ten IMRT plans with degrading qualities were generated from each dataset and were ranked based on CI{sub DD} and other existing indices. The coefficient of variance was calculated for each dataset to compare the degree of variation. Results: The CI{sub DD} scoring system that considered spatial importance of each voxel within the PTV was successfully developed. The results demonstrated that the CI{sub DD} including four discrete factors could provide accurate rankings of plan quality by examining the relative importance of each cold spot within the PTVs. Apart from the dose penalty factor, a distance-based exponential function was employed taking the specific tumor geometry into account. Compared with other commonly used CIs, the CI{sub DD} resulted in the largest coefficient of variance among the ten IMRT plans for each dataset, indicating that its discerning power was the best among the CIs being compared. Conclusions: The CI{sub DD} scoring system was successfully developed to incorporate patient-specific spatial dose information and provide a geometry-based physical index for comparison of IMRT plans for head-and-neck cancers. By taking individual tumor geometry into account, the superiority of CI{sub DD} in plan discerning power was demonstrated. The use of CI{sub DD} could provide an effective means of benchmarking performance, reducing treatment plan variability, and advancing the quality of current IMRT planning.

60 APPLIED LIFE SCIENCES↗

Comparison of recent SnIa datasets

We rank the six latest Type Ia supernova (SnIa) datasets (Constitution (C), Union (U), ESSENCE (Davis) (E), Gold06 (G), SNLS 1yr (S) and SDSS-II (D)) in the context of the Chevalier-Polarski-Linder (CPL) parametrization w(a) = w{sub 0}+w{sub 1}(1−a), according to their Figure of Merit (FoM), their consistency with the cosmological constant (ΛCDM), their consistency with standard rulers (Cosmic Microwave Background (CMB) and Baryon Acoustic Oscillations (BAO)) and their mutual consistency. We find a significant improvement of the FoM (defined as the inverse area of the 95.4% parameter contour) with the number of SnIa of these datasets ((C) highest FoM, (U), (G), (D), (E), (S) lowest FoM). Standard rulers (CMB+BAO) have a better FoM by about a factor of 3, compared to the highest FoM SnIa dataset (C). We also find that the ranking sequence based on consistency with ΛCDM is identical with the corresponding ranking based on consistency with standard rulers ((S) most consistent, (D), (C), (E), (U), (G) least consistent). The ranking sequence of the datasets however changes when we consider the consistency with an expansion history corresponding to evolving dark energy (w{sub 0},w{sub 1}) = (−1.4,2) crossing the phantom divide line w = −1 (it is practically reversed to (G), (U), (E), (S), (D), (C)). The SALT2 and MLCS2k2 fitters are also compared and some peculiar features of the SDSS-II dataset when standardized with the MLCS2k2 fitter are pointed out. Finally, we construct a statistic to estimate the internal consistency of a collection of SnIa datasets. We find that even though there is good consistency among most samples taken from the above datasets, this consistency decreases significantly when the Gold06 (G) dataset is included in the sample.

79 ASTRONOMY AND ASTROPHYSICS↗

The SUMup Dataset: Compiled Measurements of Surface Mass Balance Components over Ice Sheets and Sea Ice with Analysis over Greenland

Increasing atmospheric temperatures over ice cover affect surface processes, including melt, snowfall, and snow density. Here, we present the Surface Mass Balance and Snow on Sea Ice Working Group (SUMup) dataset, a standardized dataset of Arctic and Antarctic observations of surface mass balance components. The July 2018 SUMup dataset consists of three subdatasets, snow/firn density (https://doi.org/10.18739/A2JH3D23R), at least near-annually resolved snow accumulation on land ice (https://doi.org/10.18739/A2DR2P790), and snow depth on sea ice (https://doi.org/10.18739/A2WS8HK6X), to monitor change and improve estimates of surface mass balance. The measurements in this dataset were compiled from field notes, papers, technical reports, and digital files. SUMup is a compiled, community-based dataset that can be and has been used to evaluate modeling efforts and remote sensing retrievals. Active submission of new or past measurements is encouraged. Analysis of the dataset shows that Greenland Ice Sheet density measurements in the top 1m do not show a strong relationship with annual temperature. At Summit Station, Greenland, accumulation and surface density measurements vary seasonally with lower values during summer months. The SUMup dataset is a dynamic, living dataset that will be updated and expanded for community use as new measurements are taken and new processes are discovered and quantified.

Montgomery, Lynn↗

Evaluation of the Event Driven Phenology Model Coupled with the VegET Evapotranspiration Model Through Comparisons with Reference Datasets in a Spatially Explicit Manner

A new model coupling scheme with remote sensing data assimilation was developed for estimation of daily actual evapotranspiration (ET). The scheme represents a mix of the VegET, a physically based model to estimate ET from a water balance, and an event driven phenology model (EDPM), where the EDPM is an empirically derived crop specific model capable of producing seasonal trajectories of canopy attributes. In this experiment, the scheme was deployed in a spatially explicit manner within the croplands of the Northern Great Plains. The evaluation was carried out using 2007-2009 land surface forcing data from the North American Land Data Assimilation System (NLDAS) and crop maps derived from remotely sensed data of NASA's Moderate Resolution Imaging Spectroradiometer (MODIS). We compared the canopy parameters produced by the phenology model with normalized difference vegetation index (NDVI) data derived from the MODIS nadir bi-directional reflectance distribution function (BRDF) adjusted reflectance (NBAR) product. The expectations of the EDPM performance in prognostic mode were met, producing determination coefficient (r2) of 0.8 +/-.0.15. Model estimates of NDVI yielded root mean square error (RMSE) of 0.1 +/-.0.035 for the entire study area. Retrospective correction of canopy dynamics with MODIS NDVI brought the errors down to just below 10% of observed data range. The ET estimates produced by the coupled scheme were compared with ones from the MODIS land product suite. The expected r2=0.7 +/-.15 and RMSE = 11.2 +/-.4 mm per 8 days were met and even exceeded by the coupling scheme0 functioning in both prognostic and retrospective modes. Minor setbacks of the EDPM and VegET performance (r2 about 0.5 and additional 30 % of RMSR) were found on the peripheries of the study area and attributed to the insufficient EDPM training and to spatially varying accuracy of crop maps. Overall the experiment provided sufficient evidence of soundness and robustness of the EDPM and VegET coupling scheme, assuring its potential for spatially explicit applications.

Kovalskyy, V.↗

Visual detection of spatial contrast patterns: evaluation of five simple models

The ModelFest Phase One dataset is a collection of luminance contrast thresholds for 43 two-dimensional monochromatic spatial patterns confined to an area of approximately two by two degrees. These data were collected by a collaboration among twelve laboratories, and were designed to provide a common database for calibration and testing of spatial vision models. Here I report fits of the ModelFest data with five models: Peak Contrast, Contrast Energy, Generalized Energy, a Gabor Channels model, and a Discrete Cosine Transform model. The Gabor Channels model provides the best fit, though the other, simpler models, with the exception of Peak Contrast, provide remarkably good fits as well. Though there are clear individual differences, regularities in the data suggest the possibility of constructing a standard observer for spatial vision. c2000 Optical Society of America.

NASA Discipline Space Human Factors↗

RODEM Jet Datasets

We present the RODEM Jet Datasets, a comprehensive collection of simulated large-radius jets designed to support the development and evaluation of machine-learning algorithms in particle physics. These datasets encompass a diverse range of jet sources, including quark/gluon jets, jets from the decay of W bosons, top quarks, and heavy new-physics particles. The datasets provide detailed substructure information, including jet kinematics, constituent kinematics, and track displacement details, enabling a wide range of applications in jet tagging, anomaly detection, and generative modelling.

FOS: Physical sciences↗

Adoption of image-driven machine learning for microstructure characterization and materials design: A Perspective

Microstructure characterization enables the development of structure-processing-property relationships critical to several research areas within the broad field of materials science, from alloy design to the assessment of corrosion resistance, and failure analysis. Conventional approaches to material characterization have relied on either qualitative inference by the human ex-pert or software applications that can extract high-level features from images, such as boundary segmentation, average grain diameter, etc. Such approaches rely heavily on subject matter expert user intervention and knowledge of what phases or more generally, what microstructural features, are of interest. The recent surge in the adoption of machine learning techniques to address problems in materials engineering has brought with it an increased interest and application of Image Driven Machine Learning (IDML) approaches. In this work, we review the applications of IDML to the field of materials characterization. A canonical hierarchy of stages is defined, which when put sequentially together completes an IDML study: problem definition, dataset building, model selection and training, model evaluation, and integration with existing instrumentation or simulation workflow. The studies reviewed in this work are analyzed from the perspective of each of these stages. Such a review permits agranular assessment of the field, for example the impact of IDML on materials characterization at the nanoscale, the size of a typical dataset required to train a semantic segmentation model on electron microscopy images, ubiquitousness of transfer learning in the domain, etc. Finally, we discuss the importance of interpretability and explainability in the field of IDML for materials characterization, and provide an overview of two emerging techniques in the field: semantic segmentation and generative adversarial networks.

Baskaran, Arun↗

Performance of wind assessment datasets in United States coastal areas

The atmospheric dynamics that occur near the intersection of land and water offer exciting and challenging opportunities for wind energy deployment in coastal locations. New models and tools are continually being developed in support of wind resource assessment, and three recent products are explored in this work for their performance in representing characteristics of the wind resource at coastal locations: the Global Wind Atlas 3 (GWA3), the 2023 National Offshore Wind dataset (NOW-23), and the wind climate simulations that are a component of the Wind Integration National Dataset (WIND) Toolkit Long-Term Ensemble Dataset (WTK-LED Climate). These relatively new products are freely available and user-friendly so that anyone – from a utility-scale developer to a resident or business owner – can evaluate the potential for wind energy generation at their location of interest. The validations in this work provide guidance on the accuracy of wind resource assessments for coastal customers interested in installing small or midsize wind turbines (≤ 1 MW in capacity) to support energy needs at the residential, business, or community scale, such as the island and remotely located participants of the U.S. Department of Energy's Energy Transitions Initiative Partnership Project. At 23 coastal locations across the United States, dataset performance varies according to different evaluation metrics. All three recent datasets tend to overestimate the observed coastal wind resource. GWA3 produces the smallest annual average wind speed relative errors, whereas WTK-LED Climate is in best agreement in terms of representing diurnal wind speed cycles. NOW-23 is the highest performing of the datasets for representing seasonal and interannual trends in the coastal wind resource. While GWA3 and WTK-LED Climate are relatively insensitive to the dataset output heights selected for wind resource assessment at small and midsize wind turbine hub heights (20–60 m), significant variation in the NOW-23 representation of wind shear across the wind profile in the lowest 100 m of the atmosphere leads to notable differences in wind speed estimates according to the dataset output heights selected for evaluation. GWA3 exhibits challenges in the representation of observed wind speed diurnal cycles at small and midsize turbine hub heights, likely due to the dataset's consistent treatment of hourly wind speed trends regardless of altitude.

17 WIND ENERGY↗

Role of copper in nitrous oxide accumulation in wetland soils from Illinois and South Carolina and stream sediments from Tennessee

Denitrification is microbially-mediated through enzymes containing metal cofactors. Laboratory studies of pure cultures have highlighted that the availability of copper (Cu), required for the multicopper enzyme nitrous oxide reductase, can limit nitrous oxide (N2O) reduction. However, in natural aquatic systems, such as wetlands and hyporheic zones in stream beds, the role of Cu in controlling denitrification remains incompletely understood. In this study, we collected soils and sediments from three natural environments -- riparian wetlands, marsh wetlands, and a stream -- to investigate their nitrogen species transformation activity at background Cu levels and different supplemented Cu loadings. All of the systems contained solid-phase associated Cu below or around geological levels (40–280 nmol g-1) and exhibited low dissolved Cu (3–50 nM), which made them appropriate sites for evaluating the effect of limited Cu availability on denitrification.The dataset contains the variation in the nutrient (nitrate (NO3-), nitrite (NO2-), and ammonium (NH4+)) and N2O concentrations during the incubation experiments to evaluate the effect of Cu. We have also reported the variation in metal concentrations (Cu, iron(Fe) and manganese (Mn)) and dissolved organic carbon (DOC) concentrations during the incubations. The parameters obtained using a kinetic model to quantitatively report the effect of Cu on nitrogen species conversion in natural aquatic systems are also added in the dataset. The dataset also contains the labile concentration of Cu estimated using a speciation model in MINTEQ. All data are provided in text-based CSV format with header sections indicating the data contained in each file and the corresponding units.Our study suggests that high concentrations of N2O accumulated in all microcosms lacking Cu amendment except for one stream sediment sample. With Cu added to provide dissolved concentrations at trace levels (10–300 nM), the reduction rate of N2O to N2 in the wetland soils and stream sediments was enhanced. A kinetic model could account for the trends in nitrogen species by combining the reactions for microbial reduction of NO3- to NO2-/N2O/N2 and abiotic reduction of NO2 to nitrogen (N2_. The model revealed that the rate of N2O to N2 conversion increased significantly in the presence of Cu. For riparian wetland soils and stream sediments, the kinetic model also suggested that overall denitrification is driven by abiotic reduction of NO2- in the presence of inorganic electron donors. This study demonstrated that natural aquatic systems containing Cu at concentrations less than or equal to crustal abundances may display incomplete reduction of N2O to N2 that would cause N2O accumulation and release to the atmosphere.

54 ENVIRONMENTAL SCIENCES↗

Gulf of Mexico Miocene CO2 Site Characterization Mega Transect

The project Gulf of Mexico Miocene CO2 Site Characterization Mega Transect is one of 9 site characterization projects that were implemented as part of ARRA (American Recovery and Reinvestment Act). Data from this project was used to improve resolution of data in NATCARB in the area of study. Data related to this study has already been incorporated in NATCARB Atlas. The University of Texas at Austin study focused on (1) collecting high-resolution, 3-D (HR3D) seismic datasets of the shallow geologic section to evaluate the capacity of potential reservoirs and the sealing capacity of the confining system, and (2) producing a regional CO2 atlas of Miocene age units of the upper Texas coast submerged lands (southeast Texas). The goal of the study was to evaluate the capability of the Miocene age geologic section of Texas submerged lands to permanently store large volumes of anthropogenic CO2. The geographic locations of three HR3D, “P-Cable” seismic surveys collected in the near-shore waters of southeast Texas offers a vertical and horizontal view (data “cube”) example from the second survey. The dataset's high quality allowed for defining recognizable geologic morphologies (shapes) and possible fluid content, which are important for determining an area’s prospects for geologic storage. The regional CO2 atlas of Miocene age units highlights several topics, including: (1) a regional analyses of petroleum systems as analogs for CO2 storage; (2) petrography, petrology, and extent of confining systems of the Miocene section; (3) static capacity estimate; and (4) examples of characterization methodologies of prospect areas. The HR3D datasets and regional analyses within the CO2 atlas of the offshore Texas Miocene provide a sound basis for a future generation of specific CO2 prospects in the study area.

.las↗

SU-E-T-157: CARMEN: A MatLab-Based Research Platform for Monte Carlo Treatment Planning (MCTP) and Customized System for Planning Evaluation

Purpose: Although there exist several radiotherapy research platforms, such as: CERR, the most widely used and referenced; SlicerRT, which allows treatment plan comparison from various sources; and MMCTP, a full MCTP system; it is still needed a full MCTP toolset that provides users complete control of calculation grids, interpolation methods and filters in order to “fairly” compare results from different TPSs, supporting verification with experimental measurements. Methods: This work presents CARMEN, a MatLab-based platform including multicore and GPGPU accelerated functions for loading RT data; designing treatment plans; and evaluating dose matrices and experimental data.CARMEN supports anatomic and functional imaging in DICOM format, as well as RTSTRUCT, RTPLAN and RTDOSE. Besides, it contains numerous tools to accomplish the MCTP process, managing egs4phant and phase space files.CARMEN planning mode assist in designing IMRT, VMAT and MERT treatments via both inverse and direct optimization. The evaluation mode contains a comprehensive toolset (e.g. 2D/3D gamma evaluation, difference matrices, profiles, DVH, etc.) to compare datasets from commercial TPS, MC simulations (i.e. 3ddose) and radiochromic film in a user-controlled manner. Results: CARMEN has been validated against commercial RTPs and well-established evaluation tools, showing coherent behavior of its multiple algorithms. Furthermore, CARMEN platform has been used to generate competitive complex treatment that has been published in comparative studies. Conclusion: A new research oriented MCTP platform with a customized validation toolset has been presented. Despite of being coded with a high-level programming language, CARMEN is agile due to the use of parallel algorithms. The wide-spread use of MatLab provides straightforward access to CARMEN’s algorithms to most researchers. Similarly, our platform can benefit from the MatLab community scientific developments as filters, registration algorithms etc. Finally, CARMEN arises the importance of grid and filtering control in treatment plan comparison.

60 APPLIED LIFE SCIENCES↗

Probing Accuracy-Speedup Tradeoff in Machine Learning Surrogates for Molecular Dynamics Simulations

The performance promise of machine learning surrogates of molecular dynamics simulations of soft materials is significant but generally comes at the cost of acquiring large training datasets to learn the complex relationships between input soft material attributes and output properties. Under the constraint of limited high-performance computing resources, optimizing the size of the training datasets becomes paramount. Using an artificial neural network based surrogate for molecular dynamics simulations of confined electrolytes, we explore the tradeoff between surrogate accuracy and computational gains. Accuracy is assessed by computing the root-mean-square errors between the surrogate predictions and the ground truth results obtained via molecular dynamics simulations. The computational performance is judged by evaluating the speedup which incorporates the training dataset creation time. Improvement in accuracy occurs with a loss of speedup, which scales as the inverse of the training dataset size. Furthermore, the link between surrogate generalizability and the accuracy-speedup tradeoff is assessed by examining the errors incurred in surrogate predictions on unseen, interpolated input variables and developing a net speedup metric to capture the associated gains.

Anions↗

Anomalous cluster detection in spatiotemporal meteorological fields

Finding anomalous regions in spatiotemporal climate data is an important problem with a need for greater accuracy. The collective and contextual nature of anomalies (e.g., heat waves) coupled with the real‐valued, seasonal, multimodal, highly correlated, and gridded nature of climate variable observations poses a multitude of challenges. Existing anomaly detection methods have limitations in the specific setting of real‐valued areal spatiotemporal data. In this paper, we develop a method for extreme event detection in meteorological datasets that follows from well known distribution‐based anomaly detection approaches. The method models spatial and temporal correlations explicitly through a piecewise parametric assumption and generalizes the Mahalanobis distance across distributions of different dimensionalities. The result is an effective method to mine contiguous spatiotemporal anomalous regions from meteorological fields which improves upon the current standard approach in climatology. The proposed method has been evaluated on a real global surface temperature dataset and validated using historical records of extreme events.

Ramachandra, Bharathkumar↗

Analysis of Remotely Sensed Datasets to Detect Changes in Waste Sites

This work evaluated various indirect remote techniques for detecting surficial changes and structural land deformation at the Hanford site. Given the partial collapse of the PUREX tunnel, and possible surface subsidence at other burial sites containing radioactive waste and contaminated equipment, timely alert and knowledge related to land surface elevation changes has emerged as a management priority. Findings of this study suggests that space borne RADAR and multispectral based-analysis would be of significant value in detecting changes in surficial and structural land features that may be otherwise undetectable or unnoticed from field observations alone. A continuous, and potentially semi-automated, monitoring program would add critical insight to recovering from, and predicting episodic type events, such as the tunnel collapse. Furthermore, an indirect approach ultimately saves resources and time, and avoids unnecessary risk associated with ground-based monitoring. Various options for supporting such an effort are described in the final section of this document.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Guidance on the Evaluation of Models for LNG fires

This report provides datasets that can be used to validate models for Liquefied Natural Gas ( LNG ) fires, specifically those used to predict the thermal hazards from various types of fires from accidents involving LNG at land-based facilities. The datasets presented include pool fires, trench fires, jet fires, fireballs, and vapor cloud fires. Recommendations are provided regarding the method of quantification and the presentation of information for reporting the comparison.

03 NATURAL GAS↗

CEAZ: Accelerating Parallel I/O Via Hardware-Algorithm Co-Designed Adaptive Lossy Compression

As supercomputers continue to grow to exa-scale, the amount of data that needs to be saved or transmitted is exploding. To this end, many previous works have studied using error-bounded lossy compressors to reduce the data size and improve the I/O performance. However, little work has been done for effectively offloading lossy compression onto FPGA-based SmartNICs to reduce the compression overhead. In this paper, we propose a hardware-algorithm co-design of efficient and adaptive lossy compressor for scientific data on FPGAs (called CEAZ) to accelerate parallel I/O. Our contribution is fourfold: (1) We propose an efficient Huffman coding approach that can adaptively update Huffman codewords online based on codewords generated offline (from a variety of representative scientific datasets). (2) We derive a theoretical analysis to support a precise control of compression ratio under an error-bounded compression mode, enabling accurate offline Huffman codewords generation. This also help us create a fixed-ratio compression mode for consistent throughput. (3) We develop an efficient compression pipeline by adopting cuSZ’s dual-quantization algorithm to our hardware use case. (4) We evaluate CEAC on five real-world datasets with both a single FPGA board and 256 nodes from Bridges2 supercomputer. Experiments show that CEAZ outperforms the second-best FPGA-based lossy compressor by 2× of throughput and 9.6× of compression ratio. It also improves MPI_File_write and MPI_Gather throughputs by up to 32.7× and 31.4×, respectively.

Zhang, Chengming↗

Performance and Reliability Assessment of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) Data Advisor (ADA)

The Atmospheric Radiation Measurement (ARM) User Facility provides one of the world's largest openly accessible repositories of atmospheric observations through the ARM Data Discovery platform. Although the repository contains more than three decades of measurements collected from permanent observatories, mobile facilities, aircraft campaigns, and field experiments, identifying appropriate datasets can be challenging, particularly for new users unfamiliar with ARM instrumentation and datastream organization. To improve data accessibility, the ARM Data Center developed the ARM Data Advisor (ADA), an artificial intelligence-powered assistant designed to facilitate scientific data discovery, dataset interpretation, and user guidance. This report evaluates ADA's performance as a domain-specific scientific assistant using realistic atmospheric science workflows. The evaluation examines five key capabilities: data retrieval and curation efficiency, hallucination resistance, scientific reasoning, response to ambiguous queries, and content retention and session continuity. Representative prompts were developed to simulate typical interactions between researchers and the ARM Data Discovery platform, and ADA's responses were assessed for retrieval completeness, scientific accuracy, consistency, and practical usefulness. In these representative tests, ADA reduced the complexity of discovering and accessing ARM datasets by recommending appropriate datastreams, explaining instrumentation, interpreting metadata, and assisting with data processing workflows. ADA also exhibits strong domain knowledge of atmospheric science terminology and generally resists hallucination by acknowledging unavailable datasets and requesting clarification when appropriate. Overall, the results indicate that ADA represents a promising advancement in scientific data discovery within the ARM User Facility and has considerable potential to improve researcher productivity, particularly for new users and interdisciplinary scientists seeking efficient access to ARM observations.

Salvador, Christian [ORNL] (ORCID:0000000283287777↗