Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

TU-AB-BRA-11: Evaluation of Fully Automatic Volumetric GBM Segmentation in the TCGA-GBM Dataset: Prognosis and Correlation with VASARI Features

Purpose: Reproducible definition and quantification of imaging biomarkers is essential. We evaluated a fully automatic MR-based segmentation method by comparing it to manually defined sub-volumes by experienced radiologists in the TCGA-GBM dataset, in terms of sub-volume prognosis and association with VASARI features. Methods: MRI sets of 67 GBM patients were downloaded from the Cancer Imaging archive. GBM sub-compartments were defined manually and automatically using the Brain Tumor Image Analysis (BraTumIA), including necrosis, edema, contrast enhancing and non-enhancing tumor. Spearman’s correlation was used to evaluate the agreement with VASARI features. Prognostic significance was assessed using the C-index. Results: Auto-segmented sub-volumes showed high agreement with manually delineated volumes (range (r): 0.65 – 0.91). Also showed higher correlation with VASARI features (auto r = 0.35, 0.60 and 0.59; manual r = 0.29, 0.50, 0.43, for contrast-enhancing, necrosis and edema, respectively). The contrast-enhancing volume and post-contrast abnormal volume showed the highest C-index (0.73 and 0.72), comparable to manually defined volumes (p = 0.22 and p = 0.07, respectively). The non-enhancing region defined by BraTumIA showed a significantly higher prognostic value (CI = 0.71) than the edema (CI = 0.60), both of which could not be distinguished by manual delineation. Conclusion: BraTumIA tumor sub-compartments showed higher correlation with VASARI data, and equivalent performance in terms of prognosis compared to manual sub-volumes. This method can enable more reproducible definition and quantification of imaging based biomarkers and has a large potential in high-throughput medical imaging research.

60 APPLIED LIFE SCIENCES↗

Corrective Action Decision Document/Closure Report for Corrective Action Unit 234: Mud Pits, Cellars, and Mud Spills Nevada Test Site, Nevada, Revision 0

This Corrective Action Decision Document/Closure Report has been prepared for Corrective Action Unit (CAU) 234, Mud Pits, Cellars, and Mud Spills, located in Areas 2, 3, 4, 12, and 15 at the Nevada Test Site, Nevada, in accordance with the Federal Facility Agreement and Consent Order (FFACO, 1996; as amended February 2008). Corrective Action Unit 234 is comprised of the following 12 corrective action sites: •02-09-48, Area 2 Mud Plant #1 •02-09-49, Area 2 Mud Plant #2 •02-99-05, Mud Spill •03-09-02, Mud Dump Trenches •04-44-02, Mud Spill •04-99-02, Mud Spill •12-09-01, Mud Pit •12-09-04, Mud Pit •12-09-08, Mud Pit •12-30-14, Cellar •12-99-07, Mud Dump •15-09-01, Mud Pit The purpose of this Corrective Action Decision Document/Closure Report is to provide justification and documentation supporting the recommendation for closure of CAU 234 with no further corrective action. To achieve this, corrective action investigation (CAI) activities were performed as set forth in the Corrective Action Investigation Plan for Corrective Action Unit 234: Mud Pits, Cellars, and Mud Spills (NNSA/NSO, 2007). The purpose of the CAI was to fulfill the following data needs as defined during the data quality objective (DQO) process: •Determine whether contaminants of concern are present. •If contaminants of concern are present, determine their extent. •Provide sufficient information and data to complete appropriate corrective actions. The CAU 234 dataset from the investigation results was evaluated based on the data quality indicator parameters. This evaluation demonstrated the quality and acceptability of the dataset for use in fulfilling the DQO data needs.

54 ENVIRONMENTAL SCIENCES↗

Exudate-based diabetic macular edema detection in fundus images using publicly available datasets

Diabetic macular edema (DME) is a common vision threatening complication of diabetic retinopathy. In a large scale screening environment DME can be assessed by detecting exudates (a type of bright lesions) in fundus images. In this work, we introduce a new methodology for diagnosis of DME using a novel set of features based on colour, wavelet decomposition and automatic lesion segmentation. These features are employed to train a classifier able to automatically diagnose DME through the presence of exudation. We present a new publicly available dataset with ground-truth data containing 169 patients from various ethnic groups and levels of DME. This and other two publicly available datasets are employed to evaluate our algorithm. We are able to achieve diagnosis performance comparable to retina experts on the MESSIDOR (an independently labelled dataset with 1200 images) with cross-dataset testing (e.g., the classifier was trained on an independent dataset and tested on MESSIDOR). Our algorithm obtained an AUC between 0.88 and 0.94 depending on the dataset/features used. Additionally, it does not need ground truth at lesion level to reject false positives and is computationally efficient, as it generates a diagnosis on an average of 4.4 s (9.3 s, considering the optic nerve localization) per image on an 2.6 GHz platform with an unoptimized Matlab implementation.

59 BASIC BIOLOGICAL SCIENCES↗

Decentralized Distributed Proximal Policy Optimization (DD-PPO) for High Performance Computing Scheduling on Multi-User Systems

Resource allocation in High Performance Computing (HPC) environments presents a complex and multifaceted challenge for job scheduling algorithms. Beyond the efficient allocation of system resources, schedulers must account for and optimize multiple performance metrics, including job wait time and system throughput. Traditional heuristic-based scheduling algorithms increasingly struggle and lack the efficiency needed to meet the demands and address the complexity and scale of modern HPC systems. Consequently, recent research efforts have focused on leveraging advancements in Artificial Intelligence (AI) and Deep Learning (DL), particularly Reinforcement Learning (RL), to develop more adaptable and intelligent scheduling strategies. Previous RL-based scheduling approaches have explored a range of algorithms, from Deep Q-Networks (DQN) to Proximal Policy Optimization (PPO), and more recently, hybrid methods that integrate Graph Neural Networks (GNNs) with RL techniques. However, a common limitation across these methods is their reliance on relatively small datasets, with few methods being evaluated using large-scale, multi-million-job trace datasets representative of real-world HPC workloads. Moreover, existing RL schedulers face scalability issues due to centralized policy updates, which hinder training efficiency and performance when applied to large datasets. This study introduces a novel RL-based scheduler utilizing Decentralized Distributed Proximal Policy Optimization (DD-PPO) algorithm, which supports large-scale distributed training across multiple workers without requiring parameter synchronization at every step. By eliminating reliance on centralized updates to a shared policy, the DD-PPO scheduler enhances scalability, training efficiency, and sample utilization. Experimental validation using a large real-world dataset containing over 11.5 million job traces collected from petascale HPC systems over six years assesses the influence of dataset scale on training effectiveness and compares DD-PPO performance to traditional and advanced scheduling approaches. The experimental results demonstrate improved scheduling performance in comparison to both heuristic-based schedulers and existing RL-based scheduling algorithms.

AI↗

Automatic Diabetic Macular Edema Detection in Fundus Images Using Publicly Available Datasets

Diabetic macular edema (DME) is a common vision threatening complication of diabetic retinopathy. In a large scale screening environment DME can be assessed by detecting exudates (a type of bright lesions) in fundus images. In this work, we introduce a new methodology for diagnosis of DME using a novel set of features based on colour, wavelet decomposition and automatic lesion segmentation. These features are employed to train a classifier able to automatically diagnose DME. We present a new publicly available dataset with ground-truth data containing 169 patients from various ethnic groups and levels of DME. This and other two publicly available datasets are employed to evaluate our algorithm. We are able to achieve diagnosis performance comparable to retina experts on the MESSIDOR (an independently labelled dataset with 1200 images) with cross-dataset testing. Our algorithm is robust to segmentation uncertainties, does not need ground truth at lesion level, and is very fast, generating a diagnosis on an average of 4.4 seconds per image on an 2.6 GHz platform with an unoptimised Matlab implementation.

59 BASIC BIOLOGICAL SCIENCES↗

Corrective Action Decision Document for Corrective Action Unit 568. Area 3 Plutonium Dispersion Sites, Nevada National Security Site, Nevada Revision 0

The purpose of this Corrective Action Decision Document is to identify and provide the rationale for the recommendation of corrective action alternatives (CAAs) for the 14 CASs within CAU 568. Corrective action investigation (CAI) activities were performed from April 2014 through May 2015, as set forth in the Corrective Action Investigation Plan for Corrective Action Unit 568: Area 3 Plutonium Dispersion Sites, Nevada National Security Site, Nevada; and in accordance with the Soils Activity Quality Assurance Plan, which establishes requirements, technical planning, and general quality practices. The purpose of the CAI was to fulfill data needs as defined during the DQO process. The CAU 568 dataset of investigation results was evaluated based on a data quality assessment. This assessment demonstrated that the dataset is complete and acceptable for use in fulfilling the DQO data needs. Based on the evaluation of analytical data from the CAI, review of future and current operations at the 14 CASs, and the detailed and comparative analysis of the potential CAAs, the following corrective actions are recommended for CAU 568: • No further action is the preferred corrective action for CASs 03-23-17, 03-23-22, 03-23-26. • Closure in place is the preferred corrective action for CAS 03-23-19; 03-45-01; the SE DCBs at CASs 03-23-20, 03-23-23, 03-23-31, 03-23-32, 03-23-33, and 03-23-34; and the Pascal-BHCA at CAS 03-23-31. • Clean closure is the preferred corrective action for CASs 03-08-04, 03-23-30, and 03-26-04; and the four well head covers at CASs 03-23-20, 03-23-23, 03-23-31, and 03-23-33.

54 ENVIRONMENTAL SCIENCES↗

Trace: a high-throughput tomographic reconstruction engine for large-scale datasets

Abstract Background Modern synchrotron light sources and detectors produce data at such scale and complexity that large-scale computation is required to unleash their full power. One of the widely used imaging techniques that generates data at tens of gigabytes per second is computed tomography (CT). Although CT experiments result in rapid data generation, the analysis and reconstruction of the collected data may require hours or even days of computation time with a medium-sized workstation, which hinders the scientific progress that relies on the results of analysis. Methods We present Trace, a data-intensive computing engine that we have developed to enable high-performance implementation of iterative tomographic reconstruction algorithms for parallel computers. Trace provides fine-grained reconstruction of tomography datasets using both (thread-level) shared memory and (process-level) distributed memory parallelization. Trace utilizes a special data structure called replicated reconstruction object to maximize application performance. We also present the optimizations that we apply to the replicated reconstruction objects and evaluate them using tomography datasets collected at the Advanced Photon Source. Results Our experimental evaluations show that our optimizations and parallelization techniques can provide 158× speedup using 32 compute nodes (384 cores) over a single-core configuration and decrease the end-to-end processing time of a large sinogram (with 4501 × 1 × 22,400 dimensions) from 12.5 h to <5 min per iteration. Conclusion The proposed tomographic reconstruction engine can efficiently process large-scale tomographic data using many compute nodes and minimize reconstruction times.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

TU-CD-BRB-04: Automated Radiomic Features Complement the Prognostic Value of VASARI in the TCGA-GBM Dataset

Purpose: To compare the complementary prognostic value of automated Radiomic features to that of radiologist-annotated VASARI features in TCGA-GBM MRI dataset. Methods: For 96 GBM patients, pre-operative MRI images were obtained from The Cancer Imaging Archive. The abnormal tumor bulks were manually defined on post-contrast T1w images. The contrast-enhancing and necrotic regions were segmented using FAST. From these sub-volumes and the total abnormal tumor bulk, a set of Radiomic features quantifying phenotypic differences based on the tumor intensity, shape and texture, were extracted from the post-contrast T1w images. Minimum-redundancy-maximum-relevance (MRMR) was used to identify the most informative Radiomic, VASARI and combined Radiomic-VASARI features in 70% of the dataset (training-set). Multivariate Cox-proportional hazards models were evaluated in 30% of the dataset (validation-set) using the C-index for OS. A bootstrap procedure was used to assess significance while comparing the C-Indices of the different models. Results: Overall, the Radiomic features showed a moderate correlation with the radiologist-annotated VASARI features (r = −0.37 – 0.49); however that correlation was stronger for the Tumor Diameter and Proportion of Necrosis VASARI features (r = −0.71 – 0.69). After MRMR feature selection, the best-performing Radiomic, VASARI, and Radiomic-VASARI Cox-PH models showed a validation C-index of 0.56 (p = NS), 0.58 (p = NS) and 0.65 (p = 0.01), respectively. The combined Radiomic-VASARI model C-index was significantly higher than that obtained from either the Radiomic or VASARI model alone (p = <0.001). Conclusion: Quantitative volumetric and textural Radiomic features complement the qualitative and semi-quantitative annotated VASARI feature set. The prognostic value of informative qualitative VASARI features such as Eloquent Brain and Multifocality is increased with the addition of quantitative volumetric and textural features from the contrast-enhancing and necrotic tumor regions. These results should be further evaluated in larger validation cohorts.

60 APPLIED LIFE SCIENCES↗

Ionizing radiation effects in SONOS-based neuromorphic inference accelerators

Here, we evaluate the sensitivity of neuromorphic inference accelerators based on Silicon-Oxide-Nitride-Oxide-Silicon (SONOS) charge trap memory arrays to total ionizing dose (TID) effects. Data retention statistics were collected for 16 Mbit of 40 nm SONOS digital memory exposed to ionizing radiation from a Co-60 source, showing good retention of the bits up to the maximum dose of 500 krad(Si). Using this data, we formulate a rate-equation-based model for the TID response of trapped charge carriers in the ONO stack, and predict the effect of TID on intermediate device states between ‘program’ and ‘erase’. This model is then used to simulate arrays of low-power, analog SONOS devices that store 8-bit neural network weights and support in situ matrix-vector multiplication. We evaluate the accuracy of the irradiated SONOS-based inference accelerator on two image recognition tasks – CIFAR-10 and the challenging ImageNet dataset – using state-of-the-art convolutional neural networks, such as ResNet-50. We find that across the datasets and neural networks evaluated, the accelerator tolerates a maximum TID between 10 krad(Si) and 100 krad(Si), with deeper networks being more susceptible to accuracy losses due to TID.

43 PARTICLE ACCELERATORS↗

Multi-model Mean Nitrogen and Sulfur Deposition from the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP): Evaluation of Historical and Projected Future Changes

We present multi-model global datasets of nitrogen and sulfate deposition covering time periods from 1850 to 2100, calculated within the Atmospheric Chemistry and Climate Model Intercomparison Project (ACCMIP). The computed deposition fluxes are compared to surface wet deposition and ice-core measurements. We use a new dataset of wet deposition for 2000-2002 based on critical assessment of the quality of existing regional network data. We show that for present-day (year 2000 ACCMIP time-slice), the ACCMIP results perform similarly to previously published multi-model assessments. The analysis of changes between 1980 and 2000 indicates significant differences between model and measurements over the United States, but less so over Europe. This difference points towards misrepresentation of 1980 NH3 emissions over North America. Based on ice-core records, the 1850 deposition fluxes agree well with Greenland ice cores but the change between 1850 and 2000 seems to be overestimated in the Northern Hemisphere for both nitrogen and sulfur species. Using the Representative Concentration Pathways to define the projected climate and atmospheric chemistry related emissions and concentrations, we find large regional nitrogen deposition increases in 2100 in Latin America, Africa and parts of Asia under some of the scenarios considered. Increases in South Asia are especially large, and are seen in all scenarios, with 2100 values more than double 2000 in some scenarios and reaching >1300 mgN/m2/yr averaged over regional to continental scale regions in RCP 2.6 and 8.5, ~30-50% larger than the values in any region currently (2000). Despite known issues, the new ACCMIP deposition dataset provides novel, consistent and evaluated global gridded deposition fields for use in a wide range of climate and ecological studies.

ACCMIP↗

Optimizing Error-Bounded Lossy Compression for Scientific Data by Dynamic Spline Interpolation

Today's scientific simulations are producing vast volumes of data that cannot be stored and transferred efficiently because of limited storage capacity, parallel I/O bandwidth, and network bandwidth. The situation is getting worse over time because of the ever-increasing gap between relatively slow data transfer speed and fast-growing computation power in modern supercomputers. Error-bounded lossy compression is becoming one of the most critical techniques for resolving the big scientific data issue, in that it can significantly reduce the scientific data volume while guaranteeing that the reconstructed data is valid for users because of its compression-error-bounding feature. In this paper, we present a novel error-bounded lossy compressor based on a state-of-the-art prediction-based compression framework. Our solution exhibits substantially better compression quality than all of the existing error-bounded lossy compressors, with comparable compression speed. Specifically, our contribution is threefold. (1) We provide an in-depth analysis of why the best-existing prediction-based lossy compressor can only minimally improve the compression quality. (2) We propose a dynamic spline interpolation approach with a series of optimization strategies that can significantly improve the data prediction accuracy, substantially improving the compression quality in turn. (3) We perform a thorough evaluation using six real-world scientific simulation datasets across different science domains to evaluate our solution vs. all other related works. Experiments show that the compression ratio of our solution is higher than that of the second-best lossy compressor by 20%similar to 460% with the same error bound in most of the cases.

Zhao, Kai↗

LU60645GT and MA132843GT Catalogues of Lunar and Martian Impact Craters Developed Using a Crater Shape-based Interpolation Crater Detection Algorithm for Topography Data

For Mars, 57,633 craters from the manually assembled catalogues and 72,668 additional craters identified using several crater detection algorithms (CDAs) have been merged into the MA130301GT catalogue. By contrast, for the Moon the most complete previous catalogue contains only 14,923 craters. Two recent missions provided higher-quality digital elevation maps (DEMs): SELENE (in 1/16° resolution) and Lunar Reconnaissance Orbiter (we used up to 1/512°). This was the main motivation for work on the new Crater Shape-based interpolation module, which improves previous CDA as follows: (1) it decreases the number of false-detections for the required number of true detections; (2) it improves detection capabilities for very small craters; and (3) it provides more accurate automated measurements of craters' properties. The results are: (1) LU60645GT, which is currently the most complete (up to ∼D>=8 km) catalogue of Lunar craters; and (2) MA132843GT catalogue of Martian craters complete up to ∼D>=2 km, which is the extension of the previous MA130301GT catalogue. As previously achieved for Mars, LU60645GT provides all properties that were provided by the previous Lunar catalogues, plus: (1) correlation between morphological descriptors from used catalogues; (2) correlation between manually assigned attributes and automated measurements; (3) average errors and their standard deviations for manually and automatically assigned attributes such as position coordinates, diameter, depth/diameter ratio, etc; and (4) a review of positional accuracy of used datasets. Additionally, surface dating could potentially be improved with the exhaustiveness of this new catalogue. The accompanying results are: (1) the possibility of comparing a large number of Lunar and Martian craters, of e.g. depth/diameter ratio and 2D profiles; (2) utilisation of a method for re-projection of datasets and catalogues, which is very useful for craters that are very close to poles; and (3) the extension of the previous framework for evaluation of CDAs with datasets and ground-truth catalogue for the Moon.

Craters↗

Data-driven key performance indicators and datasets for building energy flexibility: A review and perspectives

Energy flexibility, through short-term demand-side management (DSM) and energy storage technologies, is now seen as a major key to balancing the fluctuating supply in different energy grids with the energy demand of buildings. This is especially important when considering the intermittent nature of ever-growing renewable energy production, as well as the increasing dynamics of electricity demand in buildings. This paper provides a holistic review of (1) data-driven energy flexibility key performance indicators (KPIs) for buildings in the operational phase and (2) open datasets that can be used for testing energy flexibility KPIs. The review identifies a total of 48 data-driven energy flexibility KPIs from 87 recent and relevant publications. These KPIs were categorized and analyzed according to their type, complexity, scope, key stakeholders, data requirement, baseline requirement, resolution, and popularity. Moreover, 330 building datasets were collected and evaluated. Of those, 16 were deemed adequate to feature building performing demand response or building-to-grid (B2G) services. The DSM strategy, building scope, grid type, control strategy, needed data features, and usability of these selected 16 datasets were analyzed. This review reveals future opportunities to address limitations in the existing literature: (1) developing new data-driven methodologies to specifically evaluate different energy flexibility strategies and B2G services of existing buildings; (2) developing baseline-free KPIs that could be calculated from easily accessible building sensors and meter data; (3) devoting non-engineering efforts to promote building energy flexibility, standardizing data-driven energy flexibility quantification and verification processes; and (4) curating and analyzing datasets with proper description for energy flexibility assessm.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Corrective Action Decision Document/Closure Report for Corrective Action Unit 550: Smoky Contamination Area Nevada National Security Site, Nevada, Revision 0

This Corrective Action Decision Document/Closure Report presents information supporting the closure of Corrective Action Unit (CAU) 550: Smoky Contamination Area, Nevada National Security Site, Nevada. CAU 550 includes 19 corrective action sites (CASs), which consist of one weapons-related atmospheric test (Smoky), three safety experiments (Ceres, Oberon, Titania), and 15 debris sites (Table ES-1). The CASs were sorted into the following study groups based on release potential and technical similarities: • Study Group 1, Atmospheric Test • Study Group 2, Safety Experiments • Study Group 3, Washes • Study Group 4, Debris The purpose of this document is to provide justification and documentation supporting the conclusion that no further corrective action is needed for CAU 550 based on implementation of the corrective actions listed in Table ES-1. Corrective action investigation (CAI) activities were performed between August 2012 and October 2013 as set forth in the Corrective Action Investigation Plan for Corrective Action Unit 550: Smoky Contamination Area; and in accordance with the Soils Activity Quality Assurance Plan. The approach for the CAI was to investigate and make data quality objective (DQO) decisions based on the types of releases present. The purpose of the CAI was to fulfill data needs as defined during the DQO process. The CAU 550 dataset of investigation results was evaluated based on a data quality assessment. This assessment demonstrated the dataset is complete and acceptable for use in fulfilling the DQO data needs.

54 ENVIRONMENTAL SCIENCES↗

Corrective Action Decision Document/Closure Report for Corrective Action Unit 546: Injection Well and Surface Releases Nevada Test Site, Nevada, Revision 0, with ROTCs 1-2

This Corrective Action Decision Document/Closure Report has been prepared for Corrective Action Unit 546, Injection Well and Surface Releases, at the Nevada Test Site, Nevada, in accordance with the Federal Facility Agreement and Consent Order (FFACO, 1996; as amended February 2008). Corrective Action Unit (CAU) 546 is comprised of two corrective action sites (CASs): • 06-23-02, U-6a/Russet Testing Area • 09-20-01, Injection Well The purpose of this Corrective Action Decision Document/Closure Report is to provide justification and documentation supporting the recommendation for closure of CAU 546. To achieve this, corrective action investigation (CAI) activities were performed from May 5 through May 28, 2008, as set forth in the Corrective Action Investigation Plan for Corrective Action Unit 546: Injection Well and Surface Releases, Nevada Test Site, Nevada (NNSA/NSO, 2008). The purpose of the CAI was to fulfill the following data needs as defined during the data quality objective (DQO) process: • Determine whether a contaminant of concern is present at a given CAS. • Determine whether sufficient information is available to evaluate potential corrective action alternatives at each CAS. The CAU 546 dataset from the investigation results was evaluated based on the data quality indicator parameters. This evaluation demonstrated the quality and acceptability of the dataset for use in fulfilling the DQO data needs. Because DQO data needs were met, and corrective actions have been implemented, it has been determined that no further corrective action (based on risk to human receptors) is necessary for the CAU 546 CASs. The U.S. Department of Energy, National Nuclear Security Administration Nevada Site Office provides the following recommendations: • No further corrective actions are needed for CAU 546 CASs. • No Corrective Action Plan is required. • A Notice of Completion to the U.S. Department of Energy, National Nuclear Security Administration Nevada Site Office is requested from the Nevada Division of Environmental Protection for closure of CAU 546. • Corrective Action Unit 546 should be moved from Appendix III to Appendix IV of the Federal Facility Agreement and Consent Order. Analytes detected during the CAI were evaluated against final action levels established in this document. No analytes were detected at concentrations exceeding final action levels. However, contaminants of concern were presumed to be present in the subsurface soil at CAS 09-20-01. Therefore, the corrective action of close in place was selected as the preferred alternative for this CAS. Potential source material was removed from CAS 06-23-02; therefore, the corrective action of clean closure was selected as the preferred alternative at this CAS.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Data Catalog Software for Hanford Site Environmental Datasets

Environmental information and data underpin achievement of the U.S. Department of Energy (DOE) Office of Environmental Management (EM) mission at the Hanford Site. The Hanford Environmental Data Management (HEDM) Program is the DOE Richland Operations Office (RL) approach to develop and implement a formal program for managing environmental data and the associated records, materials, and systems at the Hanford Site. The current project, contract, organization, and contractor-specific efforts at managing environmental data sets are insufficient to provide orderly, long-term, site-wide access. A vital element to be created within the HEDM program plan is a catalog of data sources, called the Hanford Environmental Information and Data Index (HEIDI), that will enable long-term access and retrievability for the multiple independent sources of data that might otherwise be difficult to discover. This report compares leading open source and commercial data catalog platforms using criteria to assess the functionality needed to develop the HEIDI catalog of Hanford data sources that connects and exchanges data with established Hanford Local Area Network (HLAN) enterprise information technology systems. Proprietary platforms evaluated included ArcGIS Enterprise Sites, Junar, OpenDataSoft, and Socrata, and non-proprietary platforms included Energy Data eXchange (EDX), Comprehensive Knowledge Archive Network (CKAN), and DKAN (a Drupal-based open data portal based on CKAN). Capabilities supporting data discoverability, retrieval, and archival, as well as metadata standard requirements and integration into the HLAN were rated as either failing to meet requirements (F), meeting requirements (M), or exceeding requirements by delivering additional desired features (E). The lowest rating for any capability area was assigned as the overall rating for the platform. These findings enable DOE-RL and the contractors implementing the HEDM plan to focus on candidate tools likely to meet the requirements for implementing HEIDI. All of the platforms receiving an overall rating of ‘F’ were unable to be deployed on Hanford infrastructure or within dedicated cloud resources. A propriety software-as-a-service (SaaS) model of delivering a data catalog (e.g., found in software such as Junar and OpenDataSoft) favors consistency across customers at the expense of customization and configurable roles that are needed for Hanford work. Hosting data on a shared commercial platform places limits on dataset size (maximum of 240 Mb for OpenDataSoft), a significant limitation for HEIDI implementation. EDX, a government data catalog based on CKAN, received the ‘F’ rating due to an inability to incorporate authentication from HLAN into the system. Among platforms rated ‘M’ or ‘E’, only the Socrata platform had a SaaS delivery model. In contrast to other SaaS platforms, Socrata provided custom roles and gateways that allow local datasets to be incorporated into an online catalog. Socrata also complies with the Federal Risk and Authorization Management Program, a significant benefit for cloud-based management of Hanford data. The other platforms rated ‘M’ or ‘E’, ArcGIS Enterprise Sites, CKAN, and DKAN, provide fully self-hosted options, allowing for greater control and flexibility with the HEIDI catalog. These widely used tools have supportive communities of practice, extensive customization options, and demonstrated deployments that provide evidence that they can meet requirements, often deliver additional desired features, and work well with federal government systems. Completely customized alternatives built on a collection of applications were not evaluated because achieving similar performance to CKAN or DKAN requires substantial resources, especially in the absence of the active communities that have grown to support these tools. ArcGIS Enterprise Sites, Socrata, CKAN, and DKAN were evaluated as strong candidates for successful implementation with HEIDI.

54 ENVIRONMENTAL SCIENCES↗

Citations and Readership are Poor Indicators of Research Excellence

In this paper we show that citation counts and Mendeley readership are poor indicators of research excellence. Our experimental design builds on the assumption that a good evaluation metric should be able to distinguish publications that have changed a research field from those that have not. The experiment has been conducted on a new dataset for bibliometric research which we call TrueImpactDataset. TrueImpactDataset is a collection of research publications of two types -- research papers which are considered seminal work in their area and papers which provide a survey (a literature review) of a research area. The dataset also contains related metadata, which include DOIs, titles, authors and abstracts. We describe how the dataset was built and provide overview statistics of the dataset. We propose to use the dataset for validating research evaluation metrics. By using this data, we show that widely used research metrics only poorly distinguish excellent research.

Herrmannova, Dasha↗

Evaluating the Uncertainty of Terrestrial Water Budget Components Over High Mountain Asia

This study explores the uncertainties in terrestrial water budget estimation over High Mountain Asia (HMA) using a suite of uncoupled land surface model (LSM) simulations. The uncertainty in the water balance components of precipitation (P), evapotranspiration (ET), runoff (R), and terrestrial water storage (TWS) is significantly impacted by the uncertainty in the driving meteorology, with precipitation being the most important boundary condition. Ten gridded precipitation datasets along with a mix of model-, satellite-, and gauge-based products, are evaluated first to assess their suitability for LSM simulations over HMA. The datasets are evaluated by quantifying the systematic and random errors of these products as well as the temporal consistency of their trends. Though the broader spatial patterns of precipitation are generally well captured by the datasets, they differ significantly in their means and trends. In general, precipitation datasets that incorporate information from gauges are found to have higher accuracy with low Root Mean Square Errors and high correlation coefficient values. An ensemble of LSM simulations with selected subset of precipitation products is then used to produce the mean annual fluxes and their uncertainty over HMA in P, ET, and R to be 2.11 ± 0.45, 1.26 ± 0.11, and 0.85 ± 0.36 mm per day, respectively. The mean annual estimates of the surface mass (water) balance components from this model ensemble are comparable to global estimates from prior studies. However, the uncertainty/spread of P, ET, and R is significantly larger than the corresponding estimates from global studies. A comparison of ET, snow cover fraction, and changes in TWS estimates against remote sensing-based references confirms the significant role of the input meteorology in influencing the water budget characterization over HMA and points to the need for improving meteorological inputs.

Yoon, Yeosang↗