Search NASA⌕ Search

SEARCH · Search NASA

Results for “metric categorization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Predicting measures of soil health using the microbiome and supervised machine learning

Soil health encompasses a range of biological, chemical, and physical soil properties that sustain the commercial and ecological value of agroecosystems. Monitoring soil health requires a comprehensive set of diagnostics that can be cost-prohibitive for routine analyses. The soil microbiome provides a rich source of information about soil properties, which can be assayed in a high-throughput, cost-effective way. We evaluated the accuracy of random forest (RF) and support vector machine (SVM) regression and classification models in predicting 12 measures of soil health, tillage status, and soil texture from 16S rRNA gene amplicon data with an operationally relevant sample set. We validated the efficacy of the best performing models against independent datasets and also tested best practices for processing microbiome data for use in machine learning. Soil health metrics could be predicted from microbiome data with the best models achieving a Kappa value of ~0.65, for categorical assessments, and a R2 value of ~0.8, for numerical scores. Biological health ratings were better predicted than chemical or physical ratings. Validation with independent datasets revealed that models had general predictive value for soil properties, including yield. The ecological profiles of several taxa important for model accuracy matched the observed relationships with soil health, including Pyrinomonadaceae, Nitrososphaeraceae, and Candidatus Udeaobacter. Models trained at the highest taxonomic resolution proved most accurate, with losses in accuracy resulting from rarefying, sparsity filtering, and aggregating at higher taxonomic ranks. Furthermore, our study provides the groundwork for developing scalable technology to use microbiome-based diagnostics for the assessment of soil health.

16S rRNA gene↗
DOE OSTI2023

Status Report on Regulatory Criteria Applicable to the Use of Artificial Intelligence (AI) and Machine Learning (ML)

Although the interest in the use of artificial intelligence (AI) and machine learning (ML) in nuclear energy is increasing rapidly, at present their implementation is limited. This rapid increase in interest is not surprising considering that implementing AI and ML technology would allow for continuous monitoring, facilitate the implementation of predictive maintenance with optimized staffing plans, enable automation and autonomy opportunities that could drastically reduce fixed operation and maintenance costs, and provide training for operations and maintenance. Other industries are using AI for construction, and in the nuclear arena AI could provide great benefit in decommissioning activities. The ability of AI and ML to operate in real time vastly increases their potential impact. Before AI can be used in design, operations, or as a regulatory tool, the specifics on the regulations applicable to the use of AI for nuclear power applications need to be established. The difficulty is that the specific use cases will dictate the applicability of regulations. For example, even within the application domain associated with operations, the regulations might vary if the AI is used to create a virtual reference for plant operations or is used for training, optimization of maintenance intervals, prioritization of maintenance activities, etc. Different still is if the AI is to be used for design or setting technical specifications, which will introduce additional requirements. US Nuclear Regulatory Commission (NRC) licensing reviews are based on an applicant’s design meeting its performance assessment based on (1) safety goals and objectives, (2) deterministic and/or probabilistic analysis of accident scenarios, and (3) quantitative assessment of design alternatives against the safety goals and objectives using accepted engineering tools, methodologies, and performance criteria. The current regulatory framework does not explicitly address AI or autonomous control. However, as implementing AI technology will require the use of a digital platform, it must meet the requirements of an instrumentation and control (I&C) system. The regulatory requirements for AI, which will be incorporated into the I&C system, will be very dependent on how it is used (i.e., its functionality, safety classification, etc.). The licensing process is primarily risk-based with the identification of components and systems as nonsafety, important to safety, or safety related. A risk-informed approach allows further gradation of components and systems based on risk metrics such as core damage frequency or large early release fractions. Thus, the use cases and the risk categorization of impacted systems and components will determine the regulatory requirements. Regardless of how AI is used it presents new opportunities for risk-informing operating, maintenance, and regulatory decisions. Trustworthiness, transparency, and the ability to validate and verify the results will be paramount in showing that the systems and plant still meet their performance requirements. This report describes the results of research to identify regulatory implications of AI technologies and their uses. Specifically, this report reviews current regulatory guidance relevant to the application of AI for design (including design changes or new designs including advanced reactors), construction, operations, training, maintenance, research, testing, and as a regulatory tool. AI can be automated at different levels from purely informative purposes to autonomous controls. The focus of this review included determination of constraints on the application of AI technology, identification of any regulatory gaps or uncertainties, and clarification of anticipated technical basis information likely to be important for regulatory acceptance of these technologies. Currently, any use of AI at nuclear power plants is focused on nonsafety-related applications. The NRC and other regulatory bodies are evaluating providing guidance to address gaps rather than create new regulations to address the use of AI and ML. This approach seems to be the best to encourage AI development without adding regulatory uncertainty.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Public Reference Data for Megawatt-Scale Hydrogen Electrolysis - NLR Historical Solar PV

The U.S. Department of Energy and National Laboratory of the Rockies (NLR) demonstrate hydrogen electrolysis from variable sources, hydrogen compression and storage, and hydrogen fuel cell power production using megawatt-scale equipment at NLR’s Flatirons Campus as part of the Advanced Research on Integrated Energy Systems (ARIES) research platform. This dataset represents part of that effort and is intended for academic, national laboratory, industrial, and other stakeholders to plan, design, and validate models of megawatt-scale hydrogen technologies and diverse energy infrastructure nationwide. These data provide a baseline for how existing hydrogen electrolysis technologies perform when coupled with various energy technologies. Future datasets will demonstrate how existing hydrogen fuel cell technologies can provide controllable, dispatchable, and variable power output for artificial intelligence data centers and other variable loads. This dataset entry describes the behavior of a 1.25-MW proton exchange membrane MC250 electrolyzer system, manufactured by Nel Hydrogen , [1] when fed historical data generated by the 430-kW, fixed-axis solar photovoltaic (PV) array located at NLR’s Flatirons Campus. (While the electrolyzer balance of plant supports up to 2.5 MW of electrolysis, NLR only has a single 1.25-MW electrolysis stack.) Solar PV power output data for the 2020 calendar year were categorized on a daily basis by total energy generation and standard deviation. Each day was then ranked by these metrics, and the 25th, 50th, and 100th percentiles were selected. The 75th percentile day did not exhibit sufficient variability to make for a valuable experiment. A similar process was used for the related historical wind dataset . [2] The historical days in 2020 that represented these percentiles are Dec. 19, March 29, and May 4, respectively. The entire solar day’s power profile was then fed through the MC250 electrolyzer. Due to its length, the 100th percentile day experiment was split into two parts, and the final 3 hours of the solar day were not captured. These final 3 hours contained no spikes or dips of interest and simply represented a slow decay of input solar power. Also, a single timestamp (13:13:47 on Jan. 14, 2026) was lost in the hydrogen system supervisory control and data acquisition. Finally, during the 25th percentile experiment (solar day Dec. 19, 2020) data recording was lost from 11:00:13 to 11:14:45. The roughly 15 minutes of the solar profile were rerun at the end of the experiment and spliced into this time slot during post-processing. The electrolysis system controls hydrogen production by varying direct current applied to the stack, from a maximum of 3,000 A to a minimum safe operation of 300 A, or 10%. Because the current–voltage characteristic changes as the stack ages and efficiency degrades, the actual minimum safe operating power changes over time. The historical solar profiles were translated from power (kilowatts) to current (amperes) using a curve fit with calibration data and sent to the electrolyzer power supply at 1-Hz frequency. For more details on the statistical analysis process, see the slide deck “Public Reference Data for Megawatt-Scale Hydrogen Electrolysis: NLR Historical Solar PV Analysis and Profile Generation” accessible with this data entry. These datasets report relevant hydrogen balance-of-plant and system data, all captured at 1 Hz, including hydrogen mass production measured with an Emerson Coriolis flow meter. Each .zip file represents a single solar PV electrolysis experiment and is formatted as: {technology}_{percentile}_{scaling factor} For instance, “solarPV-430kW_25_2x.zip” reports the experiment using the 25th percentile solar data from the historical 2020 solar PV dataset, scaled to 200%. Scaling factors were applied to the generated solar PV power output files to more closely match the 1.25-MW capacity of the electrolyzer. Each .zip folder contains the following files: A .csv file containing raw data. An .xlsx file explaining all the fields in the raw data. A .png plot showing the time series of hydrogen production, electrolysis power consumption, and solar power input. A PDF file detailing the historical solar data statistical analysis used to generate the solar profile. An experiment labeled “characterization_200.zip” demonstrates the MC250 electrolyzer steady-state response with 30-minute load steps for a total duration of 5 hours. Finally, a .csv file is provided with all experiments combined into one dataset labeled "combined_solarPV_experiments.csv". [1] nelhydrogen.com/product/mc-series-electrolyser . [2] data.nlr.gov/submissions/316 .

08 HYDROGEN↗
DOE OSTI2024

A network approach for multiscale catchment classification using traits

Abstract. The classification of river catchments into groups with similar biophysical characteristics is useful to understand and predict their hydrological behavior. The increasing availability of remote sensing and other large-scale geospatial datasets has enabled the use of advanced data-driven approaches to classify catchments using traits such as topography, geology, climate, land cover, land use, and human influence. Unsupervised clustering algorithms based on the Euclidean distance are commonly used for trait-based classification but are not suitable for highly dimensional data. In this study we present a new network-based method for multi-scale catchment classification, which can be applied to large datasets and used to determine the traits associated with different catchment groups. In this framework, two networks are analyzed in parallel: the first being where the nodes are traits and the second being where the nodes are catchments. In both cases, edges represent pairwise similarity, and a network cluster detection algorithm is used for the classification. The trait network is used to investigate redundancy in the trait data and to condense this information into a small number of interpretable categories. The catchments network is used to classify the catchments into clusters and to identify representative catchments for the different groups using the degree centrality metric. We apply this method to classify 9067 river catchments across the contiguous United States at both regional and continental scales using 274 non-categorical traits. At the continental scale, we identify 25 interpretable trait categories and 34 catchment clusters of sizes greater than 50. We find that catchments with similar trait categories are typically located in the same region, with different spatial patterns emerging among clusters dominated by natural and anthropogenic traits. We also find that the catchment clusters exhibit distinct hydrological behavior based on an analysis of streamflow indices. This network approach provides several advantages over traditional means of classification, including better separation of clusters, the use of alternate similarity metrics that are more suitable for highly dimensional data, and reducing redundancy in the trait information. The paired catchment–trait networks enable analysis of hydrological behavior using the dominant trait categories for each catchment cluster. The approach can be used at multiple spatial scales since the network topologies adjust automatically to reflect the trait patterns at the scale of investigation. Finally, the representative catchments identified as hub nodes in the network can be used to guide transferable observational and modeling strategies. The method is broadly applicable beyond hydrology for classification of other complex systems that utilize different types of trait datasets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Interplay of Topography, Fire History, and Climate on Interior Alaska Boreal Forest Vegetation Dynamics in the 21st Century: A Landsat Time-Series Analysis

This study investigates vegetation dynamics in boreal forests of Interior Alaska, focusing on topography, fire history, and climate influences. The study area includes Bonanza Creek Experimental Forest (BCEF) and surrounding region, categorized by topography (upland, floodplain, lowland) and fire history. Using Mann–Kendall trend and Theil–Sen slope analyses on Landsat-derived spectral metrics: Normalized Difference Vegetation Index (NDVI) and Normalized Burn Ratio (NBR), we observed a shift from browning to greening trends, particularly in historically burned areas. The photosynthetic activity in burned upland converged with unburned areas ~30 years post-fire, coincident with a shift towards deciduous dominance during post-fire succession. Normalized Difference Moisture Index (NDMI) trends revealed a significant increase in vegetation moisture content across all topographies. We introduce Effective Seasonal Precipitation Index (ESPI), which combines prior-year annual precipitation with current-year spring snow depth. Its positive correlation with NDMI highlights its potential for monitoring vegetation moisture dynamics at the landscape scale. Furthermore, by correlating dendrochronology-based climate indices, we found strong correlation between NDMI and normalized Supplemental Precipitation Index (nSPI), across topographies. Overall, this research provides critical insights into how climate and fire influence interior boreal vegetation, highlighting the effects of increased precipitation, and topography on shaping differential vegetation responses across the landscape.

Google Earth Engine↗
DOE OSTI2021

Compensatory reserve detects subclinical shock with more expeditious prediction for need of life‐saving interventions compared to systolic blood pressure and blood lactate

Abstract Introduction We conducted a prospective observational study on 205 trauma patients at a level I trauma facility to test the hypothesis that a compensatory reserve measurement (CRM) would identify higher risk for progression to shock and/or need a life‐saving interventions (LSIs) earlier than systolic blood pressure (SBP) and blood lactate (LAC). Methods A composite outcome metric included blood transfusion, procedural LSI, and mortality. Discrete measures assessed as abnormal (ab) were SBP <90 mmHg, CRM <60%, and LAC >2.0. A graded categorization of shock was defined as: no shock (normal [n] SBP [n‐SBP], n‐CRM, n‐LAC); sub‐clinical shock (ab‐CRM, n‐SBP, n‐LAC); occult shock (n‐SBP, ab‐CRM, ab‐LAC); or overt shock (ab‐SBP, ab‐CRM, ab‐LAC). Results Three patients displayed overt shock, 53 displayed sub‐clinical shock, and 149 displayed no shock. After incorporating lactate into the analysis, 86 patients demonstrated no shock, 25 were classified as sub‐clinical shock, 91 were classified as occult shock, and 3 were characterized as overt shock. Each shock subcategory revealed a graded increase requiring LSI and transfusion. Initial CRM was associated with progression to shock (odds ratio = 0.97; p < .001) at an earlier time than SBP or LAC. Conclusions Initial CRM uncovers a clinically relevant subset of patients who are not detected by SBP and LAC. Our results suggest CRM could be used to more expeditiously identify injured patients likely to deteriorate to shock, with requirements for blood transfusion or procedural LSI.

Convertino, Victor A.↗
DOE OSTI2023

Computer Vision: EO Imaging & Video Tasks, Metrics, and Datasets [Slides]

Classification is an assignment of an image to a certain category. Localization is identified by the box surrounding the object in the image. Object detection detects instances of semantic objects of a certain class. Semantic segmentation categorizes all the pixels of an image into classes of objects. Instance segmentation identifies each instance of each object in the image. Panoptic (i.e., showing or seeing everything at once) segmentation is a combination of semantic and instance segmentation and classifies all the pixels in the image.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Use of Historical EM Practices and Policies for Management of Cleanup Program and Address GAO Concerns - 20222

As Department of Energy (DOE) Environmental Management (EM) program policies and management practices are revised over time, it might be useful to periodically reflect on the DOE EM program's past to identify practices that have worked and those that might be appropriate for possible reuse in the future. An added impetus to examine alternative approaches is the recent Government Accountability Office (GAO) report entitled 'Nuclear Waste Cleanup - DOE Could Improve Program and Project Management by Better Classifying Work and Following Leading Practices.' The February 2019 GAO report states that EM categorizes most of its work in a way that does not adequately involve independent experts and DOE senior leadership. It also noted that EM has not followed leading practices for program and project management that could help keep the cleanup efforts on schedule and control costs. The GAO report recommended that EM, working with DOE Office of Project Management (PM): - Establish requirements and then assess EM's ongoing operations activities to determine if some activities should be reclassified as capital asset projects based on these newly established requirements. - Review and revise EM's 2017 cleanup policy to include program management leading practices related to scope, cost, schedule performance, and independent reviews. - Update the cleanup policy to require that earned value management (EVM) systems be maintained and used in a way that follows EVM best practices. - Develop a policy to ensure that work is categorized as level of effort (LOE) only in appropriate, specified circumstances, such as when work is not measurable or when measurement is impractical. - Integrate EVM data into EM's performance metrics for operations activities. To assist with improving program performance and implementing audit recommendations, this paper reviewed past successful cleanups such as Weldon Springs, Rocky Flats, Fernald, Ashtabula, Battelle Columbus (abs, and other historical resources and people with experience during the early years of EM program. As such, there are several past practices which may well be suited for reuse today. Some of these practices include: - In 1990's, as the DOE EM cleanup program, known as Environmental Restoration came into effect, the known cleanup scope at each geographic location or site was organized into four levels ranging from overall site-wide cleanup effort to the individual 'release sites', providing high granularity to the program. - The cleanup program was organized by geography (Northwest, Southwest, Eastern Area Programs, etc.). These 'sub-program' managers provided a degree of Headquarters oversight and management allowing upper management (EM-1, 2, and 3) to focus on major decisions and issues. - Budget levels at selected cleanup sites were funded above 'minimum-safety' levels to accelerate actual cleanup work. - Past practice utilized EVM techniques and uniform EM oversight over all discrete project-like activities, providing for a coordinated cleanup effort. - EM managed LOE activities using benchmarking and best in class management practice techniques to help manage costs. - EM managed and reviewed entire site cleanup baselines, not only individual projects, focusing on a successful overall outcome. - Sites had a 'Road map' or 'Management Action Plan' that provided the overall strategy or plan for that site's cleanup - this helped with understanding and coordination of efforts. - EM had a robust lessons-learned program and database to capture and transmit worthy ideas. - EM had well-structured cleanup baseline change control policies to control cost and schedule changes, at the contractor and higher Headquarters levels. Finally, some of the GAO observations may be valid, however it is not clear as to their extent. For example, for some operation types of work, the use of the Level of Effort (LOE) earned value method to track progress may be appropriate. Also, to ensure independence of thought, and to obtain useful insight on past practices, it may be suitable to have former EM employees to assist with understanding past practices or to verify that activities have been properly identified. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Generation and evaluation of synthetic patient data

Background: Machine learning (ML) has made a significant impact in medicine and cancer research; however, its impact in these areas has been undeniably slower and more limited than in other application domains. A major reason for this has been the lack of availability of patient data to the broader ML research community, in large part due to patient privacy protection concerns. High-quality, realistic, synthetic datasets can be leveraged to accelerate methodological developments in medicine. By and large, medical data is high dimensional and often categorical. These characteristics pose multiple modeling challenges. Methods: In this paper, we evaluate three classes of synthetic data generation approaches; probabilistic models, classification-based imputation models, and generative adversarial neural networks. Metrics for evaluating the quality of the generated synthetic datasets are presented and discussed. Results: While the results and discussions are broadly applicable to medical data, for demonstration purposes we generate synthetic datasets for cancer based on the publicly available cancer registry data from the Surveillance Epidemiology and End Results (SEER) program. Specifically, our cohort consists of breast, respiratory, and non-solid cancer cases diagnosed between 2010 and 2015, which includes over 360,000 individual cases. Conclusions: We discuss the trade-offs of the different methods and metrics, providing guidance on considerations for the generation and usage of medical synthetic data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Modern chemical graph theory

Abstract Graph theory has a long history in chemistry. Yet as the breadth and variety of chemical data is rapidly changing, so too do graph encoding methods and analyses that yield qualitative and quantitative insights. Using illustrative cases within a basic mathematical framework, we showcase modern chemical graph theory's utility in Chemists' analysis and model development toolkit. The encoding of both experimental and simulation data is discussed at various levels of granularity of information. This is followed by a discussion of the two major classes of graph theoretical analyses: identifying connectivity patterns and partitioning methods. Measures, metrics, descriptors, and topological indices are then introduced with an emphasis upon enhancing interpretability and incorporation into physical models. Challenging data cases are described that include strategies for studying time dependence. Throughout, we incorporate recent advancements in computer science and applied mathematics that are propelling chemical graph theory into new domains of chemical study. This article is categorized under: Molecular and Statistical Mechanics > Molecular Dynamics and Monte‐Carlo Methods Structure and Mechanism > Computational Materials Science Structure and Mechanism > Molecular Structures

Leite, Leonardo S. G.↗
DOE OSTI2025

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Hyper Parameter Tuning in Neural Optical Image Categorizer for the E-log (NOICE

The Fermilab Accelerator Division Electronic logbook (E-log) is a record of all the activities and events in the Division for the past 10 years and more. The Elog search function is a valuable resource and the institutional memory of the accelerator complex. About 300,000 files are stored in the E-log, of which the vast majority are images attached to entries and comments. The visual information contained in the images is not presently searchable. The goal of team NOICE (Neural Optical Image Categorizer for the E-log) was to design a neural network able to produce label categories for these images for use by searches. The group developed a dataset and trained a convolutional neural network (CNN) with optimized hyperparameter. Final performance metrics are presented.

Njekeu, Terence Franck↗
DOE OSTI2022

Prescreening-Based Subset Selection for Improving Predictions of Earth System Models With Application to Regional Prediction of Red Tide

We present the ensemble method of prescreening-based subset selection to improve ensemble predictions of Earth system models (ESMs). In the prescreening step, the independent ensemble members are categorized based on their ability to reproduce physically-interpretable features of interest that are regional and problem-specific. The ensemble size is then updated by selecting the subsets that improve the performance of the ensemble prediction using decision relevant metrics. We apply the method to improve the prediction of red tide along the West Florida Shelf in the Gulf of Mexico, which affects coastal water quality and has substantial environmental and socioeconomic impacts on the State of Florida. Red tide is a common name for harmful algal blooms that occur worldwide, which result from large concentrations of aquatic microorganisms, such as dinoflagellate Karenia brevis, a toxic single celled protist. We present ensemble method for improving red tide prediction using the high resolution ESMs of the Coupled Model Intercomparison Project Phase 6 (CMIP6) and reanalysis data. The study results highlight the importance of prescreening-based subset selection with decision relevant metrics in identifying non-representative models, understanding their impact on ensemble prediction, and improving the ensemble prediction. These findings are pertinent to other regional environmental management applications and climate services. Additionally, our analysis follows the FAIR Guiding Principles for scientific data management and stewardship such that data and analysis tools are findable, accessible, interoperable, and reusable. As such, the interactive Colab notebooks developed for data analysis are annotated in the paper. This allows for efficient and transparent testing of the results’ sensitivity to different modeling assumptions. Moreover, this research serves as a starting point to build upon for red tide management, using the publicly available CMIP, Coordinated Regional Downscaling Experiment (CORDEX), and reanalysis data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Chapter 8. Hybrid Biological–Inorganic Systems for CO 2 Conversion to Fuels

The biological reduction of CO 2 into organic carbon is integral to the formation and maintenance of all life on earth. An urgent need to mitigate the effects of CO 2 on our atmosphere has provided an imperative to exploit biological CO 2 fixation for the conversion of CO2 into valuable and energy rich fuels. Photosynthesis uses only sunlight, water, and CO2 as starting materials to form all biomass.

CO2 Conversion to Fuels↗
DOE OSTI2023

Using High-Resolution NSRDB Data to Evaluate Cloud Mask Forecast from WRF-Solar EPS

Validating spatiotemporal distributions of cloud forecasts using numerical weather prediction (NWP) models is difficult as this requires high-quality cloud-property at significantly high spatial and temporal resolution over extended periods of time. Observations of cloud properties, such as cloud mask, cloud optical thickness, and cloud type, are vital for assessing the capability of NWP models to forecast various types of clouds. Using the National Solar Radiation Database (NSRDB), this research evaluates ensemble cloud-mask predictions from the WRF-Solar ensemble prediction system (WRF-Solar EPS). From the WRF-Solar EPS, day-ahead solar forecasts for the contiguous United States (CONUS) for 2018 are simulated. Given the NSRDB data is accessible at a resolution of 2 km, we can calculate the cloud fraction across the 9-km grid of WRF-Solar EPS. This allows us to spatially assess the cloud-mask forecasts using two methods against the high-resolution NSRDB: (a) considering all 2-km NSRDB clouds in the forecast domain (EMAll), and (b) using a minimum cloud fraction threshold of 50% to designate a pixel as cloudy (EMP50). The low-resolution cloud masks from WRF-Solar EPS are evaluated directly against the cloud-resolving scale gridded observations from NSRDB using EMAll. With EMP50, we presume that scenes with less than 50% cloud cover from the 2-km NSRDB are clear. Thus, this assessment approach allows for a fair comparison with WRF-Solar EPS resolved to a 9-km grid. A method of point-by-point verification is used to evaluate dichotomous (yes/no) cloud mask predictions against the NSRDB. For each pixel of model extent, cloud frequency and traditional metrics (e.g., probability of detection, false alarm rate, and hit rate, etc.) are computed and compared with satellite-derived data sets. Mismatched cloud frequency (MCF) is computed to measure the present capability of WRF-Solar EPS in representing various types of clouds, which are categorized using three levels of cloud top height (CTH) and cloud optical depth (COD) across entire CONUS. Preliminary results show that the WRF-Solar EPS provides MCF values ranging from 9% to 46%, 16% to 33%, and 8% to 27% for low-level, middle-level, and high-level clouds, respectively, for three CTHs. The model produces MCFs ranging from 27% to 46%, 13% to 34%, and 8% to 19% for thin, medium-thickness, and thick clouds, respectively, for three CODs. The presentation will include a detailed description of the current outcomes as well as potential future extensions. The evaluation approach established in this study is readily extensible to the evaluation of cloud predictions from different ensemble NWP models. In addition, the findings of the suggested evaluation technique aid in identifying model weaknesses and will ultimately lead to advances in WRF-Solar EPS's skill in predicting clouds and solar irradiance.

cloud mask forecast↗
DOE OSTI2020

A semblance measure for model comparison

Algorithmic and computational advances have made it possible that geophysical survey and earth model design can be aided by many systematic trial inverse-modelling runs with synthetic data. Such may, for example, come up in machine-learning approaches. Automated image appraisal pertaining to such applications will involve common statistical tests for goodness-of-data fit as a primary evaluation method. However, solution non-uniqueness may render multiple images equivalent in terms of their data fit, requiring secondary categorizers. A logical choice for classifying synthetic-imaging results quantifies the goodness of model fit where a known reference model replaces the observational input. The task of model intercomparison in terms of measuring the resemblance to the reference model poses challenges to common distance-based metrics like root mean square error and mean absolute error. First, distance-based metrics can introduce spurious contributions when smooth models with fuzzy target contours are to be compared against a sharp reference. Second, large differences due to parameter-estimation overshoots can dominate distance metrics. Here, we propose a remedy that is referred to as semblance and is based on the idea of logistic functions, where a binary-dependent variable adds non-zero or zero accumulation terms for the, respectively, passing or failing of preset target thresholds. This classifying approach is amenable to an objective where model feature recognition is primary. Numerical comparisons to distance-based metrics provide evidence for the advantages of the semblance in view of this objective. Geophysical imaging in conjunction with machine-learning is seen as a benefitting upcoming application area.

58 GEOSCIENCES↗
DOE OSTI2021

On the Utility of Transmission Color Analysis i: Differentiating Super-Earths and Sub-Neptunes

The majority of exoplanets found to date have been discovered via the transit method, and transmission spectroscopy represents the primary method of studying these distant worlds. Currently, in-depth atmospheric characterization of transiting exoplanets entails the use of spectrographs on large telescopes, requiring significant observing time to study each planet. Previous studies have demonstrated trends for solar system worlds using color–color photometry of reflectance spectra, as well as trends within transmission spectra for hot Jupiters. Building on these concepts, we have investigated the use of transmission color photometric analysis for efficient, coarse categorization of exoplanets and for assessing the nature of these worlds, with a focus on resolving the bulk composition degeneracy to aid in discriminating super-Earths and sub-Neptunes. We present our methodology and first results, including spectrum models, model comparison frameworks, and wave band selection criteria. We present our results for different transmission “color” metrics, filter selection methods, and numbers of filters. Assuming noise-free spectra of isothermal atmospheres in chemical equilibrium, with our pipeline, we are able to constrain atmospheric mean molecular weight in order to distinguish between super-Earth and sub-Neptune atmospheres with >90% overall accuracy using specific low-resolution filter combinations, . We also found that increasing the number of filters does not substantially impact this performance. This method could allow for broad characterization of large numbers of planets much more efficiently than current methods permit, enabling population- and system-level studies. Additionally, data collected via this method could inform follow-up observing time by large telescopes for more detailed studies of worlds of interest.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

ORNL Progress Report on Microreactor Functional Containment Economics

A summary of Oak Ridge National Laboratory’s (ORNL’s) initial contributions to the economic and functional containment evaluation approach for microreactors is presented. A final report will be completed at the end of 2022. This initial progress report does not represent a final product or an expression of a complete technical analysis. Initial contributions from ORNL can be categorized into two parts: (1) description of functional containment as a design optimization tool, and (2) summary of high level technology concepts for an evaluation of economic and functional containment analysis. Additionally, the initial portion of a functional containment evaluation is presented for a generic high temperature gas reactor technology. Remaining tasks include completing the functional containment evaluation by connecting economic and consequence metrics, as well as performing the same effort for other select reactor technologies.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗