Results for “data quality metrics” — Page 24

DOE OSTI2020

Quantifying Solar PV Variability at Multiple Timescales for Power Systems Studies: Preprint

Increasing penetration of solar photovoltaics (PV) has concerned power system operators about their ability to balance load and generation over multiple timescales and maintain acceptable frequency profile because of the variability and uncertainty associated with solar PV. This paper focuses on developing metrics to quantify solar PV variability at multiple temporal scales ranging from seconds to an hour to help power system operators and planners better understand and account for solar PV variability under high solar PV penetration. Fourier coefficients of solar irradiation spectrum and average size of solar irradiation ramps over a day are the two metrics that are used to quantity the variability. It is shown that by using the two metrics the days with highest and lowest solar irradiation variability can be identified, which is useful information for operations planning or long-term power system planning studies. The key findings of the paper are (i) single sensor-based variability quantification can significantly exaggerate the variability of a solar PV plant, particularly at seconds to 1-min temporal resolution, (ii) at 1-minute or lower temporal resolutions, single sensor-based variability quantification of an entire plant can be fairly accurate, with the accuracy increasing as the temporal resolution reduces from 1-minute to 1-hour, (iii) at each 1-minute and lower temporal resolutions, maximum solar irradiation ramps appear to be independent of the geographical location of the sensor, and (iv) significant reduction in solar PV plant variability over a region with multiple plants is possible at all temporal scales, but more high quality, high temporal resolution data is needed to quantify the reduction with high confidence.

photovoltaics↗

NASA NTRS2011

Report on the Global Data Assembly Center (GDAC) to the 12th GHRSST Science Team Meeting

In 2010/2011 the Global Data Assembly Center (GDAC) at NASA's Physical Oceanography Distributed Active Archive Center (PO.DAAC) continued its role as the primary clearinghouse and access node for operational Group for High Resolution Sea Surface Temperature (GHRSST) datastreams, as well as its collaborative role with the NOAA Long Term Stewardship and Reanalysis Facility (LTSRF) for archiving. Here we report on our data management activities and infrastructure improvements since the last science team meeting in June 2010.These include the implementation of all GHRSST datastreams in the new PO.DAAC Data Management and Archive System (DMAS) for more reliable and timely data access. GHRSST dataset metadata are now stored in a new database that has made the maintenance and quality improvement of metadata fields more straightforward. A content management system for a revised suite of PO.DAAC web pages allows dynamic access to a subset of these metadata fields for enhanced dataset description as well as discovery through a faceted search mechanism from the perspective of the user. From the discovery and metadata standpoint the GDAC has also implemented the NASA version of the OpenSearch protocol for searching for GHRSST granules and developed a web service to generate ISO 19115-2 compliant metadata records. Furthermore, the GDAC has continued to implement a new suite of tools and services for GHRSST datastreams including a Level 2 subsetter known as Dataminer, a revised POET Level 3/4 subsetter and visualization tool, a Google Earth interface to selected daily global Level 2 and Level 4 data, and experimented with a THREDDS catalog of GHRSST data collections. Finally we will summarize the expanding user and data statistics, and other metrics that we have collected over the last year demonstrating the broad user community and applications that the GHRSST project continues to serve via the GDAC distribution mechanisms. This report also serves by extension to summarize the activities of the GHRSST Data Assembly and Systems Technical Advisory Group (DAS-TAG).

sea surface temperature (SST)↗

DOE OSTI2025

Using a Large Language Model as a Building Block to Generate Usable Validation and Verification Suite for OpenMP

In the HPC area, both hardware and software move quickly. Often new hardware is developed and deployed, the corresponding software stack, including compilers and other tools, are under active development while leading edge software developers are working to port and tune their applications, all at the same time. While the software ecosystem is in flux, one of the key challenges for users is obtaining insight into the state of implementation of key features in the programming languages and models their applications are using – whether they have been implemented, and whether the implementation conforms to the specification, especially for newly implemented features (less tested by widespread use). OpenMP is one of the most prominent shared memory programming models used for on-node programming in HPC. With the shift towards accelerators (such as GPUs and FPGAs) and heterogeneous programming OpenMP features are getting more complex. It is natural to ask whether generative AI approaches, and large language models (LLMs) in particular, can help in producing validation and verification test suites to allow users better and faster insights into the availability and correctness of OpenMP features of interest. In this work, we explore the use of ChatGPT-4 to generate a suite of tests for OpenMP features. We have chosen a set of directives and clauses, a total of 78 combinations, which first appeared in OpenMP 3.0 (released in May 2008) but are also relevant for accelerators. We prompted ChatGPT to generate tests in the C and Fortran languages, for both host (CPU) and device (accelerator). On the Summit super-computer using the GNU implementation, we found that, of the 78 generated tests 67 C tests and 43 Fortran tests compiled successfully and fewer than those executed to completion. On further analysis we show that not all generated tests are valid. We document the process, results, and provide detailed analysis regarding the quality of tests generated. With the aim of providing input to a production quality validation and verification suite, we manually implement the corrections required to make the tests valid according to the current OpenMP specification. We quantify this effort as small, medium, or large, and record the lines of code changed to correct the invalid tests. With the corrected tests we validate recent implementations from HPE, AMD, and GNU on the Frontier supercomputer. Our experiment and subsequent analysis show that although LLMs are capable of producing HPC specific codes, they are limited by their understanding of the deeper semantics and restrictions of programming models such as OpenMP. Unsurprisingly more commonly used features have better support, while some OpenMP 3.0 directives such as sections and tasking are not universally supported on accelerators. We demonstrate that successful compilation and execution to completion are inadequate metrics for evaluating generated code and that, at this time, commodity LLMs require expert intervention for code verification. This points to gaps in the training data that is currently available for HPC. We demonstrate that with "small" effort 37% of generated invalid C tests and 63% of generated invalid Fortran tests could be corrected. This improves productivity of test generation as we circumvent writing from scratch and the common programming errors associated with it.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗

NASA NTRS2021

Quantifying the Impacts of PM2.5 Constituents and Relative Humidity on Visibility Impairment in a Suburban Area of Eastern Asia Using Long-Term In-Situ Measurements.

The deterioration of visibility due to air pollutants and relative humidity has been a serious environmental problem in eastern Asia. In most previous studies, chemical compositions of atmospheric particles were provided using filter-based offline analyses, which were unable to provide long-term and in-situ measurements that resolve sufficient temporal variations of air pollution and meteorology, hindering the resolution of the relationship between air quality and visibility. Here, we present a year-long continuously measured data from a comprehensive suite of online instruments to investigate diurnal and seasonal impacts of the aerosol chemical compositions in PM2.5 on visibility seasonally and diurnally. The measured dry aerosol extinction at λ = 550 nm reached a closure with that predicted by aerosol compositions within 12%. However, the hygroscopic growth of particles under ambient RH could enhance the aerosol extinction by a factor of 2 – 6, matching the perceptive visibility of the public. Particulate ammonium nitrate was most sensitive to reducing visibility, while ammonium sulfate contributed the most to the light extinction. In spring and winter, the monsoon and stagnant air masses reduced the visibility and increased PM2.5 (> 35 μg m-3).The moisture was found to substantially enhance the light extinction under RH = 60 – 90%,reducing visibility by approximately 15 km, largely attributed to hygroscopic inorganic salts.This study serves as a metric to highlight the need to consider the influence of RH, and aqueous reactions in producing secondary inorganic aerosols on atmospheric visibility, underpinning the more accurate mitigation strategies of air pollution.

visibility↗

DOE OSTI2023

Enhanced Surgical Decision-Making Tools in Breast Cancer: Predicting 2-Year Postoperative Physical, Sexual, and Psychosocial Well-Being following Mastectomy and Breast Reconstruction (INSPiRED 004)

Background: We sought to predict clinically meaningful changes in physical, sexual, and psychosocial well-being for women undergoing cancer-related mastectomy and breast reconstruction 2 years after surgery using machine learning (ML) algorithms trained on clinical and patient-reported outcomes data. Patients and Methods: We used data from women undergoing mastectomy and reconstruction at 11 study sites in North America to develop three distinct ML models. We used data of ten sites to predict clinically meaningful improvement or worsening by comparing pre-surgical scores with 2 year follow-up data measured by validated Breast-Q domains. We employed ten-fold cross-validation to train and test the algorithms, and then externally validated them using the 11th site’s data. We considered area-under-the-receiver-operating-characteristics-curve (AUC) as the primary metric to evaluate performance. Results: Overall, between 1454 and 1538 patients completed 2 year follow-up with data for physical, sexual, and psychosocial well-being. In the hold-out validation set, our ML algorithms were able to predict clinically significant changes in physical well-being (chest and upper body) (worsened: AUC range 0.69–0.70; improved: AUC range 0.81–0.82), sexual well-being (worsened: AUC range 0.76–0.77; improved: AUC range 0.74–0.76), and psychosocial well-being (worsened: AUC range 0.64–0.66; improved: AUC range 0.66–0.66). Baseline patient-reported outcome (PRO) variables showed the largest influence on model predictions. Conclusions: Machine learning can predict long-term individual PROs of patients undergoing postmastectomy breast reconstruction with acceptable accuracy. This may better help patients and clinicians make informed decisions regarding expected long-term effect of treatment, facilitate patient-centered care, and ultimately improve postoperative health-related quality of life.

60 APPLIED LIFE SCIENCES↗

NASA NTRS2019

The ILRS: Approaching 20 Years and Planning for the Future

The International Laser Ranging Service (ILRS) was established by the International Association of Geodesy (IAG) in 1998 to support programs in geodesy, geophysics, fundamental constants and lunar research, and to provide the International Earth Rotation Service with data products that are essential to the maintenance and improvement in the International Terrestrial Reference Frame (ITRF), the basis for metric measurements of changes in the Earth and Earth–Moon system. Other scientific products derived from laser ranging include precise geocentric positions and motions of ground stations, satellite orbits, components of Earth’s gravity field and their temporal variations, Earth Orientation Parameters, precise lunar ephemerides and information about the internal structure of the Moon. Laser ranging systems are already measuring the one-way distance to remote optical receivers in space and are performing very accurate time transfer between remote sites in the Earth and in Space. The ILRS works closely with the IAG’s Global Geodetic Observing System. The ILRS develops (1) the standards and specifications necessary for product consistency, and (2) the priorities and tracking strategies required to maximize network efficiency. The service collects, merges, analyzes, archives and distributes satellite and lunar laser ranging data to satisfy a variety of scientific, engineering, and operational needs and encourages the application of new technologies to enhance the quality, quantity, and cost effectiveness of its data products. The ILRS works with (1) new satellite missions in the design and building of retroreflector targets to maximize data quality and quantity, and (2) science programs to optimize scientific data yield. Since its inception, the ILRS has grown to include forty laser ranging stations distributed around the world. The ILRS stations track more than ninety satellites from low Earth orbit (LEO) to the geosynchronous orbit altitude as well as retroreflector arrays on the surface of the Moon. Applications have been expanded to include time transfer, asynchronous ranging for targets at extended ranges, free space quantum telecommunications, and the tracking of space debris. Laser ranging technology is moving to lower energy, higher repetition rates (kHz), single-photon-sensitive detectors, shorter pulse widths, shorter normal point intervals for faster data acquisition, and increased pass interleaving, automated to autonomous operation with remote access, and embedded software for real-time updates and decision making. An example of pass interleaving is presented for the Yarragadee station (see Fig. 4); tracking of LEO satellites is often accommodated during break in LEO and GNSS passes. New satellites arrays provide more compact targets and work continues on the development of lighter less expensive arrays for satellites and the moon. The service now provides operational ITRF products including daily/ weekly station positions and daily resolution Earth orientation products; the flow of weekly combination of satellite orbit files for LAGEOS/Etalon-1 and -2 has recently been established. New products are under testing through a pilot project on systematic error monitoring currently underway. The article will give an overview of activities underway within the service, paths forward presently envisioned, and current issues and challenges.

Laser retroreflectors↗

NASA NTRS2010

HOPE: An On-Line Piloted Handling Qualities Experiment Data Book

A novel on-line database for capturing most of the information obtained during piloted handling qualities experiments (either flight or simulated) is described. The Hyperlinked Overview of Piloted Evaluations (HOPE) web application is based on an open-source object-oriented Web-based front end (Ruby-on-Rails) that can be used with a variety of back-end relational database engines. The hyperlinked, on-line data book approach allows an easily-traversed way of looking at a variety of collected data, including pilot ratings, pilot information, vehicle and configuration characteristics, test maneuvers, and individual flight test cards and repeat runs. It allows for on-line retrieval of pilot comments, both audio and transcribed, as well as time history data retrieval and video playback. Pilot questionnaires are recorded as are pilot biographies. Simple statistics are calculated for each selected group of pilot ratings, allowing multiple ways to aggregate the data set (by pilot, by task, or by vehicle configuration, for example). Any number of per-run or per-task metrics can be captured in the database. The entire run metrics dataset can be downloaded in comma-separated text for further analysis off-line. It is expected that this tool will be made available upon request

Jackson, E. B.↗

DOE OSTI2024

Algal Biomass Production via Open Pond Algae Farm Cultivation: 2023 State of Technology and Future Research

The annual State of Technology (SOT) assessment is an essential activity for platform research conducted under the Bioenergy Technologies Office (BETO). It allows for the impact of research progress (both directly achieved in-house at the National Renewable Energy Laboratory [NREL] and furnished by partner organizations) to be quantified in terms of economic improvements in the overall biofuel production process for a particular biomass processing pathway, whether based on terrestrial or algal biomass feedstocks. As such, initial benchmarks can be established for currently demonstrated performance, and progress can be tracked toward out-year goals to ultimately demonstrate economically viable biofuel technologies. NREL's algae SOT benchmarking efforts historically focused both on front-end algal biomass production and separately on back-end conversion to fuels through NREL's "combined algae processing" (CAP) pathway. The production model is based on outdoor long-term cultivation data, enabled by comprehensive algal biomass production trials conducted under the Development of Integrated Screening, Cultivar Optimization, and Verification Research (DISCOVR) consortium efforts, driven by data furnished by Arizona State University (ASU) at the Arizona Center for Algae Technology and Innovation (AzCATI) testbed site. The CAP model is based on experimental efforts conducted primarily under NREL research and development projects. This report focuses on front-end algal biomass production, documenting the pertinent algal biomass cultivation parameters that were input to the NREL open pond algae farm model. Through partnerships under DISCOVR, collaborators at ASU furnished details on cultivation performance metrics including biomass productivity and harvest densities for recent growth trials done at the AzCATI site. The resulting biomass productivity was calculated at 16.7 g/m 2 /day (ash-free dry weight [AFDW], annual average) for seasonal cultivation of Picochlorum celeri TG2 and Monoraphidium minutum 26B-AM biomass strains at the ASU site. Picochlorum celeri achieved the best productivity from April to September, with Monoraphidium minutum 26B-AM being used between October and March. Tetraselmis striata LANL1001, usually part of the strain rotation in previous cultivation SOTs, was supplanted by Monoraphidium minutum 26B-AM in this year's outdoor cultivation trials. Finally, building from an industry case study presented in the 2022 SOT report, in the Appendix of this report we provide an update on further improved data furnished by an industry collaborator and resultant impacts on economics reflecting several seasonal scenarios. This case study provides a supplementary datapoint on work being performed elsewhere with a more dedicated focus on improved compositional quality, producing biomass enriched in lipids as may be more optimal for conversion upgrading to fuels and products.

09 BIOMASS FUELS↗

DOE OSTI2021

JGI QC impact on assembly, binning, phylogenomics, and functional analysis

Background Investigators using metagenomic sequencing to study their microbiomes are often provided data that has been trimmed and decontaminated or do it themselves without knowing the effect these procedures can have on their downstream analyses. Here we evaluated the impact that JGI trimming and decontamination procedures had on assembly and binning metrics, placement of metagenome assembled genomes into species trees, and functional profiles of metagenome-assembled genomes (MAGs) extracted from twenty three complex rhizosphere metagenomes. We also investigated how more aggressive trimming impacts these binning metrics. Results We found that JGI trimmed and decontamination of input reads had some significant impacts in assembly and binning metrics compared to raw reads, and that differences in placement of MAGs in species trees increased with decreasing completeness and contamination thresholds. More aggressive trimming beyond those used by JGI were found to reduce MAG counts. Conclusions Mild trimming and decontamination of metagenomics reads prior to assembly can change an investigator’s answer to the questions, “Who is there and what are they doing? However, mild trimming and decontamination of metagenomic reads with high quality scores is recommended for those who elect to do so.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2021

Impact of BBDuk metagenomic read trimming and decontamination

Background Investigators using metagenomic sequencing to study their microbiomes are often provided data that has been trimmed and decontaminated or do it themselves without knowing the effect these procedures can have on their downstream analyses. Here we evaluated the impact that JGI trimming and decontamination procedures had on assembly and binning metrics, placement of metagenome assembled genomes into species trees, and functional profiles of metagenome-assembled genomes (MAGs) extracted from twenty three complex rhizosphere metagenomes. We also investigated how more aggressive trimming impacts these binning metrics. Results We found that JGI trimmed and decontamination of input reads had some significant impacts in assembly and binning metrics compared to raw reads, and that differences in placement of MAGs in species trees increased with decreasing completeness and contamination thresholds. More aggressive trimming beyond those used by JGI were found to reduce MAG counts. Conclusions Mild trimming and decontamination of metagenomics reads prior to assembly can change an investigator’s answer to the questions, “Who is there and what are they doing? However, mild trimming and decontamination of metagenomic reads with high quality scores is recommended for those who elect to do so.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2015

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

NASA NTRS2015

Improving Climate Projections Using "Intelligent" Ensembles

Recent changes in the climate system have led to growing concern, especially in communities which are highly vulnerable to resource shortages and weather extremes. There is an urgent need for better climate information to develop solutions and strategies for adapting to a changing climate. Climate models provide excellent tools for studying the current state of climate and making future projections. However, these models are subject to biases created by structural uncertainties. Performance metrics-or the systematic determination of model biases-succinctly quantify aspects of climate model behavior. Efforts to standardize climate model experiments and collect simulation data-such as the Coupled Model Intercomparison Project (CMIP)-provide the means to directly compare and assess model performance. Performance metrics have been used to show that some models reproduce present-day climate better than others. Simulation data from multiple models are often used to add value to projections by creating a consensus projection from the model ensemble, in which each model is given an equal weight. It has been shown that the ensemble mean generally outperforms any single model. It is possible to use unequal weights to produce ensemble means, in which models are weighted based on performance (called "intelligent" ensembles). Can performance metrics be used to improve climate projections? Previous work introduced a framework for comparing the utility of model performance metrics, showing that the best metrics are related to the variance of top-of-atmosphere outgoing longwave radiation. These metrics improve present-day climate simulations of Earth's energy budget using the "intelligent" ensemble method. The current project identifies several approaches for testing whether performance metrics can be applied to future simulations to create "intelligent" ensemble-mean climate projections. It is shown that certain performance metrics test key climate processes in the models, and that these metrics can be used to evaluate model quality in both current and future climate states. This information will be used to produce new consensus projections and provide communities with improved climate projections for urgent decision-making.

Baker, Noel C.↗

NASA NTRS2016

Demonstration of Mars Crosslink Occultation Measurements for Future Small Spacecraft Constellations

Future planetary atmospheric profiling via radio occultations (RO) benefits significantly from increased received signal-to-noise ratio as well as geometrical coverage of links between two or more spacecraft in orbit around a target planet. These can be small spacecraft, possibly dedicated to quality radio-metrics with precision clock reference. Motivated by long-term research investigating the optimum SNR for the science performance, the optimum reference clock stability, the number and combination of wavelengths driving the number of transmitters and antennas, the optimum number and orbital spacecraft configuration, and design modifications to existing radio communication systems that would allow these new science objectives on future missions, three crosslink occultation experiments have been acquired between the Mars Odyssey and Mars Reconnaissance Orbiter spacecraft to probe the Martian atmosphere. While crosslink occultations between Earth orbiting satellites have long been used to profile the Earth’s atmosphere, this represents the first demonstration of crosslink occultation measurements at another planet. These measurements leverage the proximity link telecommunication payloads on each orbiter, which were designed to provide relay communication and navigation services to Mars landers and rovers. Analysis of the observed Doppler shift on each crosslink measurement reveals a clear signature of the Martian atmosphere, primarily the ionosphere. Inversion of the observed Doppler data yields vertical profiles of the Martian refractivity and electron density. The electron density profiles show the presence of two layers with peak densities and peak heights that are consistent with empirical models.

Asmar, Sami↗

NASA NTRS1994

Method for optimizing resource allocation in a government organization

The managers in Federal agencies are challenged to control the extensive activities in government and still provide high-quality products and services to the American taxpayers. Considering today's complex social and economic environment and the $3.8 billion daily cost of operating the Federal Government, it is evident that there is a need to develop decision-making tools for accurate resource allocation and total quality management. The goal of this thesis is to provide a methodical process that will aid managers in Federal Government to make budgetary decisions based on the cost of services, the agency's objectives, and the customers' perception of the agency's product. A general resource allocation procedure was developed in this study that can be applied to any government organization. A government organization, hereafter the 'organization,' is assumed to be a multidivision enterprise. This procedure was applied to a small organization for the proof of the concept. This organization is the Technical Services Directorate (TSD) at the NASA Lewis Research Center in Cleveland, Ohio. As part of the procedure, a nonlinear programming model was developed to account for the resources of the organization, the outputs produced by the organization, the decision-maker's views, and the customers' satisfaction with the organization. The information on the resources of the organization was acquired from current budget levels of the organization and the human resources assigned to the divisions. The outputs of the organization were defined and measured by identifying metrics that assess the outputs, the most challenging task in this study. The decision-maker's views are represented in the model as weights assigned to the various outputs and were quantified by using the analytic hierarchy process. The customer's opinions regarding the outputs of the organization were collected through questionnaires that were designed for each division individually. Following the philosophy of total quality management, information on customers' satisfaction is presented in the model as the quality of output. The model is a nonlinear one whose objective is to maximize customers' satisfaction such that the total cost of operation does not exceed the organization's budget. This model represents a structured approach or policy mechanism, at the agency level, to make capital investment decisions based on the priorities of the agency and the quality of outputs. This procedure applied to TSD resulted in a resources allocation scheme that was reasonable and acceptable to the decision-makers and, as expected, dependent on the assumptions and accuracy of the data used in the model.

Afarin, James↗

DOE OSTI2023

Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process

Local Interpretable and Model-agnostic Explanation (LIME), an explainable artificial intelligence (XAI) approach is adapted to identify the globally important time-frequency bands for predicting average surface roughness (Ra) in a smart grinding process. The smart grinding setup consisted of a Supertech CNC precision surface grinding machine, instrumented with a Dytran piezoelectric accelerometer attached to the tailstock quill along the tangential direction (Y-axis). For every grinding pass, vibration signatures were captured, and the ground truth surface roughness values were recorded using a Mahr Marsurf M300C portable surface roughness profilometer. The roughness values ranged from 0.06 to 0.14 microns over the complete set of experiments. Time-frequency domain spectrogram frames were extracted for each of the vibration signals collected during the grinding process. Convolutional Neural Networks (CNNs) were modeled to predict the surface roughness based on these spectrogram frames and their image augmentations. The best CNN model was able to predict the roughness values with an overall R2-score of 0.95, training R2-score of 0.99, and testing R2-score of 0.81 with only 80 sets of vibration signals corresponding to 4 experiments with 20 trials each. Although the data size is not large enough to guarantee such performance metrics in real-world scenarios, one can extract statistically consistent explanations underlying the relationships these complex deep learning models capture. Further, the LIME methodology was implemented on the developed surface roughness CNN model to identify the important time-frequency bands (i.e., the superpixels of a spectrogram) influencing the predictions. Based on the identified important regions on the spectrogram frames, the corresponding frequency characteristics were determined that influence the surface roughness predictions. The important frequency range based on LIME results was approximately 11.7 to 19.1 kHz. The power of XAI was demonstrated by cutting down the sampling rate from 160 kHz to 30, 20, 10, and 5 kHz based on the important frequency range and considering Nyquist criteria. Separate CNN models were developed for these ranges by only extracting time-frequency contents below their corresponding Nyquist cut-offs. A proper data acquisition strategy is proposed by comparing the model performances to argue the selection of a sufficient sampling rate to capture the grinding process successfully and robustly.

42 ENGINEERING↗

DOE OSTI2023

Reprint of: Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process

Local Interpretable and Model-agnostic Explanation (LIME), an explainable artificial intelligence (XAI) approach is adapted to identify the globally important time-frequency bands for predicting average surface roughness (Ra) in a smart grinding process. The smart grinding setup consisted of a Supertech CNC precision surface grinding machine, instrumented with a Dytran piezoelectric accelerometer attached to the tailstock quill along the tangential direction (Y-axis). For every grinding pass, vibration signatures were captured, and the ground truth surface roughness values were recorded using a Mahr Marsurf M300C portable surface roughness profilometer. The roughness values ranged from 0.06 to 0.14 microns over the complete set of experiments. Time-frequency domain spectrogram frames were extracted for each of the vibration signals collected during the grinding process. Convolutional Neural Networks (CNNs) were modeled to predict the surface roughness based on these spectrogram frames and their image augmentations. The best CNN model was able to predict the roughness values with an overall R2-score of 0.95, training R2-score of 0.99, and testing R2-score of 0.81 with only 80 sets of vibration signals corresponding to 4 experiments with 20 trials each. Although the data size is not large enough to guarantee such performance metrics in real-world scenarios, one can extract statistically consistent explanations underlying the relationships these complex deep learning models capture. Further, the LIME methodology was implemented on the developed surface roughness CNN model to identify the important time-frequency bands (i.e., the superpixels of a spectrogram) influencing the predictions. Based on the identified important regions on the spectrogram frames, the corresponding frequency characteristics were determined that influence the surface roughness predictions. The important frequency range based on LIME results was approximately 11.7 to 19.1 kHz. The power of XAI was demonstrated by cutting down the sampling rate from 160 kHz to 30, 20, 10, and 5 kHz based on the important frequency range and considering Nyquist criteria. Separate CNN models were developed for these ranges by only extracting time-frequency contents below their corresponding Nyquist cut-offs. A proper data acquisition strategy is proposed by comparing the model performances to argue the selection of a sufficient sampling rate to capture the grinding process successfully and robustly.

47 OTHER INSTRUMENTATION↗

DOE OSTI2025

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2026

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗