Search NASA⌕ Search

SEARCH · Search NASA

Results for “science metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Identifying materials-level sources of performance variation in superconducting transmon qubits

The Superconducting Quantum Materials and Systems Center, a U. S. Department of Energy National Quantum Information Science Research Center, has conducted a comprehensive and coordinated study using superconducting transmon qubit chips with known performance metrics to identify the underlying materials-level sources of device-to-device performance variation. Following qubit coherence measurements, these qubits of varying base superconducting metals and substrates have been examined with various non-destructive and invasive material characterization techniques at Northwestern University, Ames National Laboratory, and Fermilab as part of a blind study. We find trends in variations of the depth of the etched substrate trench, the thickness of the surface oxide, and the geometry of the sidewall, which when combined, lead to correlations with the T1 lifetime across different qubits on the same chip. In addition, we provide a list of features that varied from device to device, for which the impact on performance requires further studies. Finally, we identify two low-temperature characterization techniques that may potentially serve as proxy tools for qubit measurements. These insights provide materials-oriented solutions to not only reduce performance variations across neighboring devices but also to engineer and fabricate devices with optimal geometries to achieve performance metrics beyond the state-of-the-art values.

Murthy, Akshay A. [Fermilab] (ORCID:00000001767768↗
DOE OSTI2023

A Mitigation Strategy for the Prediction Inconsistency of Neural Phase Pickers

Neural phase pickers—neural networks designed and trained to pick seismic phase arrivals—have proven to be a powerful tool for developing earthquake catalogs. However, these pickers suffer from prediction inconsistency in which the results they produce change, sometimes substantially, even under a small perturbation to the input waveform. This problem has not been addressed by the developers and users of these pickers. In this study, we show how prediction inconsistency can negatively affect the completeness of earthquake catalogs developed using neural phase pickers. Further, we show that simply using a small step size for the sliding window when processing continuous waveform data and aggregating the results significantly mitigates this problem. We also highlight the importance of training datasets for increasing the consistency and other performance metrics.

58 GEOSCIENCES↗
DOE OSTI2022

How Reproducible are Surface Areas Calculated from the BET Equation?

Abstract Porosity and surface area analysis play a prominent role in modern materials science. At the heart of this sits the Brunauer–Emmett–Teller (BET) theory, which has been a remarkably successful contribution to the field of materials science. The BET method was developed in the 1930s for open surfaces but is now the most widely used metric for the estimation of surface areas of micro‐ and mesoporous materials. Despite its widespread use, the calculation of BET surface areas causes a spread in reported areas, resulting in reproducibility problems in both academia and industry. To prove this, for this analysis, 18 already‐measured raw adsorption isotherms were provided to sixty‐one labs, who were asked to calculate the corresponding BET areas. This round‐robin exercise resulted in a wide range of values. Here, the reproducibility of BET area determination from identical isotherms is demonstrated to be a largely ignored issue, raising critical concerns over the reliability of reported BET areas. To solve this major issue, a new computational approach to accurately and systematically determine the BET area of nanoporous materials is developed. The software, called “BET surface identification” (BETSI), expands on the well‐known Rouquerol criteria and makes an unambiguous BET area assignment possible.

36 MATERIALS SCIENCE↗
DOE OSTI2025

SITCOMTN-154: Initial studies of photometric redshifts with LSSTComCam from DP1

This technote holds reports based on the first analyses of the Data Preview 1 (DP1) data by the Science Unit for photometric redshifts. Although photometric redshifts are not an official DP1 data product, the "Photo-z Science Unit" generated photo-z estimates for every galaxy in DP1 using the available multi-band imaging on a best-effort basis. This work included developing training and test datasets by matching DP1 data to high-quality reference redshifts obtained with spectroscopy, Grism data, and multi-band photometry. The Science Unit used the RAIL software package to make photometric redshift estimates using eight different algorithms, developed simple scientific performance metrics, used those metrics to explore how the performance of the algorithms varied with configuration changes, derived more optimized configurations of the algorithms and tested the performance of those configurations. This work, the resulting data products and expected data distribution mechanism are all described there.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Quantification of heterogeneity in human CD8 + T cell responses to vaccine antigens: an HLA-guided perspective

Vaccines have historically played a pivotal role in controlling epidemics. Effective vaccines for viruses causing significant human disease, e.g., Ebola, Lassa fever, or Crimean Congo hemorrhagic fever virus, would be invaluable to public health strategies and counter-measure development missions. Here, we propose coverage metrics to quantify vaccine-induced CD8 + T cell-mediated immune protection, as well as metrics to characterize immuno-dominant epitopes, in light of human genetic heterogeneity and viral evolution. Proof-of-principle of our approach and methods are demonstrated for Ebola virus, SARS-CoV-2, and Burkholderia pseudomallei (vaccine) proteins.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

American cities in a time of global environmental change: the case of the Baltimore Social-Environmental Collaborative

The Baltimore Social-Environmental Collaborative (BSEC) Urban Integrated Field Laboratory seeks a new paradigm for urban climate research. Motivated by deep uncertainties in urban climate and the future of urban systems, BSEC works collaboratively across institutions and stakeholder groups to co-generate the science needed to advance energy security and resilience to extreme events across the city of Baltimore, Maryland, USA, and to do so in a manner that can inform similar efforts in other cities. BSEC begins with stakeholder priorities (health, affordable energy, etc) and designs observation networks and models to deliver climate science to address them. This takes the form of an iterative collaborative cycle, in which an initial research strategy is repeatedly updated in conversation with community partners, and researchers and stakeholders learn from each other. To date, this cycle has included multiple rounds of collaborative deliberation on urban heat mitigation, in which a multicriteria decision tool has been updated with more community-relevant spatial structure and modified optimization metrics. The guiding objective of this cycle is to inform potential ‘secure and resilient pathways’ for energy and infrastructure. In doing so, BSEC addresses fundamental urban science questions in natural and social sciences. It also tests our ability to integrate this science in a manner that advances participatory decision-making for urban resilience.

climate↗
DOE OSTI2022

Achievements and Lessons Learned From Successful Small Satellite Missions for Space Weather‐Oriented Research

Abstract When the first CubeSats were launched nearly two decades ago, few people believed that the miniature satellites would likely prove to be a useful scientific tool. Skeptics abounded. However, the last decade has seen the highly successful implementation of space missions that make creative and innovative use of fast‐advancing CubeSat and small satellite technology to carry out important science experiments and missions. Several projects now have used CubeSats to obtain first‐of‐their‐kind observations and findings that have formed the basis for high‐profile engineering and science publications, thereby establishing without doubt the scientific value and broad utility of CubeSats. In this paper, we describe recent achievements and lessons learned from a representative selection of successful CubeSat missions with a space weather focus. We conclude that these missions were successful in part because their limited resources promoted not only mission focus but also appropriate risk‐taking for comparatively high science return. Quantitative analysis of refereed publications from these CubeSat missions and several larger missions reveals that mission outcome metrics compare favorably when publication number is normalized by mission cost or if expressed as a weighted net scientific impact of all mission publications.

Spence, Harlan E.↗
DOE OSTI2020

Replicating Machine Learning Experiments in Materials Science

Transparency and reproducibility are important aspects of validation for Machine Learning (ML) models that are not fully understood and applies independently of the application domain.We offer a case study of reproducibility that highlights the challenges encountered when attempting to reproduce analyzes obtained with Machine Learning methods in materials informatics. Our study explores prediction results obtained with ML models and issues in training data serving as input. We discuss challenges related to theory-driven and numerical errors in training data, lack of reproducibility across platforms and versions, and effects of randomness when varying hyperparameters. In addition to model accuracy, a main metric of interest in the ML community, our results show that model sensitivity may be equally important for applying ML in domain applications such a materials science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Pathway to Decarbonization Through Industrial Energy Efficiency: Micro and Macro Perspectives from Compressed Air Usage

Abstract Energy audits directly provided the industrial sector with reduced energy costs and avoided emissions. Still, they also lead to far-reaching indirect and induced local, regional, and national benefits. This paper aims to present the techno-economic-environmental analysis to achieve decarbonization through implementing industrial energy efficiency at micro and macro levels. An integrated techno-economic-environmental methodology is developed. Case studies of micro-level carbon reduction efforts through industrial energy efficiency technologies are presented. The broader macroeconomic and environmental effects of technology on society are analyzed using data from 206 energy audits of industrial compressed air systems conducted over 13 years. The impacts show that energy-efficient improvements lead to direct cost savings for manufacturers, boost economic activity across sectors, and affect carbon dioxide emissions both short-term and long-term in the region. Given their extensive benefits, energy audits significantly influence policymaking. We devised a methodology to link micro-level energy audit data with macroeconomic and environmental analyses to quantify these cascading benefits. The economic scenario analysis shows that $228 M has been saved from direct industrial energy savings from implementing all compressed air recommendations in the studied periods and the region. In addition, the investment made through manufacturers would create 2,025 jobs and $383 M annually, cascading regional economic impacts. The environmental analysis shows that the regional manufacturers have directly avoided about 2.8 M metric tons of carbon dioxide emissions.

Engineering↗
DOE OSTI2022

Effect of substrate orientation and anisotropic strength on corrosion pits

The complexities of pitting corrosion and its associated pit morphologies are critical for predicting and preventing pit-related failure in materials. With the advances in computational tools, modeling of pitting corrosion has lately gained momentum. We recently developed a phase field model for 3-D pit morphology, and in the current work, the model is utilized to gain insight into the role of substrate orientation and anisotropic strength on crystallographic pits. The influence of these parameters on pit morphology and corrosion metrics such as pit volume, current, pit area, current density, and pit depth are studied. The crystallographic pit morphology changes with substrate orientation but not significantly with anisotropic strength. Finally, the anisotropic strength strongly influences the corrosion metrics whereas most corrosion metrics remain unaffected by substrate orientation.

36 MATERIALS SCIENCE↗
DOE OSTI2025

An Observational Evaluation of RKW Theory over the U.S. Southern Great Plains

The theory of Rotunno et al. (“RKW” theory) addresses the behavior of squall-line cold pools in vertically sheared flows. It predicts that, within a given thermodynamic environment, a balance between baroclinic vorticity generation by the cold pool and low-level environmental vertical wind shear induces an upright updraft along the gust front that maximizes the initiation of new convective cells. Although this theory has been evaluated numerically, its applicability to observed systems remains unclear and is limited by a lack of critical measurements, including high-frequency thermodynamic and wind profiles across the gust front. Herein, observations from the Atmospheric Radiation Measurement Southern Great Plains (ARM-SGP) observatory near Lamont, Oklahoma, are used to evaluate RKW theory for 10 well-observed squall lines over a 11-yr period. For this evaluation, RKW parameters including cold-pool intensity (c), low-level ambient, line-normal vertical shear (ΔV n ), subcloud and cloud-layer updraft tilts, and multiple measures of system intensity are estimated. Furthermore, the c estimates rely on thermodynamic retrievals from the Atmosphere Emitted Radiance Interferometer (AERI), which are uncertain but verify reasonably well against independent observations. As predicted by the theory, for c/ΔV n ≥ 1, c/ΔV n correlates positively with updraft tilt and negatively with system intensity, but these results are not always statistically significant and are also sensitive to the method by which ΔV n is evaluated. Specifically, ΔV n evaluations that extend above the cold-pool top yield greater consistency with RKW predictions. Also, some measures of intensity correlate more strongly with standard moist instability metrics than with RKW parameters.

Cold pools↗
DOE OSTI2020

Discrete Element Method Analysis for Metal Powders Used in Additive Manufacturing, and DEM Simulation Tutorial Using LIGGGHTS-PUBLIC [PowerPoint and paper]

Discrete Element Method (DEM) is a method of analysis to evaluate the dynamic interactions between granular particles. This method has been used in the pharmaceutical industry to improve the powder compaction process for tablet manufacturing. There are also applications in agriculture, food industry, and manufacturing. Direct energy deposition is an additive manufacturing technique which uses metallic powders fed through a nozzle, melted using a directed laser, and transformed into a solid object layer by layer. One way of feeding metal particles into the system involves the use of a vibrating hopper. Given a specified amplitude and frequency input, the hopper will enable the powder to travel up a path, and inject through the system with assistance from a stream of gas. The mechanical properties of a printed object can vary, depending on the characteristics of the powder flow and the particles’ as-received properties. Improved understanding of dynamic interactions of flowing powders could enable additive manufacturing components with 2D or 3D variations in mechanical properties, e.g., density. This work uses DEM simulation software to investigate the effects of particle cohesion, friction, and density on the quality of the flow by performing an angle of repose simulation, which is often used as a metric to evaluate the flowability of powders.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Quantification of morphological change in materials based on image data utilizing machine learning techniques

Computed tomography (CT) resolution has become high enough to monitor morphological changes due to aging in materials in long-term applications. We explored the utility of the critic of a generative adversarial network (GAN) to automatically detect such changes. The GAN was trained with images of pristine Pharmatose, which is used as a surrogate energetic material. It is important to note that images of the material with altered morphology were only used during the test phase. The GAN-generated images visually reproduced the microstructure of Pharmatose well, although some unrealistic particle fusion was seen. Calculated morphological metrics (volume fraction, interfacial line length, and local thickness) for the synthetic images also showed good agreement with the training data, albeit with signs of mode collapse in the interfacial line length. While the critic exposed changes in particle size, it showed limited ability to distinguish images by particle shape. The detection of shape differences was also a more challenging task for the selected morphological metrics that related to energetic material performance. We further tested the critic with images of aged Pharmatose. Subtle changes due to aging are difficult for the human analyst to detect. Both critic and morphological metrics analysis showed image differentiation.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Doppler effect in rotating systems and Mössbauer rotor experiments

We address once more the problem of the physical interpretation of the Mössbauer effect in a rotating system in the view of the recently discovered extra energy shift (EES) between emitted and absorbed resonant radiation, which emerges in addition to the usual second order Doppler shift for an orbiting absorber. We highlight that both fractional components of the total energy shift have the same sign and correspond to the blueshift of the resonant radiation propagating from the rotational axis to the rotor rim. We indicate a possible way of understanding the origin of the EES in terms of combining metric effects in rotating systems with the quantum mechanical description of resonant nuclei confined in crystal cells by taking into account the recoil-free essence of the Mössbauer effect, where no energy is transmitted to the source (absorber) during the emission (absorption) of resonant radiation.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Matching Crystal Structures Atom-to-Atom

Finding an optimal match between two different crystal structures underpins many important materials science problems, including describing solid-solid phase transitions and developing models for interface and grain boundary structures. Here, we formulate the matching of crystals as an optimization problem where the goal is to find the alignment and the atom-to-atom map that minimize a given cost function such as the Euclidean distance between the atoms. We construct an algorithm that directly solves this problem for large finite portions of the crystals and retrieves the periodicity of the match subsequently. We demonstrate its capacity to describe transformation pathways between known polymorphs and to reproduce experimentally realized structures of semi-coherent interfaces. Additionally, from our findings, we define a rigorous metric for measuring distances between crystal structures that can be used to properly quantify their geometric (Euclidean) closeness.

36 MATERIALS SCIENCE↗
DOE OSTI2026

ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation

A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Infrastructure Team at Lawrence Livermore National Laboratory (LLNL) to facilitate the use of long-term, high-frequency measurements from the ARM Facility in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The Coupled Model Intercomparison Project (CMIP) model data sets are also included in the package to enable model intercomparison as demonstrated in Zhang et al. (2017). The mean of the CMIP model can serve as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, and radiative fluxes, with plan to extend to more fields, such as aerosol and microphysics properties. Process-oriented diagnostics focusing on individual cloud- and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. The version 1.0 package is designed based on data collected at ARM’s Southern Great Plains (SGP) Research Facility, with the plan to extend to other ARM sites. The metrics and diagnostics package is currently built upon standard Python libraries and additional Python packages developed by DOE (such as CDMS and CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first present the input data, which constitutes the core content of the metrics and diagnostics package in section 2, and a user's guide documenting the workflow/structure of the version 1.0 codes, and including step-by-step instruction for running the package in section 3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Benchmarking performance changes in the simulation of extratropical modes of variability across CMIP generations

In this work, we evaluate extratropical modes of variability in the three most recent phases of the Coupled Model Intercomparison Project (CMIP3, 5, and 6) to gauge improvement of climate models over time. A suite of high-level metrics is employed to objectively evaluate how well climate models simulate the qobserved Northern Annular Mode (NAM), North Atlantic Oscillation (NAO), Pacific North America pattern (PNA), Southern Annular Mode (SAM), Pacific Decadal Oscillation (PDO), North Pacific Oscillation (NPO), and North Pacific Gyre Oscillation (NPGO). We apply a Common Basis Function (CBF) approach that projects model anomalies onto observed empirical orthogonal functions (EOFs), together with the traditional EOF approach, to CMIP Historical and AMIP models. We find simulated spatial patterns of those modes have been significantly improved in the newer models, although the skill improvement is sensitive to the mode and season considered. We identify some potential contributions to the pattern improvement of certain modes (e.g., the Southern Hemisphere jet and High-top vertical coordinate), however the performance changes are likely attributed to gradual improvement of the base climate and multiple relevant processes. Less performance improvement is evident in the mode amplitude of these modes and systematic overestimation of the mode amplitude in spring remains in the newer climate models. We find that the post-dominant season amplitude errors in atmospheric modes are not limited to coupled runs but are often already evident in AMIP simulations. This suggests that rectifying the egregious post-dominant season amplitude errors found in many models can be addressed in an atmospheric-only framework making it more tractable to address in the model development process.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Customs: Inspect and Report Python Imports (customs) v1.0.0

We have developed a simple, minimally invasive monitoring framework that enables us to capture key metrics from Python user processes. This is part of a larger effort to collect data across our entire data-intensive science workload at NERSC. The approach for Python is inspired by a similar framework developed for Blue Waters at NCSA. Both frameworks leverage standard Python features, sitecustomize and atexit, to capture Python imports of interest and other job data. The customs package decomposes the information capture into separate inspection and reporting interfaces.

Thomas, Rollin↗