Search NASA⌕ Search

SEARCH · Search NASA

Results for “science metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Characterizing gravitational wave detector networks: from A # to cosmic explorer

Gravitational-wave observations by the laser interferometer gravitational-wave observatory (LIGO) and Virgo have provided us a new tool to explore the Universe on all scales from nuclear physics to the cosmos and have the massive potential to further impact fundamental physics, astrophysics, and cosmology for decades to come. In this paper we have studied the science capabilities of a network of LIGO detectors when they reach their best possible sensitivity, called A # , given the infrastructure in which they exist and a new generation of observatories that are factor of 10 to 100 times more sensitive (depending on the frequency), in particular a pair of L-shaped cosmic explorer (CE) observatories (one 40 km and one 20 km arm length) in the US and the triangular Einstein telescope with 10 km arms in Europe. We use a set of science metrics derived from the top priorities of several funding agencies to characterize the science capabilities of different networks. The presence of one or two A # observatories in a network containing two or one next generation observatories, respectively, will provide good localization capabilities for facilitating multimessenger astronomy (MMA) and precision measurement of the Hubble parameter. Two CE observatories are indispensable for achieving precise localization of binary neutron star events, facilitating detection of electromagnetic counterparts and transforming MMA. Their combined operation is even more important in the detection and localization of high-redshift sources, such as binary neutron stars, beyond the star-formation peak, and primordial black hole mergers, which may occur roughly 100 million years after the Big Bang. The addition of the Einstein Telescope to a network of two CE observatories is critical for accomplishing all the identified science metrics including the nuclear equation of state, cosmological parameters, the growth of black holes through cosmic history, but also make new discoveries such as the presence of dark matter within or around neutron stars and black holes, continuous gravitational waves from rotating neutron stars, transient signals from supernovae, and the production of stellar-mass black holes in the early Universe. For most metrics the triple network of next generation terrestrial observatories are a factor 100 better than what can be accomplished by a network of three A # observatories.

Einstein telescope↗
DOE OSTI2020

Science Resource Office Peer Review and Metrics Factsheet [Slides]

The Science Resource Office-Peer Review and Metrics (SRO-PRM) Team hosts and maintains LANL's central repository for Science, Technology, and Engineering (STE) data. The Team tracks, integrates, analyzes, and reports qualitative and quantitative data measuring performance, quality, impact, and productivity. These metrics are indicators of the effectiveness and impact of STE efforts.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Evaluating energy justice metrics in early-stage science and technology research using the JUST-R metrics framework

Embedding principles of energy justice throughout all aspects of clean energy technology research and development (R&D) can facilitate a more just energy transition; yet gaps remain in our understanding of how to best integrate energy justice from the earliest R&D stages. The Justice Underpinning Science and Technology Research (JUST-R) metrics framework has been developed to enable early-stage energy researchers to assess and address justice considerations associated with their research, but the impacts of the framework, and others like it, have yet to be evaluated. This study seeks to evaluate the JUST-R metrics framework in terms of its effectiveness and appeal to researchers engaged in early-stage technical R&D using qualitative analyses of documents and workshop transcripts. We find that the metrics framework helps researchers identify problems and potential solutions surrounding the energy justice implications of their work and spurs a change in perspective for researchers, but, simultaneously, there is no evidence of solution follow-through within the evaluation timeframe. Greater institutional support, specialization to research areas, knowledge of energy justice fundamentals, and earlier incorporation of energy justice considerations in research projects arise as factors needed to aid continued use of the framework and pursuit of identified solutions. This evaluation protocol and these findings can serve as a guide for improving other frameworks with similar goals of encouraging sociotechnical engagement in early-stage energy R&D.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Metrics as tools for bridging climate science and applications

In climate science and applications, the term “metric” is used to describe the distillation of complex, multifaceted evaluations to summarize the overall quality of a model simulation, or other data product, and/or as a means to quantify some response to climate change. Metrics provide insights into the fidelity of processes and outcomes from climate models and can assist with both differentiating models' representation of variables or processes and informing whether models are “fit for purpose.” Metrics can also provide a valuable reference point for co-production of knowledge between climate scientists and climate impact practitioners. Although continued metric developments enable model developers to better understand the impacts of decisions made in the model design process, metrics also have implications for the characterization of uncertainty and facilitating analyses of underlying physical processes. As a result, comprehensive evaluation with multiple metrics enhances usability of climate information by both scientific and stakeholder communities. Here, this paper presents examples of insights gained from the development and appropriate use of metrics, and provides examples of how metrics can be used to engage with stakeholders and inform decision-making.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

Abstract In-situ irradiation transmission electron microscopy (TEM) offers unique insights into the millisecond-timescale post-cascade process, such as the lifetime and thermal stability of defect clusters, vital to the mechanistic understanding of irradiation damage in nuclear materials. Converting in-situ irradiation TEM video data into meaningful information on defect cluster dynamic properties (e.g., lifetime) has become the major technical bottleneck. Here, we present a solution called the DefectTrack , the first dedicated deep learning-based one-shot multi-object tracking (MOT) model capable of tracking cascade-induced defect clusters in in-situ TEM videos in real-time. DefectTrack has achieved a Multi-Object Tracking Accuracy (MOTA) of 66.43% and a Mostly Tracked (MT) of 67.81% on the test set, which are comparable to state-of-the-art MOT algorithms. We discuss the MOT framework, model selection, training, and evaluation strategies for in-situ TEM applications. Further, we compare the DefectTrack with four human experts in quantifying defect cluster lifetime distributions using statistical tests and discuss the relationship between the material science domain metrics and MOT metrics. Our statistical evaluations on the defect lifetime distribution suggest that the DefectTrack outperforms human experts in accuracy and speed.

42 ENGINEERING↗
DOE OSTI2021

The Making of a Metric: Co-Producing Decision-Relevant Climate Science

Developing decision-relevant science for adaptation requires the identification of climatic parameters that are both actionable for practitioners as well as tractable for modelers. In many sectors, these decision-relevant climatic metrics and the approaches that enable their identification remain largely unknown. “Co-production” of science with scientists and decision-makers is one potential way to identify these metrics, but there is little research describing specific and successful co-production approaches. This paper examines the negotiations and outcomes from Project Hyperion, wherein scientists and water managers jointly developed decision-relevant climatic metrics for adaptive water management. We identify successful co-production strategies by analyzing the project’s numerous back-and-forth engagements and tracing the evolution of the science during these engagements. We found that effective mediation between scientists and managers needed dedicated “boundary spanners” with significant modeling expertise. Translating practitioners’ information needs into tractable climatic metrics required direct and indirect methods of eliciting knowledge. We identified four indirect methods that were particularly salient for extracting tacitly held knowledge and enabling shared learning: developing a hierarchical framework linking management issues with metrics, starting discussions from the planning challenges, collaboratively exploring the planning relevance of new scientific capabilities, and using analogies of other “good” metrics. The decision-relevant metrics we developed provide insights into advancing adaptation-relevant climate science in the water sector. The co-production strategies we identified can be used to design and implement productive scientist–decision-maker interactions. Overall, the approaches and metrics we developed can help climate science to expand in new and more use-inspired directions.

Planning↗
DOE OSTI2024

A Cohesive Deep Drilling Field Strategy for LSST Cosmology

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will image billions of astronomical objects in the wide–fast–deep primary survey and in a set of minisurveys including intensive observations of a group of deep drilling fields (DDFs). The DDFs are a critical piece of three key aspects of the LSST Dark Energy Science Collaboration (DESC) cosmological measurements: they provide a required calibration for photometric redshifts (photo- z ) and weak gravitational lensing (WL) measurements and they directly contribute to cosmological constraints from the most distant Type Ia supernovae (SNe Ia). We present a set of cohesive DDF strategies fulfilling science requirements relevant to DESC and following the guidelines of the Survey Cadence Optimization Committee. We propose a method to estimate the observing strategy parameters and we perform simulations of the corresponding surveys. We define a set of metrics for each science case to assess the performance of the proposed observing strategies. We show that the most promising results are achieved with deep rolling surveys characterized by two sets of fields: ultradeep fields ( z ≲ 1.1) observed at a high cadence with a large number of visits over a limited number of seasons, and deep fields ( z ≲ 0.7), observed with a cadence of ~3 nights for 10 yr. These encouraging results should be confirmed with realistic simulations using the LSST scheduler. A DDF budget of ~8.5% is required to design observing strategies satisfying all the cosmological requirements. A lower DDF budget leads to surveys that either do not fulfill the photo- z /WL requirements or are not optimal for SN Ia cosmology.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

RTN-117: Image Calibration and Instrument Signal Removal for the First Year of the LSST

The NSF-DOE Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) requires calibration products that provide uniform, stable, and accurate photometric and astrometric performance across the 3.2 gigapixel focal plane and throughout the 10-year survey. This paper details the algorithms and workflows used to produce instrument calibrations and remove instrumental artifacts for Data Preview 2 (DP2)---the first end-to-end processing demonstration using on-sky data with the LSST Camera (LSSTCam). We describe the verification, acceptance, and certification framework used to assess calibration quality and quantify residual systematics. We show baseline metrics on calibrated science images to evaluate the robustness of the calibration and instrument signature removal (ISR) data processing pipelines for DP2. Finally, we summarize the known limitations observed in DP2 production and outline expected algorithmic improvements for the first public LSST data release (Data Release~1, DR1).

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Developing and Evaluating Energy Justice Metrics for Early-Stage Materials Research

Materials science is a central component of early-stage research and development of virtually all clean energy technologies. But as much as material breakthroughs often hold the key to high efficiencies, long lifetimes, and high stability in eventual devices, early-stage choices about material types, structures, and processing can also serve to lock in long-term social and equity impacts of deployed energy technologies. Thus, to achieve a just and sustainable energy transition, tools to assess the energy justice impacts of early-stage materials research are critical. Here, we discuss development of the Justice Underpinning Science and Technology Research (JUST-R) metrics framework - a suite of metrics targeted at early-stage researchers to assess energy justice considerations in their work. The framework is evaluated for its appeal to researchers and effectiveness at promoting integration of energy justice into research through case studies, which reveal its ability to broaden researcher perspectives and key avenues for future improvement.

energy justice↗
DOE OSTI2024

Characterization of Non-Science Grade DESI CCDs and Creating Additional ICARUS Monitoring Metrics for Data Quality

DESI, otherwise known as the Dark Energy Spectroscopic Instrument, is an astronomical project measuring millions of optical spectra to characterize Dark Energy. The survey has been running for 3 years. My work delves into the data taking/analysis side of the DESI CCDs (Charge-Coupled Devices) which will help in swapping out broken CCDs on the instrument. Additionally, I worked on making new metrics for ICARUS, Imaging Cosmic and Rare Underground Signals for their monitoring of run data. This paper will talk about the types of characterizations done for DESI and additional metrics for data quality and tools used for ICARUS monitoring.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Characterization of Non-Science Grade DESI CCDs and Creating Additional ICARUS Monitoring Metrics

DESI, otherwise known as the Dark Energy Spectroscopic Instrument, is a tool to measure optical spectra from the effects of dark energy on the universe and has been running for 3 years. My work delves into the data taking/analysis side of the DESI CCDs which will help in swapping out broken CCDs on the instrument. Additionally, I worked on making new metrics for ICARUS, Imaging Cosmic and Rare Underground Signals for their monitoring of run data. This paper will talk about the types of characterizations done for DESI and some metrics for data quality and tools used for ICARUS monitoring.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

DOE FAIR Surrogate Benchmarks Supporting AI and Simulation Research (SBI Surrogate Benchmark Initiative) (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia(UVA). SBI repositories include data, code, and all relevant collateral artifacts, that the science and engineering community needs to use and reuse these data sets and surrogates. SBI repositories generate active research from both participants in SBI and the broader AI and domain science communities. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and capture them as surrogate benchmarks with a rich set of metadata, covering. Data; Model; Metrics specification; Machine specification; Science, Speed, Power Results, We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non Surrogate benchmarks that have many common features and similar issues regarding FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, benchmarks have datasets, models, and metadata, and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates, including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Design, construction, and test of the Gas Pixel Detectors for the IXPE mission

Due to be launched in late 2021, the Imaging X-ray Polarimetry Explorer (IXPE) is a NASA Small Explorer mission designed to perform polarization measurements in the 2–8 keV band, complemented with imaging, spectroscopy and timing capabilities. At the heart of the focal plane is a set of three polarization-sensitive Gas Pixel Detectors (GPD), each based on a custom ASIC acting as a charge-collecting anode. In this paper we shall review the design, manufacturing, and test of the IXPE focal-plane detectors, with particular emphasis on the connection between the science drivers, the performance metrics and the operational aspects. We shall present a thorough characterization of the GPDs in terms of effective noise, trigger efficiency, dead time, uniformity of response, and spectral and polarimetric performance. In addition, we shall discuss in detail a number of instrumental effects that are relevant for high-level science analysis—particularly as far as the response to unpolarized radiation and the stability in time are concerned.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

PSTN-051: Survey Strategy and Cadence Choices for the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST)

A summary of survey strategy and cadence choices, simulated and evaluated by the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) Scheduler Team, prepared for the Survey Cadence and Optimization Committee (SCOC). A large telescope survey, covering the entire visible sky repeatedly every few days in multiple bandpasses over the course of ten years, is the core idea of the LSST. An area of about 20,000 square degrees observed under a wide range of conditions to deep coadded limiting magnitudes in bandpasses ugrizy enables cosmological studies and studies of the Milky Way structure with unprecedented precision; the same survey, when cadenced well, can serve to open new windows into our understanding of transient and variable stars, and extend our knowledge of small bodies throughout the Solar System by orders of magnitude. The outlines of, and some basic necessary requirements for these goals are outlined in the LSST Science Requirements Document (SRD). Finding options for the observing strategy to meet more detailed needs of an even wider range of science goals, as well as building the LSST Scheduler and Metrics Analysis Framework, has been the work of the LSST Scheduler Team with support and input from the astronomical community, including the Community Observing Strategy Evaluation Paper (COSEP), the Call for White Papers, numerous performance metrics, and guidance from the LSST Science Advisory Committee in their Recommendations for Operations Simulator Experiments. This report will help enable the SCOC to make its recommendation for the preferred survey strategy.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Extreme metrics from large ensembles: investigating the effects of ensemble size on their estimates

Abstract. We consider the problem of estimating the ensemble sizes required to characterize the forced component and the internal variability of a number of extreme metrics. While we exploit existing large ensembles, our perspective is that of a modeling center wanting to estimate a priori such sizes on the basis of an existing small ensemble (we assume the availability of only five members here). We therefore ask if such a small-size ensemble is sufficient to estimate accurately the population variance (i.e., the ensemble internal variability) and then apply a well-established formula that quantifies the expected error in the estimation of the population mean (i.e., the forced component) as a function of the sample size n, here taken to mean the ensemble size. We find that indeed we can anticipate errors in the estimation of the forced component for temperature and precipitation extremes as a function of n by plugging into the formula an estimate of the population variance derived on the basis of five members. For a range of spatial and temporal scales, forcing levels (we use simulations under Representative Concentration Pathway 8.5) and two models considered here as our proof of concept, it appears that an ensemble size of 20 or 25 members can provide estimates of the forced component for the extreme metrics considered that remain within small absolute and percentage errors. Additional members beyond 20 or 25 add only marginal precision to the estimate, and this remains true when statistical inference through extreme value analysis is used. We then ask about the ensemble size required to estimate the ensemble variance (a measure of internal variability) along the length of the simulation and – importantly – about the ensemble size required to detect significant changes in such variance along the simulation with increased external forcings. Using the F test, we find that estimates on the basis of only 5 or 10 ensemble members accurately represent the full ensemble variance even when the analysis is conducted at the grid-point scale. The detection of changes in the variance when comparing different times along the simulation, especially for the precipitation-based metrics, requires larger sizes but not larger than 15 or 20 members. While we recognize that there will always exist applications and metric definitions requiring larger statistical power and therefore ensemble sizes, our results suggest that for a wide range of analysis targets and scales an effective estimate of both forced component and internal variability can be achieved with sizes below 30 members. This invites consideration of the possibility of exploring additional sources of uncertainty, such as physics parameter settings, when designing ensemble simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Single Bimodular Sensor for Differentiated Detection of Multiple Oxidative Gases

Semiconductive metal-oxide sensors suffer from cross-sensitivities under mixed chemical condition, specifically upon mixture of multiple oxidative or reductive gases. Herein, a single bimodular sensor is demonstrated for smart differentiation of multiple oxidative analytes by relating the resistance-metric mode to impedance-metric mode. The sensor construct based on ZnO nanorods readily outputs three response datasets upon exposure of oxidative-gas mixture including O 2 , SO 2 , and NO 2 , the resistance, real part impedance, and imaginary part impedance. The differentiative and correlated nature between these response signals allows such a single sensor platform to differentiate these oxidative gases accurately and robustly. Linear and non-linear decision boundaries are established over a large gas-concentration range from 2 ppm to 3% through a combination of principal component analysis and artificial neural network training. A facile user interface is demonstrated for recognition and measurement of unknown gas analytes, with the error of the predicted analyte-concentration as low as 2%.

36 MATERIALS SCIENCE↗
DOE OSTI2025

FAIR Surrogate Benchmarks Supporting AI and Simulation Research (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia (UVA). SBI repositories include data, code, and all relevant collateral artifacts that the science and engineering community need to use and reuse these data sets and surrogates. SBI repositories generate active research from both the participants in SBI and the broad community of AI and domain scientists. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and captures them as surrogate benchmarks with a rich set of metadata covering: Data; Model; Metrics specification; Machine specification; and Science, Speed, and Power Results. We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non-Surrogate benchmarks that have many common features and similar issues as regards FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, Benchmarks have datasets, models, and metadata and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗