Search NASA⌕ Search

SEARCH · Search NASA

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

NASA EOSDIS Data Usage Metrics- Insight and Assessment

NASA's Earth Science Data and Information System (ESDIS) Project collects Earth science data usage metrics on a daily basis through the ESDIS Metrics System (EMS). This includes metrics on distribution of data products, users, data volumes, and number of files, which are key parameters in evaluating system-level performance of any of the Distributed Active Archive Centers (DAACs) encompassed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS). EOSDIS data usage metrics illustrate the benefits of making NASA data openly available to the public and show a rapid growth in data distribution to a worldwide user community. In fact, each year since 2014 the EOSDIS has distributed over one billion data files of products from EOS satellite, airborne, and in situ observations. An assessment of the long-term trends of data usage metrics and user characterization provides insights into data usability.This study will focus on describing the EMS as a metrics collection tool and will provide a comprehensive analysis of EOSDIS data usage metrics over the last 10 years. This study will also characterize the product distribution metrics by various tools and services, such as Giovanni, the Open-source Project for a Network Data Access Protocol (OPeNDAP), and subsets, to address how these tools/services have extended the usage of data in the EOSDIS collection. Data usage patterns based on discipline and study area will further assist in understanding how EOSDIS data user needs have evolved over time. Results from this study will provide useful information for the DAACs that can help them improve the functionality of their tools and services as well as more efficiently allocate the resources necessary for enhanced access and availability of their data products. Knowledge of these metrics may also benefit user discovery of data in the EOSDIS collection, promote research collaboration, and stimulate new ideas from work and research conducted using specific datasets and data collections.

Kafle, Durga N.↗

Integrated Analysis of Multiple User Metrics - A “Sequel”; and Introducing the Google Analytic

For decades, the Goddard Earth Sciences Data and Information Services Center (GES DISC) has archived and distributed enormous volumes of NASA Earth science data (accompanied with many developed tools and services) to various research/applications communities and the general public. Being “immersed” in the Big Data era, we have inevitably faced the challenges of our continually increasing archived data in both volume and variety, as well as enhanced user needs and demands. In recent years, we have actively analyzed different types of user metrics, such as operational distribution metrics (recording numbers of distinct users and downloaded data files, size of distributed data volume): user publication metrics (mining info from our Giovanni users’ publications): and Bugzilla metrics (collecting info from user questions or feedback from user assistance tickets). Such metrics have helped us achieve a better understanding of user needs, demands, characteristics, and behaviors, which has then helped us improve our user services. Now we will present a “Sequel” of integrated analysis of multiple metrics at the GES DISC by introducing and adding one new kind of metrics acquired via utilizing our recently implemented Google Analytic 360 suite. Several “newer” reports, e.g., “What web site features and links are the most popular (and least)?” and “What are the top 25 dataset Keyword searches?” retrieved from this new metrics set will be presented, along with the aforementioned “traditional” metrics results.

Shie, Chung-Lin↗

A New CE TSUNAMI-3D Capability for Calculating Undersampling Metrics and Biases

This study investigates the viability of several statistical metrics for predicting the undersampling biases in continuous-energy (CE) Monte Carlo transport simulation tally estimates. Undersampling is a phenomenon in which a Monte Carlo simulation fails to use enough particles per generation to adequately sample fission points in all regions of a problem, resulting in biases in tally estimates that are much larger than the statistical uncertainties, potentially leading to erroneous conclusions regarding system performance and safety. Previous work found that biases due to undersampling could be as large as several hundred per cent mille (pcm) for eigenvalue estimates and up to tens or hundreds of percent for flux tally estimates. This paper documents the encapsulation of undersampling metrics within CE TSUNAMI-3D in the SCALE code system and examines the correlation between the various undersampling metric scores and the magnitude of various undersampling biases. Of the metrics examined, the RHW metric, the Tally Entropy metric, and the uncertainty in the scores for these metrics produce scores that correlated well with the fraction of the undersampling biases in various eigenvalue, flux, and energy-integrated reaction rate tallies in an infinitely reflected model of a PWR fuel assembly. The average number of nonzero scores per generation encountered by a tally was also found to show promise at predicting the magnitude of undersampling biases, and it also found use in detecting and filtering extremely undersampled tallies in the application of the other metrics. Future work will strengthen these conclusions by improving the degree of convergence for the examined tally and undersampling bias estimates, apply these metrics to a more broad variety of systems, and investigate variance reduction approaches for mitigating undersampling in Monte Carlo tally estimates. (authors)

42 ENGINEERING↗

A biology-informed similarity metric for simulated patches of human cell membrane

Complex scientific inquiries rely increasingly upon large and autonomous multiscale simulation campaigns, which fundamentally require similarity metrics to quantify ‘sufficient’ changes among data and/or configurations. However, subject matter experts are often unable to articulate similarity precisely or in terms of well-formulated definitions, especially when new hypotheses are to be explored, making it challenging to design a meaningful metric. Furthermore, the key to practical usefulness of such metrics to enable autonomous simulations lies in in situ inference, which requires generalization to possibly substantial distributional shifts in unseen, future data. Here, we address these challenges in a cancer biology application and develop a meaningful similarity metric for ‘patches’—regions of simulated human cell membrane that express interactions between certain proteins of interest and relevant lipids. In the absence of well-defined conditions for similarity, we leverage several biology-informed notions about data and the underlying simulations to impose inductive biases on our metric learning framework, resulting in a suitable similarity metric that also generalizes well to significant distributional shifts encountered during the deployment. We combine these intuitions to organize the learned embedding space in a multiscale manner, which makes the metric robust to incomplete and even contradictory intuitions. Our approach delivers a metric that not only performs well on the conditions used for its development and other relevant criteria, but also learns key spatiotemporal relationships without ever being exposed to any such information during training.

97 MATHEMATICS AND COMPUTING↗

Metrics as tools for bridging climate science and applications

In climate science and applications, the term “metric” is used to describe the distillation of complex, multifaceted evaluations to summarize the overall quality of a model simulation, or other data product, and/or as a means to quantify some response to climate change. Metrics provide insights into the fidelity of processes and outcomes from climate models and can assist with both differentiating models' representation of variables or processes and informing whether models are “fit for purpose.” Metrics can also provide a valuable reference point for co-production of knowledge between climate scientists and climate impact practitioners. Although continued metric developments enable model developers to better understand the impacts of decisions made in the model design process, metrics also have implications for the characterization of uncertainty and facilitating analyses of underlying physical processes. As a result, comprehensive evaluation with multiple metrics enhances usability of climate information by both scientific and stakeholder communities. Here, this paper presents examples of insights gained from the development and appropriate use of metrics, and provides examples of how metrics can be used to engage with stakeholders and inform decision-making.

54 ENVIRONMENTAL SCIENCES↗

A Metric Space with LCOE Isolines for Research Guidance in wind and hydrokinetic energy systems

This paper introduces a new Metric Space to guide the design of advanced wind energy systems and hydrokinetic energy converters such as tidal, ocean current and riverine turbines. The Metric Space can analyse farms that combine different or identical turbines and stand-alone turbines. The first metric (M1) of the space considers the efficiency of the turbines in the farm, which is also proportional to the specific power per swept area at a given wind/water velocity (W/m 2 ). The second metric (M2) describes the specific rotor area per unit of mass of the turbines (m 2 /kg). Both metrics depend on the primary design characteristics of the turbines, such as swept area, system size and mass, materials and efficiency, and are independent at first from external characteristics, such as atmospheric and ocean site conditions, cost of materials and economic factors. Combining both metrics, and for a given set of external characteristics, the resulting Metric Space M2/M1 displays the Levelized Cost of Energy (LCOE) standards as isolines. This graphical representation provides a quick understanding of the cost and state of the technology. It also offers a practical guidance to choose the research tasks and strategy to design advanced wind and hydrokinetic energy systems. The paper applies the new Metric Space to several case studies, including large and small onshore wind turbines, floating and bottom-fixed offshore wind turbines, downwind rotors, multi-rotor and hybrid systems, airborne wind energy systems, wind farms and tidal energy converters.

17 WIND ENERGY↗

Measuring Fundamental Improvements in Sustainable Urban Mobility: The Mobility-Energy Productivity Metric

Recent technological advancements in mobility are creating many options for connecting citizens with employment, goods, and services, particularly in urban areas where modes such as bike and car shares, electric scooters, ridesourcing, and ridesharing are proliferating at a rapid pace. Analysis and tools for overall transportation planning are dominated by urban regional travel demand models whose roots in highway operations poorly reflect the system dynamics in denser areas where parking costs, convenience, and availability - not to mention sustainability concerns and quality of life - are driving people to an ever-greater spectrum of mobility services. In this paper, we present a new paradigm for evaluating mobility options within an urban area. First developed for the U.S. Department of Energy's Energy Efficient Mobility System research program, this metric is termed the mobility-energy productivity (MEP) metric. At its heart, the MEP metric measures accessibility and appropriately weights it with travel time, cost, and energy of modes that provide access to opportunities in any given location. The proposed metric is versatile in that it can be computed from readily available data sources or derived from outputs of regional travel demand models. End times associated with parking, curb access, cost, and reliability and frequency of service need to be carefully considered to obtain an appropriate and accurate perspective when computing the metric. Ultimately, the MEP metric can be used to reflect the impacts of new mobility technologies (transportation network companies, electric scooters), business models (car shares and bike shares), and land-use practices (such as transit-oriented development) on sustainable urban mobility. This paper lays out the need, requirements, and framework for this new metric, and offers it, in collaboration with the American Society for Civil Engineers (ASCE), as a foundational metric for smart city assessment.

30 DIRECT ENERGY CONVERSION↗

SU-E-I-71: Quality Assessment of Surrogate Metrics in Multi-Atlas-Based Image Segmentation

Purpose: With the ever-growing data of heterogeneous quality, relevance assessment of atlases becomes increasingly critical for multi-atlas-based image segmentation. However, there is no universally recognized best relevance metric and even a standard to compare amongst candidates remains elusive. This study, for the first time, designs a quantification to assess relevance metrics’ quality, based on a novel perspective of the metric as surrogate for inferring the inaccessible oracle geometric agreement. Methods: We first develop an inference model to relate surrogate metrics in image space to the underlying oracle relevance metric in segmentation label space, with a monotonically non-decreasing function subject to random perturbations. Subsequently, we investigate model parameters to reveal key contributing factors to surrogates’ ability in prognosticating the oracle relevance value, for the specific task of atlas selection. Finally, we design an effective contract-to-noise ratio (eCNR) to quantify surrogates’ quality based on insights from these analyses and empirical observations. Results: The inference model was specialized to a linear function with normally distributed perturbations, with surrogate metric exemplified by several widely-used image similarity metrics, i.e., MSD/NCC/(N)MI. Surrogates’ behaviors in selecting the most relevant atlases were assessed under varying eCNR, showing that surrogates with high eCNR dominated those with low eCNR in retaining the most relevant atlases. In an end-to-end validation, NCC/(N)MI with eCNR of 0.12 compared to MSD with eCNR of 0.10 resulted in statistically better segmentation with mean DSC of about 0.85 and the first and third quartiles of (0.83, 0.89), compared to MSD with mean DSC of 0.84 and the first and third quartiles of (0.81, 0.89). Conclusion: The designed eCNR is capable of characterizing surrogate metrics’ quality in prognosticating the oracle relevance value. It has been demonstrated to be correlated with the performance of relevant atlas selection and ultimate label fusion.

60 APPLIED LIFE SCIENCES↗

SU-F-T-312: Identifying Distinct Radiation Therapy Plan Classes Through Multi-Dimensional Analysis of Plan Complexity Metrics

Purpose: To determine whether body site-specific treatment plans form unique “plan class” clusters in a multi-dimensional analysis of plan complexity metrics such that a single beam quality correction determined for a representative plan could be universally applied within the “plan class”, thereby increasing the dosimetric accuracy of a detector’s response within a subset of similarly modulated nonstandard deliveries. Methods: We collected 95 clinical volumetric modulated arc therapy (VMAT) plans from four body sites (brain, lung, prostate, and spine). The lung data was further subdivided into SBRT and non-SBRT data for a total of five plan classes. For each control point in each plan, a variety of aperture-based complexity metrics were calculated and stored as unique characteristics of each patient plan. A multiple comparison of means analysis was performed such that every plan class was compared to every other plan class for every complexity metric in order to determine which groups could be considered different from one another. Statistical significance was assessed after correcting for multiple hypothesis testing. Results: Six out of a possible 10 pairwise plan class comparisons were uniquely distinguished based on at least nine out of 14 of the proposed metrics (Brain/Lung, Brain/SBRT lung, Lung/Prostate, Lung/SBRT Lung, Lung/Spine, Prostate/SBRT Lung). Eight out of 14 of the complexity metrics could distinguish at least six out of the possible 10 pairwise plan class comparisons. Conclusion: Aperture-based complexity metrics could prove to be useful tools to quantitatively describe a distinct class of treatment plans. Certain plan-averaged complexity metrics could be considered unique characteristics of a particular plan. A new approach to generating plan-class specific reference (pcsr) fields could be established through a targeted preservation of select complexity metrics or a clustering algorithm that identifies plans exhibiting similar modulation characteristics. Measurements and simulations will better elucidate potential plan-class specific dosimetry correction factors.

60 APPLIED LIFE SCIENCES↗

Description of the Sandia National Laboratories science, technology & engineering metrics process.

There has been a concerted effort since 2007 to establish a dashboard of metrics for the Science, Technology, and Engineering (ST&E) work at Sandia National Laboratories. These metrics are to provide a self assessment mechanism for the ST&E Strategic Management Unit (SMU) to complement external expert review and advice and various internal self assessment processes. The data and analysis will help ST&E Managers plan, implement, and track strategies and work in order to support the critical success factors of nurturing core science and enabling laboratory missions. The purpose of this SAND report is to provide a guide for those who want to understand the ST&E SMU metrics process. This report provides an overview of why the ST&E SMU wants a dashboard of metrics, some background on metrics for ST&E programs from existing literature and past Sandia metrics efforts, a summary of work completed to date, specifics on the portfolio of metrics that have been chosen and the implementation process that has been followed, and plans for the coming year to improve the ST&E SMU metrics process.

99 GENERAL AND MISCELLANEOUS↗

Estimating Sonic Boom Metrics Across a Community Using a Kalman Filter

As part of the Quesst mission, NASA will fly the X‑59 aircraft over selected communities to evaluate community responses to low-intensity sonic booms. The purpose of these community tests is to determine the dose-response relationship between the noise exposure metrics and the community response. The independent variables for the dose-response relationship are the noise exposure metrics experienced by survey respondents within the community. Two sources of noise exposure metrics are available in each community: measurements at sparse locations throughout the community, and calculations from propagation models across the community. Both the measurements and calculations are subject to uncertainty. A Kalman filter is proposed to combine the measured and calculated noise exposure metrics to obtain the best estimate of the true noise exposure metrics across the community. The noise exposure metrics estimated by the Kalman filter have lower uncertainty than either the measured or calculated noise exposure metrics alone. Simulations demonstrate that the Kalman filter produces a more accurate estimate of the true noise exposure metrics than other noise estimation methods.

Sonic boom↗

Measuring Fundamental Improvements in Sustainable Urban Mobility: The Mobility-Energy Productivity Metric: Preprint

Recent technological advancements in mobility are creating many options for connecting citizens with employment, goods, and services, particularly in urban areas where modes such as bike and car shares, electric scooters, ridesourcing, and ridesharing are proliferating at a rapid pace. Analysis and tools for overall transportation planning are dominated by urban regional travel demand models whose roots in highway operations poorly reflect the system dynamics in denser areas where parking costs, convenience, and availability - not to mention sustainability concerns and quality of life - are driving people to an ever-greater spectrum of mobility services. In this paper, we present a new paradigm for evaluating mobility options within an urban area. First developed for the U.S. Department of Energy's Energy Efficient Mobility System research program, this metric is termed the Mobility-Energy Productivity (MEP) metric. At its heart, the MEP metric measures accessibility and appropriately weights it with travel time, cost, and energy of modes that provide access to opportunities in any given location. The proposed metric is versatile in that it can be computed from readily available data sources or derived from outputs of regional travel demand models. End times associated with parking, curb access, cost, and reliability and frequency of service need to be carefully considered to obtain an appropriate and accurate perspective when computing the metric. Ultimately, the MEP metric can be used to reflect the impacts of new mobility technologies (transportation network companies, electric scooters), business models (car shares and bike shares), and land-use practices (such as transit-oriented development) on sustainable urban mobility. This paper lays out the need, requirements, and framework for this new metric, and offers it, in collaboration with the American Society for Civil Engineers (ASCE), as a foundational metric for Smart City assessment.

30 DIRECT ENERGY CONVERSION↗

A spectral metric for collider geometry

By quantifying the distance between two collider events, one can triangulate a metric space and reframe collider data analysis as computational geometry. One popular geometric approach is to first represent events as an energy flow on an idealized celestial sphere and then define the metric in terms of optimal transport in two dimensions. In this paper, we advocate for representing events in terms of a spectral function that encodes pairwise particle angles and products of particle energies, which enables a metric distance defined in terms of one-dimensional optimal transport. This approach has the advantage of automatically incorporating obvious isometries of the data, like rotations about the colliding beam axis. It also facilitates first-principles calculations, since there are simple closed-form expressions for optimal transport in one dimension. Up to isometries and event sets of measure zero, the spectral representation is unique, so the metric on the space of spectral functions is a metric on the space of events. At lowest order in perturbation theory in electron-positron collisions, our metric is simply the summed squared invariant masses of the two event hemispheres. Going to higher orders, we present predictions for the distribution of metric distances between jets in fixed-order and resummed perturbation theory as well as in parton-shower generators. Finally, we speculate on whether the spectral approach could furnish a useful metric on the space of quantum field theories.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Are light curve classification metrics good proxies for SN Ia cosmological constraining power?

Context. When selecting a light curve classifier for use as part of a photometric supernova Ia (SN Ia) cosmological analysis, it is common to make decisions based on metrics of classification performance, such as the contamination within the photometrically classified SN Ia sample, rather than a measure of cosmological constraining power. If the former is an appropriate proxy for the latter, this practice would eliminate the computational expense of a full cosmology forecast in the analysis pipeline design process. Aims. This study tests the assumption that light curve classification metrics are an appropriate proxy for cosmology metrics. Methods. We emulated photometric SN Ia cosmology light curve samples with controlled contamination rates of individual contaminant classes and evaluated each of them under a set of classification metrics. We then derived cosmological parameter constraints from all samples under two common analysis approaches and quantified the impact of contamination by each contaminant class on the resulting cosmological parameter estimates. Results. We observe that cosmology metrics are sensitive to both the contamination rate and the class of the contaminating population, whereas the classification metrics are shown to be insensitive to the latter. Conclusions. Based on these findings, we discourage any exclusive reliance on light curve classification-based metrics for analysis design decisions, which (counterintuitively) include but are not limited to the classifier choice. Instead, we recommend optimising science analysis pipeline design choices using a metric of the information gained about the physical parameters of interest.

79 ASTRONOMY AND ASTROPHYSICS↗

Progress toward a self-consistent set of 1D ignition capsule metrics in ICF

One-dimensional metrics can be considered an upper bound on the performance that can be achieved given an implosion with an imploding mass Mimp and a given adiabat α, driven to an implosion velocity V by a specified ablation pressure Pa for single shell capsules with low-opacity ablators. The quantitative value of an ignition metric depends on the definition of ignition. We review the choices that have been made by various authors before settling on a definition based on yield amplification of 30 where yield amplification is the ratio of the yield from an implosion that includes the alpha particle and neutron deposition relative to the yield obtained from PdV work alone. We then derive improved 1D ignition metrics for radiation-driven inertial confinement fusion targets that span a wide range of drive pressures, adiabats, and scales. This includes emphasizing the importance of the total imploding mass and kinetic energy inside the ablation front, including the remaining ablator mass and kinetic energy, on the 1D ignition metrics. We have also explicitly included the sensitivity to ablation pressure driving the implosion, as well as the sensitivity to the coupling efficiency between the total incoming kinetic energy and the hot spot. For implosions in which the remaining ablator mass contributes to the total stagnated mass, we have developed an approach based on defining an effective implosion adiabat which incorporates the properties and the mass of the stagnated ablator as well as the cold DT fuel. We have added a dependence on total ρr to account for the fact that the time available for self-heating increases as the total ρr increases. This extends previous work in that the value of the product of stagnation pressure times burn width required for ignition depends on the total ρr as well as on the fuel ion temperature. Additionally, we show that ignition metrics derived from a requirement on the product of the hotspot ρr and the hot spot temperature are equivalent to ignition metrics derived from a requirement on the product of stagnation pressure and burn width. Further, we discuss those ignition metrics, developed as metrics for the underlying hydrodynamics in the absence of alpha heating, which can be used when alpha heating is present.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

The Making of a Metric: Co-Producing Decision-Relevant Climate Science

Developing decision-relevant science for adaptation requires the identification of climatic parameters that are both actionable for practitioners as well as tractable for modelers. In many sectors, these decision-relevant climatic metrics and the approaches that enable their identification remain largely unknown. “Co-production” of science with scientists and decision-makers is one potential way to identify these metrics, but there is little research describing specific and successful co-production approaches. This paper examines the negotiations and outcomes from Project Hyperion, wherein scientists and water managers jointly developed decision-relevant climatic metrics for adaptive water management. We identify successful co-production strategies by analyzing the project’s numerous back-and-forth engagements and tracing the evolution of the science during these engagements. We found that effective mediation between scientists and managers needed dedicated “boundary spanners” with significant modeling expertise. Translating practitioners’ information needs into tractable climatic metrics required direct and indirect methods of eliciting knowledge. We identified four indirect methods that were particularly salient for extracting tacitly held knowledge and enabling shared learning: developing a hierarchical framework linking management issues with metrics, starting discussions from the planning challenges, collaboratively exploring the planning relevance of new scientific capabilities, and using analogies of other “good” metrics. The decision-relevant metrics we developed provide insights into advancing adaptation-relevant climate science in the water sector. The co-production strategies we identified can be used to design and implement productive scientist–decision-maker interactions. Overall, the approaches and metrics we developed can help climate science to expand in new and more use-inspired directions.

Planning↗

Advanced Life Support System Value Metric

The NASA Advanced Life Support (ALS) Program is required to provide a performance metric to measure its progress in system development. Extensive discussions within the ALS program have led to the following approach. The Equivalent System Mass (ESM) metric has been traditionally used and provides a good summary of the weight, size, and power cost factors of space life support equipment. But ESM assumes that all the systems being traded off exactly meet a fixed performance requirement, so that the value and benefit (readiness, performance, safety, etc.) of all the different systems designs are considered to be exactly equal. This is too simplistic. Actual system design concepts are selected using many cost and benefit factors and the system specification is defined after many trade-offs. The ALS program needs a multi-parameter metric including both the ESM and a System Value Metric (SVM). The SVM would include safety, maintainability, reliability, performance, use of cross cutting technology, and commercialization potential. Another major factor in system selection is technology readiness level (TRL), a familiar metric in ALS. The overall ALS system metric that is suggested is a benefit/cost ratio, SVM/[ESM + function (TRL)], with appropriate weighting and scaling. The total value is given by SVM. Cost is represented by higher ESM and lower TRL. The paper provides a detailed description and example application of a suggested System Value Metric and an overall ALS system metric.

Jones, Harry W.↗

A Complexity Metric for Automated Separation

A metric is proposed to characterize airspace complexity with respect to an automated separation assurance function. The Maneuver Option metric is a function of the number of conflict-free trajectory change options the automated separation assurance function is able to identify for each aircraft in the airspace at a given time. By aggregating the metric for all aircraft in a region of airspace, a measure of the instantaneous complexity of the airspace is produced. A six-hour simulation of Fort Worth Center air traffic was conducted to assess the metric. Results showed aircraft were twice as likely to be constrained in the vertical dimension than the horizontal one. By application of this metric, situations found to be most complex were those where level overflights and descending arrivals passed through or merged into an arrival stream. The metric identified high complexity regions that correlate well with current air traffic control operations. The Maneuver Option metric did not correlate with traffic count alone, a result consistent with complexity metrics for human-controlled airspace.

Aweiss, Arwa↗