SEARCH · Search NASA

Results for “science metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2019

Collecting and Processing Earth Science Data Metrics at NASA ESDIS

Since the launch of Terra satellite in 1999, the number of Earth Science remote sensing data products created and distributed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS) has increased from a few hundred to nearly ten thousand. NASA's Earth Science Data and Information System (ESDIS) Metrics System (EMS) collects metrics on data ingest, archive, and distribution by its Distributed Active Archive Centers (DAACs) and the Science Investigator-led Systems (SIPS), known as Data Providers. These metrics are critical in helping NASA management as well as data producers in resource planning and gaining a wide range of knowledge of data users and data usage.EMS receives flat files, or log files of data archive, ingest, and distribution either in their raw format, such as Apache web logs, or text files of log records formatted by the Data Providers. Tens of millions of records are processed each day to extract metrics on data products, user information, distribution protocols and services, and so on. The metrics are then made available to designated parties.This presentation provides an overview of the EMS processing workflow and improvement efforts made in recent years to handle ever-increasing number of data records and new metrics requirements, discusses several key steps including mapping log records to data products and identifying user communities along with geo-distribution, and demonstrates typical metrics capabilities produced by the EMS system. Challenges and potential approaches to improve the system are also discussed.

Pan, Jianfu↗

DOE OSTI2024

Characterizing gravitational wave detector networks: from A # to cosmic explorer

Gravitational-wave observations by the laser interferometer gravitational-wave observatory (LIGO) and Virgo have provided us a new tool to explore the Universe on all scales from nuclear physics to the cosmos and have the massive potential to further impact fundamental physics, astrophysics, and cosmology for decades to come. In this paper we have studied the science capabilities of a network of LIGO detectors when they reach their best possible sensitivity, called A # , given the infrastructure in which they exist and a new generation of observatories that are factor of 10 to 100 times more sensitive (depending on the frequency), in particular a pair of L-shaped cosmic explorer (CE) observatories (one 40 km and one 20 km arm length) in the US and the triangular Einstein telescope with 10 km arms in Europe. We use a set of science metrics derived from the top priorities of several funding agencies to characterize the science capabilities of different networks. The presence of one or two A # observatories in a network containing two or one next generation observatories, respectively, will provide good localization capabilities for facilitating multimessenger astronomy (MMA) and precision measurement of the Hubble parameter. Two CE observatories are indispensable for achieving precise localization of binary neutron star events, facilitating detection of electromagnetic counterparts and transforming MMA. Their combined operation is even more important in the detection and localization of high-redshift sources, such as binary neutron stars, beyond the star-formation peak, and primordial black hole mergers, which may occur roughly 100 million years after the Big Bang. The addition of the Einstein Telescope to a network of two CE observatories is critical for accomplishing all the identified science metrics including the nuclear equation of state, cosmological parameters, the growth of black holes through cosmic history, but also make new discoveries such as the presence of dark matter within or around neutron stars and black holes, continuous gravitational waves from rotating neutron stars, transient signals from supernovae, and the production of stellar-mass black holes in the early Universe. For most metrics the triple network of next generation terrestrial observatories are a factor 100 better than what can be accomplished by a network of three A # observatories.

Einstein telescope↗

NASA NTRS1983

Metric analysis and data validation across FORTRAN projects

The desire to predict the effort in developing or explaining the quality of software has led to the proposal of several metrics. As a step toward validating these metrics, the Software Engineering Laboratory (SEL) has analyzed the software science metrics, cyclomatic complexity, and various standard program measures for their relation to effort (including design through acceptance testing), development errors (both discrete and weighted according to the amount of time to locate and fix), and one another. The data investigated are collected from a project FORTRAN environment and examined across several projects at once, within individual projects and by reporting accuracy checks demonstrating the need to validate a database. When the data comes from individual programmers or certain validated projects, the metrics' correlations with actual effort seem to be strongest. For modules developed entirely by individual programmers, the validity ratios induce a statistically significant ordering of several of the metrics' correlations. When comparing the strongest correlations, neither software science's E metric cyclomatic complexity not source lines of code appears to relate convincingly better with effort than the others.

Basili, Victor R.↗

NASA NTRS1989

Initial Ada components evaluation

The SAIC has the responsibility for independent test and validation of the SSE. They have been using a mathematical functions library package implemented in Ada to test the SSE IV and V process. The library package consists of elementary mathematical functions and is both machine and accuracy independent. The SSE Ada components evaluation includes code complexity metrics based on Halstead's software science metrics and McCabe's measure of cyclomatic complexity. Halstead's metrics are based on the number of operators and operands on a logical unit of code and are compiled from the number of distinct operators, distinct operands, and total number of occurrences of operators and operands. These metrics give an indication of the physical size of a program in terms of operators and operands and are used diagnostically to point to potential problems. McCabe's Cyclomatic Complexity Metrics (CCM) are compiled from flow charts transformed to equivalent directed graphs. The CCM is a measure of the total number of linearly independent paths through the code's control structure. These metrics were computed for the Ada mathematical functions library using Software Automated Verification and Validation (SAVVAS), the SSE IV and V tool. A table with selected results was shown, indicating that most of these routines are of good quality. Thresholds for the Halstead measures indicate poor quality if the length metric exceeds 260 or difficulty is greater than 190. The McCabe CCM indicated a high quality of software products.

Moebes, Travis↗

NASA NTRS2006

Integrated Modeling of Advanced Opto-Mechanical Systems

The design of optical hardware for space applications is particularly challenging when developing high performance, novel systems that have no precedent. Integrated modeling and analysis of such opto-mechanical systems seeks to describe the end-to-end performance of the hardware using mission-relevant metrics. This multidisciplinary analysis might start with thermal disturbances from observation maneuvers, compute the system temperatures, compute the distorted positions and shapes of the hardware and compute the resulting optical performance. Dynamic disturbances such as reaction wheel imbalance or inertia imbalance of optical delay lines might be applied to a structural dynamic model and used in a guidance and control analysis. Mission-relevant science metrics might include wavefront quality, pointing error or imaging stability. Assembling a tool chain that can be both nimble and effective when scaled to the high fidelity models of detail design has been challenging. An integrated thermal, mechanical and optical analysis capability suitable for detail design has been developed and verified through experimental measurement. This capability was used in the design of flight-like breadboard hardware and development of a test apparatus that established both the level of performance of the hardware and the validity of the analysis. The analysis includes prediction of the thermal environment of the test chamber, detailed temperature distributions on the breadboard hardware, fine scale deformations of the optical elements, and computation of the wavefront quality using geometric optics. A battery of tests were conducted to assess the experiment data acquisition, measurement and control system and to establish the performance of the hardware design and accuracy of the integrated modeling. Thermal loads that represent operational observing maneuvers were imposed and the hardware optical performance was measured and compared to analytical predictions.

optomechanical analysis↗

NASA NTRS2015

Lower Limits on Aperture Size for an ExoEarth Detecting Coronagraphic Mission

The yield of Earth-like planets will likely be a primary science metric for future space-based missions that will drive telescope aperture size. Maximizing the exoEarth candidate yield is therefore critical to minimizing the required aperture. Here we describe a method for exoEarth candidate yield maximization that simultaneously optimizes, for the first time, the targets chosen for observation, the number of visits to each target, the delay time between visits, and the exposure time of every observation. This code calculates both the detection time and multiwavelength spectral characterization time required for planets. We also refine the astrophysical assumptions used as inputs to these calculations, relying on published estimates of planetary occurrence rates as well as theoretical and observational constraints on terrestrial planet sizes and classical habitable zones. Given these astrophysical assumptions, optimistic telescope and instrument assumptions, and our new completeness code that produces the highest yields to date, we suggest lower limits on the aperture size required to detect and characterize a statistically motivated sample of exoEarths.

Earth-like planets↗

NASA NTRS2007

A Contrast in Use of Metrics in Earth Science Data Systems

In recent years there has been a surge in the number of systems for processing, archiving and distributing remotely sensed data. Such systems, working independently as well as in collaboration, have been contributing greatly to the advances in the scientific understanding of the Earth system, as well as utilization of the data for nationally and internationally important applications. Among such systems, we consider those that are developed by or under the sponsorship of NASA to fulfill one of its strategic objectives: "Study Earth from space to advance scientific understanding and meet societal needs." NASA's Earth science data systems are of varying size and complexity depending on the requirements they are intended to meet. Some data systems are regarded as NASA's "Core Capabilities" that provide the basic infrastructure for processing, archiving and distributing a set of data products to a large and diverse user community in a robust and reliable manner. Other data systems constitute "Community Capabilities". These provide specialized and innovative services to data users and/or research products offering new scientific insight. Such data systems are generally supported by NASA through peer reviewed competition. Examples of Core Capabilities are 1. Earth Observing Data and Information System (EOSDIS) with its Distributed Active Archive Centers (DAACs), Science Investigator-led Processing Systems (SIPSs), and the EOS Clearing House (ECHO); 2. Tropical Rainfall Measurement Mission (TRMM) Science Data and Information System (TSDIS); 3. Ocean Data Processing System (ODPS); and 4. CloudSat Data Processing Center. Examples of Community Capabilities are projects under the Research, Education and Applications Solutions Network (REASON), and Advancing Collaborative Connections for Earth System Science (ACCESS) Programs. In managing these data system capabilities, it is necessary to have well-established goals and to measure progress relative to them. Progress is measured through "metrics", which can be a combination of quantitative as well as qualitative assessments. The specific metrics of interest depend on the user of the metrics as well as the type of data system. The users of metrics can be data system managers, program managers, funding agency or the public. Data system managers need metrics for assessing and improving the performance of the system and for future planning. Program managers need metrics to assess progress and the value of the data systems sponsored by them. Also, there is a difference in the metrics needed for core capabilities that tend to be more complex, larger and longer-term compared to community capabilities and the community capabilities that tend to be simpler, smaller and shorter-term. Even among community capabilities there are differences; hence the same set of metrics does not apply to all. Some provide data products to users, some provide services that enable better utilization of data or interoperability among other systems, and some are a part of a larger project where provision of data or services is only a minor activity. There is also a contrast between metrics used for internal and external purposes. Examples of internal purposes are: ensuring that the system meets its requirements, and planning for evolution and growth. Examples of external purposes are: providing to sponsors indicators of success of the systems, demonstrating the contributions of the system to overall program success, etc. This paper will consider EOSDIS, REASON and ACCESS programs to show the various types of metrics needed and how they need to be tailored to the types of data systems while maintaining the overall management goals of measuring progress and contributions made by the data systems.

Ramapriyan, Hampapuram↗

DOE OSTI2020

Science Resource Office Peer Review and Metrics Factsheet [Slides]

The Science Resource Office-Peer Review and Metrics (SRO-PRM) Team hosts and maintains LANL's central repository for Science, Technology, and Engineering (STE) data. The Team tracks, integrates, analyzes, and reports qualitative and quantitative data measuring performance, quality, impact, and productivity. These metrics are indicators of the effectiveness and impact of STE efforts.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2023

Evaluating energy justice metrics in early-stage science and technology research using the JUST-R metrics framework

Embedding principles of energy justice throughout all aspects of clean energy technology research and development (R&D) can facilitate a more just energy transition; yet gaps remain in our understanding of how to best integrate energy justice from the earliest R&D stages. The Justice Underpinning Science and Technology Research (JUST-R) metrics framework has been developed to enable early-stage energy researchers to assess and address justice considerations associated with their research, but the impacts of the framework, and others like it, have yet to be evaluated. This study seeks to evaluate the JUST-R metrics framework in terms of its effectiveness and appeal to researchers engaged in early-stage technical R&D using qualitative analyses of documents and workshop transcripts. We find that the metrics framework helps researchers identify problems and potential solutions surrounding the energy justice implications of their work and spurs a change in perspective for researchers, but, simultaneously, there is no evidence of solution follow-through within the evaluation timeframe. Greater institutional support, specialization to research areas, knowledge of energy justice fundamentals, and earlier incorporation of energy justice considerations in research projects arise as factors needed to aid continued use of the framework and pursuit of identified solutions. This evaluation protocol and these findings can serve as a guide for improving other frameworks with similar goals of encouraging sociotechnical engagement in early-stage energy R&D.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

NASA NTRS2018

User Metrics in NASA Earth Science Data Systems

This presentation the collection and use of user metrics in NASA's Earth Science data systems. A variety of collection methods is discussed, with particular emphasis given to the American Customer Satisfaction Index (ASCI). User sentiment on potential use of cloud computing is presented, with generally positive responses. The presentation also discusses various forms of automatically collected metrics, including an example of the relative usage of different functions within the Giovanni analysis system.

user metrics↗

DOE OSTI2022

Metrics as tools for bridging climate science and applications

In climate science and applications, the term “metric” is used to describe the distillation of complex, multifaceted evaluations to summarize the overall quality of a model simulation, or other data product, and/or as a means to quantify some response to climate change. Metrics provide insights into the fidelity of processes and outcomes from climate models and can assist with both differentiating models' representation of variables or processes and informing whether models are “fit for purpose.” Metrics can also provide a valuable reference point for co-production of knowledge between climate scientists and climate impact practitioners. Although continued metric developments enable model developers to better understand the impacts of decisions made in the model design process, metrics also have implications for the characterization of uncertainty and facilitating analyses of underlying physical processes. As a result, comprehensive evaluation with multiple metrics enhances usability of climate information by both scientific and stakeholder communities. Here, this paper presents examples of insights gained from the development and appropriate use of metrics, and provides examples of how metrics can be used to engage with stakeholders and inform decision-making.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2003

Evaluation of Human vs. Teleoperated Robotic Performance in Field Geology Tasks at a Mars Analog Site

Exploration mission designers and planners have costing models used to assess the affordability of given missions - but very little data exists on the relative science return produced by different ways of exploring a given region. Doing cost-benefit analyses for future missions requires a way to compare the relative field science productivity of spacesuited humans vs. virtual presence/teleoperation from a nearby habitat or orbital station, vs. traditional terrestrial-controlled rover operations. The goal of this study was to define science-return metrics for comparing human and robotic fieldwork, and then obtain quantifiable science-return performance comparisons between teleoperated rovers and spacesuited humans. Test runs with a simulated 2015-class rover and with spacesuited geologists were conducted at Haughton Crater in the Canadian Arctic in July 2002. Early results imply that humans will be 1-2 orders of magnitude more productive per unit time in exploration than future terrestrially-controlled robots.

Glass, B.↗

NASA NTRS1992

NASA metric transition plan

NASA science publications have used the metric system of measurement since 1970. Although NASA has maintained a metric use policy since 1979, practical constraints have restricted actual use of metric units. In 1988, an amendment to the Metric Conversion Act of 1975 required the Federal Government to adopt the metric system except where impractical. In response to Public Law 100-418 and Executive Order 12770, NASA revised its metric use policy and developed this Metric Transition Plan. NASA's goal is to use the metric system for program development and functional support activities to the greatest practical extent by the end of 1995. The introduction of the metric system into new flight programs will determine the pace of the metric transition. Transition of institutional capabilities and support functions will be phased to enable use of the metric system in flight program development and operations. Externally oriented elements of this plan will introduce and actively support use of the metric system in education, public information, and small business programs. The plan also establishes a procedure for evaluating and approving waivers and exceptions to the required use of the metric system for new programs. Coordination with other Federal agencies and departments (through the Interagency Council on Metric Policy) and industry (directly and through professional societies and interest groups) will identify sources of external support and minimize duplication of effort.

Source record↗

NASA NTRS2019

NASA EOSDIS Data Usage Metrics- Insight and Assessment

NASA's Earth Science Data and Information System (ESDIS) Project collects Earth science data usage metrics on a daily basis through the ESDIS Metrics System (EMS). This includes metrics on distribution of data products, users, data volumes, and number of files, which are key parameters in evaluating system-level performance of any of the Distributed Active Archive Centers (DAACs) encompassed by NASA's Earth Observing System (EOS) Data and Information System (EOSDIS). EOSDIS data usage metrics illustrate the benefits of making NASA data openly available to the public and show a rapid growth in data distribution to a worldwide user community. In fact, each year since 2014 the EOSDIS has distributed over one billion data files of products from EOS satellite, airborne, and in situ observations. An assessment of the long-term trends of data usage metrics and user characterization provides insights into data usability.This study will focus on describing the EMS as a metrics collection tool and will provide a comprehensive analysis of EOSDIS data usage metrics over the last 10 years. This study will also characterize the product distribution metrics by various tools and services, such as Giovanni, the Open-source Project for a Network Data Access Protocol (OPeNDAP), and subsets, to address how these tools/services have extended the usage of data in the EOSDIS collection. Data usage patterns based on discipline and study area will further assist in understanding how EOSDIS data user needs have evolved over time. Results from this study will provide useful information for the DAACs that can help them improve the functionality of their tools and services as well as more efficiently allocate the resources necessary for enhanced access and availability of their data products. Knowledge of these metrics may also benefit user discovery of data in the EOSDIS collection, promote research collaboration, and stimulate new ideas from work and research conducted using specific datasets and data collections.

Kafle, Durga N.↗

NASA NTRS1997

Systems Engineering Metrics: Organizational Complexity and Product Quality Modeling

Innovative organizational complexity and product quality models applicable to performance metrics for NASA-MSFC's Systems Analysis and Integration Laboratory (SAIL) missions and objectives are presented. An intensive research effort focuses on the synergistic combination of stochastic process modeling, nodal and spatial decomposition techniques, organizational and computational complexity, systems science and metrics, chaos, and proprietary statistical tools for accelerated risk assessment. This is followed by the development of a preliminary model, which is uniquely applicable and robust for quantitative purposes. Exercise of the preliminary model using a generic system hierarchy and the AXAF-I architectural hierarchy is provided. The Kendall test for positive dependence provides an initial verification and validation of the model. Finally, the research and development of the innovation is revisited, prior to peer review. This research and development effort results in near-term, measurable SAIL organizational and product quality methodologies, enhanced organizational risk assessment and evolutionary modeling results, and 91 improved statistical quantification of SAIL productivity interests.

Mog, Robert A.↗

DOE OSTI2022

DefectTrack: a deep learning-based multi-object tracking algorithm for quantitative defect analysis of in-situ TEM videos in real-time

Abstract In-situ irradiation transmission electron microscopy (TEM) offers unique insights into the millisecond-timescale post-cascade process, such as the lifetime and thermal stability of defect clusters, vital to the mechanistic understanding of irradiation damage in nuclear materials. Converting in-situ irradiation TEM video data into meaningful information on defect cluster dynamic properties (e.g., lifetime) has become the major technical bottleneck. Here, we present a solution called the DefectTrack , the first dedicated deep learning-based one-shot multi-object tracking (MOT) model capable of tracking cascade-induced defect clusters in in-situ TEM videos in real-time. DefectTrack has achieved a Multi-Object Tracking Accuracy (MOTA) of 66.43% and a Mostly Tracked (MT) of 67.81% on the test set, which are comparable to state-of-the-art MOT algorithms. We discuss the MOT framework, model selection, training, and evaluation strategies for in-situ TEM applications. Further, we compare the DefectTrack with four human experts in quantifying defect cluster lifetime distributions using statistical tests and discuss the relationship between the material science domain metrics and MOT metrics. Our statistical evaluations on the defect lifetime distribution suggest that the DefectTrack outperforms human experts in accuracy and speed.

42 ENGINEERING↗

NASA NTRS2001

Integrated Modeling for the Next Generation Space Telescope "Yardstick" Concept

The so-called NASA "Yardstick" design concept for the Next Generation Space Telescope presents unique challenges for systems-level analysis. Simulations that integrate controls, optics, thermal, and structural models are required to evaluate baseline performance, study design sensitivities, and perform design optimization. An integrated modeling approach was chosen using a combination of commercial off-the-shelf and "in-house" developed codes. The resulting capability provides a foundation for linear and non-linear analysis, using both the time and frequency-domain methods. It readily allows various combinations of design parameters and environmental loads to be evaluated directly in terms of key science-related metrics, in this case the scalar RMS (root mean square) line-of-sight and RMS wavefront errors. This presentation first addresses the development of the component, or discipline, models for the Yardstick design. It will then proceed to present the integration of the component models, using linear-systems approaches, in order to support two of the most critical baseline performance analyses: jitter and thermal-elastic stability of the optical telescope assembly (OTA). The results of the jitter analysis indicate that disturbances from the reaction wheels coupled with the lightly-damped and highly-flexible structure present significant challenges to the baseline line-of-sight control architecture. Vibration isolation will be required to meet jitter error requirements. The results of the thermal-elastic analysis indicate that the mirror segment displacements due to ground-to-orbit cool-down of the telescope are within the expected capture range of the segment rigid-body control actuators. This means we will be able to align and phase the primary mirror. However, the results for the analysis of the thermal transient response following an attitude maneuver (slew) show that this telescope design is not sufficiently stable, passively, to meet the wavefront error requirements. Structural re-design is one possibility; alternatively, active thermal control of the OTA may be considered. The Yardstick integrated models were successfully used to demonstrate the feasibility of two thermal control strategies.

Mosier, Gary E.↗

NASA NTRS2013

Designing the X-Ray Microcalorimeter Spectrometer for Optimal Science Return

Recent advances in X-ray microcalorimeters enable a wide range of possible focal plane designs for the X-ray Microcalorimeter Spectrometer (XMS) instrument on the future Advanced X-ray Spectroscopic Imaging Observatory (AXSIO) or X-ray Astrophysics Probe (XAP). Small pixel designs (75 microns) oversample a 5-10" PSF by a factor of 3-6 for a 10 m focal length, enabling observations at both high count rates and high energy resolution. Pixel designs utilizing multiple absorbers attached to single transition-edge sensors can extend the focal plane to cover a significantly larger field of view, albeit at a cost in maximum count rate and energy resolution. Optimizing the science return for a given cost and/or complexity is therefore a non-trivial calculation that includes consideration of issues such as the mission science drivers, likely targets, mirror size, and observing efficiency. We present a range of possible designs taking these factors into account and their impacts on the science return of future large effective-area X-ray spectroscopic missions.

X-rays↗