Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Software-Defined Network for End-to-end Networked Science at the Exascale

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work we presen here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end-to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. This provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

One-shot gas detection with transformer paired neural networks in Mako collected longwave infrared hyperspectral imagery

To date, careful data treatment workflows and statistical detectors are used to perform hyperspectral image (HSI) detection of any gas contained in a spectral library, which is often expanded with physics models to incorporate different spectral characteristics. In general, surrounding evidence or known gas-release parameters are used to provide confidence in or confirm detection capability, respectively. This makes quantifying detection performance difficult as it is nearly impossible to develop an absolute ground truth for gas target pixel presence in collected HSI. Consequently, developing and comparing new detection methods, especially machine learning (ML)-based methods, is susceptible to subjectivity in derived detection map quality. Here, in this work, we demonstrate the first use of transformer-based paired neural networks (PNNs) for one-shot gas target detection for multiple gases while providing quantitative classification and detection metrics for their use on labeled data. Terabytes of training data are generated from a database of long-wave infrared HSI obtained from historical Mako sensor campaigns over Los Angeles. By incorporating labels, singular signature representations, and a model development pipeline, we can tune and select PNNs to detect multiple gas targets that are not seen in training on a quantitative basis. We additionally assess our test set detections using interpretability techniques widely employed with ML-based predictors, but less common with detection methods relying on learned latent spaces.

Hyperspectral imaging↗
DOE OSTI2020

Evolution of the Data Quality Monitoring and Prompt Processing System in the proto DUNE-SP experiment

The DUNE Collaboration currently operates an experimental program based at CERN which includes a beam test and an extended cosmic ray run of two large-scale prototypes of the massive Liquid Argon Time Projection Chamber (LArTPC) for the DUNE Far Detector. The volume of data collected by the single-phase prototype (protoDUNE-SP) amounts to 3PB and the sustained rate of data sent to mass storage is O(100) MB/s. Data Quality Monitoring was implemented by directing a fraction of the data stream to the protoDUNE prompt processing system (p3s) which is optimized for continuous low-latency calculation of the vital detector metrics and various graphics including event displays. It served a crucial role throughout the life cycle of the experiment. We present our experience in leveraging the CERN computing environment and operating the system over an extended period of time, while adapting to evolving requirements and computing platform.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2008

Experiences in Bridging the Gap between Science and Decision Making at NASA's GSFC Earth Science Data and Information Services Center (GES DISC)

Recognizing the significance of NASA remote sensing Earth science data in monitoring and better understanding our planet s natural environment, NASA has implemented the Decision Support Through Earth Science Research Results program (NASA ROSES solicitations). a) This successful program has yielded several monitoring, surveillance, and decision support systems through collaborations with benefiting organizations. b) The Goddard Space Flight Center (GSFC) Earth Sciences Data and Information Services Center (GES DISC) has participated in this program on two projects (one complete, one ongoing), and has had opportune ad hoc collaborations gaining much experience in the formulation, management, development, and implementation of decision support systems utilizing NASA Earth science data. c) In addition, GES DISC s understanding of Earth science missions and resulting data and information, including data structures, data usability and interpretation, data interoperability, and information management systems, enables the GES DISC to identify challenges that come with bringing science data to decision makers. d) The purpose of this presentation is to share GES DISC decision support system project experiences in regards to system sustainability, required data quality (versus timeliness), data provider understanding of how decisions are made, and the data receivers willingness to use new types of information to make decisions, as well as other topics. In addition, defining metrics that really evaluate success will be exemplified.

Kempler, Steven↗
DOE OSTI2020

A Novel and Practical Method to Quantify the Quality of Mobility: The Mobility Energy Productivity Metric: Preprint

Recent technology innovations are enabling fundamental improvements in mobility systems, including options for new travel modes, methods, and opportunities to connect people with goods, services, and employment. A desire to quantify and compare both existing and emerging transportation options motivated development of the mobility energy productivity (MEP) metric described herein. The MEP metric fundamentally measures the potential of a city's transportation system to connect a person to a variety of services and activities that define a high-quality of life, relative to the convenience, cost and energy needed to provide these connections. Fundamentally derived from accessibility theory, the MEP advances practice by using readily available travel time data (either from web-based application programming interfaces (APIs) or outputs from an urban transportation model) combined with established parameters that reflect the energy intensity and cost of various travel modes, and relative frequency of activity engagement. The construction of the MEP metric allows for aggregation and disaggregation to the appropriate spatial, modal, and trip purpose resolution, as analysis needs dictate. The MEP could be used to compare alternative futures related to technology, infrastructure investment, or policy, providing a much-needed tool for planners, researchers, and analysts.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1986

Maximum normalized acceleration as a flying qualities parameter

In 1984, Maximum Normalized Rate (MNR) was presented as a Flying Qualities parameter. Subsequent analysis of data from ground based simulation and flight test revealed the utility of a companion parameter, Maximum Normalized Acceleration (MNA). MNR and MNA profiles reveal the presence of both continuous and pulsed compensation strategies during discrete attitude tracking. In addition, MNR appears to be a suitable metric for pilot opinion in the LATHOS data base, while the MNR/MNA relationship is sensitive to pilot-induced-oscillation (PIO) and roll ratcheting problems. As part of an investigation of this problem, Northrop has developed an analysis technique known as the Step Target Method. The Step Target method is essentially a one degree-of-freedom simulation, where an attitude command in the form of a step function is presented to a closed-loop pilot/aircraft model. The two parameters MNR and MNA were shown to be useful in Flying Qualities analysis. MNR was shown to correlate with Pilot Opinion Rating in the LATHOS data base, while MNA reflects PIO and roll ratcheting. Profiles of MNR versus MNA reveal the presence of pulsed compensation strategies in both ground based and in-flight simulation. Furthermore, comparison of continuous and discrete attitude tracking simulation data reveals that these two tracking tasks exhibit independent sensitivities to aircraft characteristics.

Warner, J. S.↗
DOE OSTI2021

cuZ-Checker: A GPU-Based Ultra-Fast Assessment System for Lossy Compressions

Lossy compression is becoming an indispensable technique for the success of today's extreme-scale high-performance computing projects that produce vast volumes of data during scientific simulations or instrument data acquisitions. Comprehensively understanding the compression quality and performance of different lossy compressors is critical to selecting the best-fit compressors and using them properly and efficiently in practice. A few lossy compression assessment tools (e.g., Z-checker) have been developed, but none of them support the execution in a GPU environment. This is a significant gap because many recent extreme-scale applications and lossy compressors (e.g., cuSZ) can run entirely within GPUs. In this work, we develop an efficient lossy compression measuring system (called cuZ-Checker) on the GPU platform, which aims to perform the lossy compression quality and performance assessment completely within the GPU environment. Our contribution is threefold. (1) We develop a novel GPU-based lossy compression measuring framework using a computation pattern-based design approach. This approach classifies the computing-intensive metrics into three categories based on their patterns which creates large opportunities for kernel fusion and data reuse. (2) For each pattern in cuZ-Checker, we develop a CUDA kernel and provide fine-grained optimizations to boost its performance. (3) We thoroughly evaluate our cuZ-checker on a V100 GPU using four real-world scientific application datasets. Experiments show that cuZ-Checker can significantly accelerate the overall lossy compression assessment performance by 23X similar to 31X compared with the OpenMP-based multithreading CPU performance. To the best of our knowledge, this is the first lossy compression measuring system designed for GPU devices.

GPU↗
DOE OSTI2024

Disturbance and diversity: Lichen species richness decreases with increasing anthropogenic disturbance

Anthropogenic disturbance is rapidly increasing through habitat degradation, development, and deforestation. Gaps remain in understanding the effects of this disturbance on diverse and ecologically important organisms such as lichens. In North America, studies have focused on epiphytic macrolichens and catastrophic disturbance, largely ignoring microlichens and less severe disturbances. The Intermediate Disturbance Hypothesis predicts these moderate disturbances will lead to higher species richness. Here we examine the effects of anthropogenic disturbance in the form of land management on overall lichen richness, including microlichens, and on the species richness of eight lichen functional groups. The study draws on a comprehensive data set of 872 species, in 208 one-hectare plots throughout the Southern Appalachian Mountains, a global biodiversity hotspot in eastern North America. A habitat quality index based on an established forestry metric was used as a proxy for anthropogenic disturbance and was quantified using a 10-part score including categories such as percent native tree canopy cover and degree of fragmentation. Linear models were used to compare habitat quality scores to overall species richness and to species richness of functional groups. Rather than following the Intermediate Disturbance Hypothesis, all groups uniformly follow a negative linear relationship: as disturbance increases, species richness decreases. This pattern held even for widely variable functional groups such as morphotype. Effective conservation of lichen richness should prioritize the maintenance of existing older, less-disturbed stands within large, contiguously forested areas.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A Summary of Validation Studies for the Integrated TIGER Series Performed on ACORN Plus-up 218468/99

The Integrated TIGER Series (ITS) transport code is a valuable tool for photon-electron transport. A seven-problem validation suite exists to make sure that the ITS transport code works as intended. It is important to ensure that data from benchmark problems is correctly compared to simulated data. Additionally, the validation suite did not previously make use of a consistent quantitative metric for comparing experimental and simulated datasets. To this end, the goal of this long-term project was to expand the validation suite both in problem type and in the quality of the error assessment. To accomplish that, the seven validation problems in the suite were examined for potential drawbacks. When a drawback was identified, the problems were ranked based on severity of the drawback and approachability of a solution. We determined that meaningful improvements could be made to the validation suite by improving the analysis for the Lockwood Albedo problem and by introducing the Ross dataset as an eighth problem to the suite. The Lockwood error analysis has been completed and will be integrated in the future. The Ross data is unfinished, but significant progress has been made towards analysis.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Lessons Learned From Smallsat Microwave Sounders Data Assimilation in the NASA Goddard Earth Observing System (GEOS)

The NASA Time-Resolved Observations of Precipitation structure and storm Intensity with a Constellation of Smallsats (TROPICS) mission is a constellation of small satellites carrying state-of-art microwave temperature and humidity sounders with 12 channels between 91 GHz and 205 GHz frequency. Including TROPICS-pathfinder, launched on 30 June 2021, five TROPICS satellites operate and provide temperature and humidity data to NWP and atmospheric retrieval communities. This study evaluates the quality of radiance data from these five satellites. It assesses data impacts on NASA Goddard Earth Observing System (GEOS) global NWP analysis and forecasts while seeking answers to the following key questions: 1. What information do TROPICS satellites bring to NWP analysis? 2. How will TROPICS data assimilation affect NWP forecasts, including tropical cyclone analysis and forecasts? 3. What are the benefits of having multiple TROPICS satellites in NWP forecasts? 4. What are the weaknesses regarding instrument stability, consistency, and data quality of these CubeSats? 5. Can multiple TROPICS make similar impacts as conventional MW sensors such as ATMS or GPM Microwave Imager (GMI)? The sensitivities of each TROPICS channel to temperature, water vapor, cloud, and precipitation are examined in various weather conditions. Results from observing system experiments assimilating TROPICS radiance are used for evaluation metrics, including forecast skills and fit to other observations such as radiosondes and microwave and infrared sounders. Finally, the lessons learned from using these cube satellite radiance data in the GEOS atmospheric data assimilation system are shared.

Min-Jeong Kim↗
NASA NTRSDate not supplied

Linear Regression Model for Predictive Service Provider Selection

The increasing number of satellites in orbit has led to a growing reliance on third-party service providers for data transfer between Earth and space. Traditional approaches to managing satellite communications require human intervention, which becomes more burdensome with the escalating number of satellites. This research addresses the need for an efficient and automated system to optimize service provider selection for NASA space communication. Previous research has utilized human-operated approaches for service provider management. Our study fills a gap by developing a cognitive algorithm that automates and optimizes the selection process based on various parameters, such as data volume, priority, quality of service and cost. This novel solution reduces user burden, facilitates service management, and contributes to the development of cognitive spaceflight missions, ultimately supporting NASA’s research into Cognitive Communications technology. The algorithm design consists of three major steps: modeling data, developing a Link Selection Algorithm (LSA) based on a grading system, and applying machine learning using linear regression. The LSA evaluates providers based on user-defined constraints, considering factors such as delivery time, cost, and quality of service. We define a suitability metric which allows our algorithm to make a recommendation to a user regarding which commercial service providers to select. The addition of Linear Regression predicts the future suitability value. Our main findings demonstrate that the resulting algorithm can autonomously manage connections between satellites and providers, maximizing communication channel efficiency. This research has significant implications, as it not only addresses a pressing issue in satellite communication management but also advances the field of cognitive spaceflight missions.

Linear regression↗
NASA NTRS2015

Integrated Medical Model Project - Overview and Summary of Historical Application

Introduction: The Integrated Medical Model (IMM) Project represents one aspect of NASA's Human Research Program (HRP) to quantitatively assess medical risks to astronauts for existing operational missions as well as missions associated with future exploration and commercial space flight ventures. The IMM takes a probabilistic approach to assessing the likelihood and specific outcomes of one hundred medical conditions within the envelope of accepted space flight standards of care over a selectable range of mission capabilities. A specially developed Integrated Medical Evidence Database (iMED) maintains evidence-based, organizational knowledge across a variety of data sources. Since becoming operational in 2011, version 3.0 of the IMM, the supporting iMED, and the expertise of the IMM project team have contributed to a wide range of decision and informational processes for the space medical and human research community. This presentation provides an overview of the IMM conceptual architecture and range of application through examples of actual space flight community questions posed to the IMM project. Methods: Figure 1 [see document] illustrates the IMM modeling system and scenario process. As illustrated, the IMM computational architecture is based on Probabilistic Risk Assessment techniques. Nineteen assumptions and limitations define the IMM application domain. Scenario definitions include crew medical attributes and mission specific details. The IMM forecasts probabilities of loss of crew life (LOCL), evacuation (EVAC), quality time lost during the mission, number of medical resources utilized and the number and type of medical events by combining scenario information with in-flight, analog, and terrestrial medical information stored in the iMED. In addition, the metrics provide the integrated information necessary to estimate optimized in-flight medical kit contents under constraints of mass and volume or acceptable level of mission risk. Results and Conclusions: Historically, IMM simulations support Science and Technology planning, Exploration mission planning, and ISS program operations by supplying simulation support, iMED data information, and subject matter expertise to Crew Health and Safety and the HRP. Upcoming release of IMM version 4.0 seeks to provide enhanced functionality to increase the quality of risk decisions made using the IMM through a more accurate representation of the real world system.

Myers, J.↗
NASA NTRS1995

Metrics and Benchmarks for Visualization

What is a "good" visualization? How can the quality of a visualization be measured? How can one tell whether one visualization is "better" than another? I claim that the true quality of a visualization can only be measured in the context of a particular purpose. The same image generated from the same data may be excellent for one purpose and abysmal for another. A good measure of visualization quality will correspond to the performance of users in accomplishing the intended purpose, so the "gold standard" is user testing. As a user of visualization software (or at least a consultant to such users) I don't expect visualization software to have been tested in this way for every possible use. In fact, scientific visualization (as distinct from more "production oriented" uses of visualization) will continually encounter new data, new questions and new purposes; user testing can never keep up. User need software they can trust, and advice on appropriate visualizations of particular purposes. Considering the following four processes, and their impact on visualization trustworthiness, reveals important work needed to create worthwhile metrics and benchmarks for visualization. These four processes are (1) complete system testing (user-in-loop), (2) software testing, (3) software design and (4) information dissemination. Additional information is contained in the original extended abstract.

Uselton, Samuel P.↗
DOE OSTI2025

SITCOMTN-154: Initial studies of photometric redshifts with LSSTComCam from DP1

This technote holds reports based on the first analyses of the Data Preview 1 (DP1) data by the Science Unit for photometric redshifts. Although photometric redshifts are not an official DP1 data product, the "Photo-z Science Unit" generated photo-z estimates for every galaxy in DP1 using the available multi-band imaging on a best-effort basis. This work included developing training and test datasets by matching DP1 data to high-quality reference redshifts obtained with spectroscopy, Grism data, and multi-band photometry. The Science Unit used the RAIL software package to make photometric redshift estimates using eight different algorithms, developed simple scientific performance metrics, used those metrics to explore how the performance of the algorithms varied with configuration changes, derived more optimized configurations of the algorithms and tested the performance of those configurations. This work, the resulting data products and expected data distribution mechanism are all described there.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1997

A Validation of Object-Oriented Design Metrics as Quality Indicators

This paper presents the results of a study in which we empirically investigated the suits of object-oriented (00) design metrics introduced in another work. More specifically, our goal is to assess these metrics as predictors of fault-prone classes and, therefore, determine whether they can be used as early quality indicators. This study is complementary to the work described where the same suite of metrics had been used to assess frequencies of maintenance changes to classes. To perform our validation accurately, we collected data on the development of eight medium-sized information management systems based on identical requirements. All eight projects were developed using a sequential life cycle model, a well-known 00 analysis/design method and the C++ programming language. Based on empirical and quantitative analysis, the advantages and drawbacks of these 00 metrics are discussed. Several of Chidamber and Kamerer's 00 metrics appear to be useful to predict class fault-proneness during the early phases of the life-cycle. Also, on our data set, they are better predictors than 'traditional' code metrics, which can only be collected at a later phase of the software development processes.

Basili, Victor R.↗
NASA NTRS2004

The coverage of elliptical orbits using Ergodic theory

One of the key performance metrics for satellite constellations is the statistics of the visibility periods between the satellites and poins on the ground. Associated with this are other desirable communications statistis such as data through-put, link qualities, etc. Typically, the computation of coverage statistics requires the propagation of the trajectories. In this paper a new algorithm using differential geometry enables us to extend the Ergodic theory to eliptical orbits.

Lo, Martin W.↗
NASA NTRS2007

A Comparison of Two Balance Calibration Model Building Methods

Simulated strain-gage balance calibration data is used to compare the accuracy of two balance calibration model building methods for different noise environments and calibration experiment designs. The first building method obtains a math model for the analysis of balance calibration data after applying a candidate math model search algorithm to the calibration data set. The second building method uses stepwise regression analysis in order to construct a model for the analysis. Four balance calibration data sets were simulated in order to compare the accuracy of the two math model building methods. The simulated data sets were prepared using the traditional One Factor At a Time (OFAT) technique and the Modern Design of Experiments (MDOE) approach. Random and systematic errors were introduced in the simulated calibration data sets in order to study their influence on the math model building methods. Residuals of the fitted calibration responses and other statistical metrics were compared in order to evaluate the calibration models developed with different combinations of noise environment, experiment design, and model building method. Overall, predicted math models and residuals of both math model building methods show very good agreement. Significant differences in model quality were attributable to noise environment, experiment design, and their interaction. Generally, the addition of systematic error significantly degraded the quality of calibration models developed from OFAT data by either method, but MDOE experiment designs were more robust with respect to the introduction of a systematic component of the unexplained variance.

DeLoach, Richard↗
NASA NTRSDate not supplied

Use of Spectral Analysis of Singular Values as a Test Metric for IMMAT Trials

One of many challenges in the implementation of multiple exciter testing is establishing a reasonable set of test metrics to measure the quality of testing. This is especially true in the application of Impedance Matched Multi-Axis Testing; in that it is possible to have very large spectral density matrices that serve as reference criteria. While there exist plotting schemes to view a spectral density matrix, it is often necessary to break the overlay of reference and test results into subsections of the matrices to get sufficient resolution to interpret the data. In addition, as one attempts to control multiple locations on a structure, implementation of classical single degree-of freedom test tolerances across all channels and associated cross spectra is simply not feasible. Hence it is challenging to evaluate overall test quality. The use of spectral views of the dominant singular values from the singular value decomposition of the spectral density matrices and metrics based upon them is proposed for establishing a set of compact metrics for evaluating test quality. A laboratory experiment will be included to demonstrate this proposed technique.

Impedance Matched Multi-Axis Testing↗