Search NASA⌕ Search

SEARCH · Search NASA

Results for “scoring system”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

An Anomaly Correlation Skill Score for the Evaluation of the Performance of Hyperspectral Infrared Sounders

With the availability of very accurate forecasts, the metric of accuracy alone for the evaluation of the performance of a retrieval system can produce misleading results. A useful characterization of the quality of a retrieval system and its potential to contribute to an improved weather forecast is its skill, which we define as the ability to make retrievals of geophysical parameters which are closer to the truth than the six hour forecast, when the truth differs significantly from the forecast. We illustrate retrieval skill using one day of AMSU and AIRS data with three different retrieval algorithms, which result in retrievals for more than 90% of the potential retrievals under clear and cloudy conditions. Two of the three algorithms have better than 1 K rms "RAOB quality" accuracy on the troposphere, but only one has skill between 900 and 100 mb. AIRS was launched on the EOS Aqua spacecraft in May 2002 into a 705 km polar sun-synchronous orbit with accurately maintained 1:30 PM ascending node. Essentially uninterrupted data are freely available since September 2002.

retrieval↗
DOE OSTI2022

The Application of a Genetic Algorithm to the Optimization of a Mesoscale Model for Emergency Response

Besides solving the equations of momentum, heat, and moisture transport on the model grid, mesoscale weather models must account for subgrid-scale processes that affect the resolved model variables. These are simulated with model parameterizations, which often rely on values preset by the user. Such “free” model parameters, along with others set to initialize the model, are often poorly constrained, requiring that a user select each from a range of plausible values. Finding the values to optimize any forecasting tool can be accomplished with a search algorithm, and one such process—the genetic algorithm (GA)—has become especially popular. As applied to modeling, GAs represent a Darwinian process: an ensemble of simulations is run with a different set of parameter values for each member, and the members subsequently judged to be most accurate are selected as “parents” who pass their parameters onto a new generation. At the U.S. Department of Energy’s Savannah River Site in South Carolina, we are applying a GA to the Regional Atmospheric Modeling System (RAMS) mesoscale weather model, which supplies input to a model to simulate the dispersion of an airborne contaminant as part of the site’s emergency response preparations. An ensemble of forecasts is run each day, weather data are used to “score” the individual members of the ensemble, and the parameters from the best members are used for the next day’s forecasts. As meteorological conditions change, the parameters change as well, maintaining a model configuration that is best adapted to atmospheric conditions. Significance Statement We wanted to develop a forecasting system by which a weather model is run over the Savannah River Site each day and repeatedly adjusted according to how well it performed the previous day. To run the model, a series of values (parameters) must be set to control how the model will calculate winds, temperatures, and other desired variables. Each day the model was run several times using different combinations of these parameters and later compared with observed meteorological conditions. Parameters that produced the most accurate forecasts were preferentially reused to create the forecasts for the next day. The process was tested for the summer of 2020 and exhibited lower errors than forecasts produced by the model using default values of the parameters.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1969

Improved perceptual-motor performance measurement system

Battery of tests determines the primary dimensions of perceptual-motor performance. Eighteen basic measures range from simple tests to sophisticated electronic devices. Improved system has one unit for the subject containing test display and response elements, and one for the experimenter where test setups, programming, and scoring are accomplished.

Parker, J. F., Jr.↗
DOE OSTI2021

Developing Technology Performance Level Assessments for Early-Stage Wave Energy Converter Technologies: Preprint

The advantage of using Technology Performance Level (TPL) in conjunction with Technology Readiness Level (TRL) assessments in guiding technology development trajectories to successful outcomes in less time, at less overall cost, and with less encountered risk has been well articulated in the literature. In partnership with industry and international collaborators, a TPL assessment methodology for grid-connected applications has been developed through the application of the systems engineering approach. Metrics under seven different categories have been developed, weighted based on their relative relevance, and combined to yield a composite score. The methodology has been implemented in a spreadsheet tool plus a web application specifically aimed at assessing early stage (TRL 1-3) concepts. The target use cases are (a) technology developers improving their design, to find fatal flaws early, to get feedback on current design, to identify areas of improvement that will yield the highest return on investment, (b) reviewers assessing technologies in competitions or for making funding decisions, (c) investor or project developer doing due diligence, (d) policy makers landscaping the technology domain for formulating R&D strategy. The methodology and the tools are undergoing continuous improvement based on the experience and lessons learnt from applying it to internal and external marine energy technology development projects. The methodology is also being adapted for assessing WECs servicing markets outside the continental grid - broadly categorized as Powering the Blue Economy (PBE) applications. Such applications have vastly different functional requirements entailing a modification of the methodology to account for their higher risk tolerance, reduced price sensitivities, lower power needs, different permitting protocols, etc. This paper presents the latest status of the TPL assessment methodology and tools, describes its adaptation to select PBE markets, and explores its extension to other domains where it could provide a comprehensive and holistic measure of a nascent or disruptive technology's technoeconomic performance potential.

metrics↗
DOE OSTI2021

Developing Technology Performance Level Assessments for Early-Stage Wave Energy Converter Technologies

The advantage of using Technology Performance Level (TPL) in conjunction with Technology Readiness Level (TRL) assessments in guiding technology development trajectories to successful outcomes in less time, at less overall cost, and with less encountered risk has been well articulated in the literature. In partnership with industry and international collaborators, a TPL assessment methodology for grid-connected applications has been developed through the application of the systems engineering approach. Metrics under seven different categories have been developed, weighted based on their relative relevance, and combined to yield a composite score. The methodology has been implemented in a spreadsheet tool plus a web application specifically aimed at assessing early stage (TRL 1-3) concepts. The target use cases are (a) technology developers improving their design, to find fatal flaws early, to get feedback on current design, to identify areas of improvement that will yield the highest return on investment, (b) reviewers assessing technologies in competitions or for making funding decisions, (c) investor or project developer doing due diligence, (d) policy makers landscaping the technology domain for formulating R&D strategy. The methodology and the tools are undergoing continuous improvement based on the experience and lessons learnt from applying it to internal and external marine energy technology development projects. The methodology is also being adapted for assessing WECs servicing markets outside the continental grid - broadly categorized as Powering the Blue Economy (PBE) applications. Such applications have vastly different functional requirements entailing a modification of the methodology to account for their higher risk tolerance, reduced price sensitivities, lower power needs, different permitting protocols, etc. This paper presents the latest status of the TPL assessment methodology and tools, describes its adaptation to select PBE markets, and explores its extension to other domains where it could provide a comprehensive and holistic measure of a nascent or disruptive technology's technoeconomic performance potential.

metrics↗
DOE OSTI2020

Frequency Event Categorization in Power Distribution Systems Using Micro PMU Measurements

Frequency is an important metric for power stability. To enhance the situational awareness in power distribution systems, this article proposes a data-driven approach that utilizes the data from micro phasor measurement units (μPMU) to categorize frequency events observed in distribution feeders. The proposed approach has three layers, each of which analyzes if a frequency event is propagated from the transmission system, formed inside a distribution feeder or falsely produced by instantaneous frequency (IF) estimation algorithms, respectively. In the first layer, the Granger causality is utilized to compare the causal relations from measurements across different locations to decide if an event is propagated from the transmission side. In the second layer, a trained sparse coding dictionary is utilized to unmix the physical causes of a frequency event. In the third layer, a moving z-score based event-pair detector is applied to identify if a frequency event is algorithm-induced. The proposed approach is tested using the μPMU measurements collected from Lawrence Berkeley National Laboratory (LBNL) and Riverside Public Utility (RPU).

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

An Evaluation and Qualification of U.S.-Based Research Reactors for Irradiation Capabilities Supporting Advanced Nuclear Systems

Irradiation experiments are a prerequisite for evaluating nuclear reactor system designs, analyzing the performance of these systems, and obtaining licenses. Likewise, irradiation facilities are necessary for producing the radioisotopes used in industrial and medical applications. Recent developments in modeling and simulation capabilities and advancements in computational resources have further enabled the design of irradiation experiments for evaluating radiation-induced phenomena and determining nuclear fuel, material, and system design and safety criteria pertaining to both normal and accident scenarios. These computational tools and models require comprehensive experimental datasets acquired under prototypic radiation conditions—for exploring material and system performance under the uniquely harsh environments found in nuclear reactors—to enable verification and validation for qualification and licensing purposes. However, qualification of irradiation experimental facilities, primarily research and test reactors (RTRs), necessitates that their performance be evaluated based on the irradiation environment (e.g. flux, power, testing capabilities) using an appropriate scoring matrix. Although many university campus RTRs are available for research and development (R&D) activities and initiatives, this study focuses on evaluating and qualifying the irradiation facilities (mostly RTRs) within the United States that are suitable for advanced nuclear fuel, material, and system irradiation experiments aimed at establishing operational-performance limits and informing component and fuel designs so as to improve operational efficiencies and mitigate proliferation vulnerabilities, as well as for radioisotope production aimed at multipurpose applications. As a result, the findings of the present study support the acceleration of nuclear fuel and material qualifications, thus hastening new and advanced nuclear energy system demonstrations and radioisotope production efforts by using extended R&D.

irradiation experiment↗
DOE OSTI2025

Discrete generative diffusion models without stochastic differential equations: A tensor network approach

Diffusion models (DMs) are a class of generative machine learning methods that sample a target distribution by transforming samples of a trivial (often Gaussian) distribution using a learned stochastic differential equation. In standard DMs, this is done by learning a “score function” that reverses the effect of adding diffusive noise to the distribution of interest. Here we consider the generalisation of DMs to lattice systems with discrete degrees of freedom, and where noise is added via Markov chain jump dynamics. We show how to use tensor networks (TNs) to efficiently define and sample such “discrete diffusion models” (DDMs) without explicitly having to solve a stochastic differential equation. We show the following: (i) by parametrising the data and evolution operators as TNs, the denoising dynamics can be represented exactly; (ii) the auto-regressive nature of TNs allows to generate samples efficiently and without bias; (iii) for sampling Boltzmann-like distributions, TNs allow to construct an efficient learning scheme that integrates well with Monte Carlo. We illustrate this approach to study the equilibrium of two models with non-trivial thermodynamics, the d = 1 constrained Fredkin chain and the d = 2 Ising model. Published by the American Physical Society 2025

Causer, Luke (ORCID:0000000194243473)↗
NASA NTRS2015

The Ocean Colour Climate Change Initiative: I. A Methodology for Assessing Atmospheric Correction Processors Based on In-Situ Measurements

The Ocean Colour Climate Change Initiative intends to provide a long-term time series of ocean colour data and investigate the detectable climate impact. A reliable and stable atmospheric correction procedure is the basis for ocean colour products of the necessary high quality. In order to guarantee an objective selection from a set of four atmospheric correction processors, the common validation strategy of comparisons between in-situ and satellite derived water leaving reflectance spectra, is extended by a ranking system. In principle, the statistical parameters such as root mean square error, bias, etc. and measures of goodness of fit, are transformed into relative scores, which evaluate the relationship of quality dependent on the algorithms under study. The sensitivity of these scores to the selected database has been assessed by a bootstrapping exercise, which allows identification of the uncertainty in the scoring results. Although the presented methodology is intended to be used in an algorithm selection process, this paper focusses on the scope of the methodology rather than the properties of the individual processors.

Climate Change↗
DOE OSTI2022

Quantum model learning agent: characterisation of quantum systems through machine learning

Accurate models of real quantum systems are important for investigating their behaviour, yet are difficult to distil empirically. Here, we report an algorithm—the quantum model learning agent (QMLA)—to reverse engineer Hamiltonian descriptions of a target system. We test the performance of QMLA on a number of simulated experiments, demonstrating several mechanisms for the design of candidate Hamiltonian models and simultaneously entertaining numerous hypotheses about the nature of the physical interactions governing the system under study. QMLA is shown to identify the true model in the majority of instances, when provided with limited a priori information, and control of the experimental setup. Our protocol can explore Ising, Heisenberg and Hubbard families of models in parallel, reliably identifying the family which best describes the system dynamics. We demonstrate QMLA operating on large model spaces by incorporating a genetic algorithm to formulate new hypothetical models. The selection of models whose features propagate to the next generation is based upon an objective function inspired by the Elo rating scheme, typically used to rate competitors in games such as chess and football. In all instances, our protocol finds models that exhibit F 1 score ≥ 0.88 when compared with the true model, and it precisely identifies the true model in 72% of cases, whilst exploring a space of over 250 000 potential models. By testing which interactions actually occur in the target system, QMLA is a viable tool for both the exploration of fundamental physics and the characterisation and calibration of quantum devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

T-FSM: A Scalable Distributed Task-Based System for Frequent Subgraph Pattern Mining from a Big Graph

Finding frequent subgraph patterns in a big graph is an important problem with many applications such as classifying chemical compounds and building indexes to speed up graph queries. Since this problem is NP-hard, some recent parallel and distributed systems have been developed to accelerate the mining. However, they often have a huge memory cost, very long running time, suboptimal load balancing, poor scale-out capability, and possibly inaccurate results. In this article, we propose an efficient system called T-FSM for parallel mining of frequent subgraph patterns in a big graph. T-FSM supports a new anti-monotonic frequentness measure called Fraction-Score, which is more accurate than the widely used MNI measure. The execution engine of T-FSM supports both intra-machine parallelism and inter-machine parallelism. For intra-machine parallelism, T-FSM adopts a novel task-based execution model to ensure high multithreading concurrency, bounded memory consumption, and effective load balancing. For inter-machine parallelism, T-FSM ensures good scale-out performance with a lightweight pattern rebalancing approach that reduces workload skewness of pattern evaluations among machines. To avoid recomputing the contexts for migrated patterns, we design a novel context cache table to support concurrent and asynchronous requesting and caching of remote context data, which can timely evict and garbage collect used pattern contexts that are no longer needed to keep memory consumption bounded. Extensive experiments show that T-FSM is orders of magnitude faster than existing state-of-the-art parallel systems (more than 10×, 51×, 131×, 55× speedup over ScaleMine, DistGraph, Pangolin and Peregrine, respectively) and distributed systems (more than 42× and 88× over ScaleMine and DistGraph, respectively) for frequent subgraph pattern mining, and it scales out satisfactorily to 512 CPU cores on the Polaris supercomputer at Argonne National Laboratory.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

LLMs and GenAI Tools to Depict Contributions of Human Systems to Spaceflight Tasks Execution

Recent advancements in Artificial Intelligence and Machine Learning (AI/ML) technologies, particularly Large Language Models (LLMs) capable of sophisticated syntax analysis, offer substantial potential in automating complex processes, thereby saving time and human resources. This study explores the development of an LLM-driven model designed to analyze and categorize a diverse set of Mars mission tasks into 18 predefined Human System Task Categories (HSTCs) based on their textual descriptions. As part of developing the Crew Health and Performance – Probabilistic Risk Assessment (CHP-PRA projects Performance Risk Model (PRisM) proof-of-concept, we established a framework to project performance scores from small-scale tests onto a preliminary list of Mars tasks. The foundation of our model was a comprehensive spreadsheet populated by NASA experts and clinicians, which detailed each Mars task alongside binary indicators of HSTC involvement. This dataset enabled the initial application of supervised ML, training and testing on existing HSTC labels. The HSTCs were originally defined from a medical system perspective, focusing on task impairments due to deteriorated human health. To expand our model's scope to include categories impacting performance, we face the challenge of generating binary labels (0 or 1) for new categories without pre-existing data. We address this by employing Generative AI (GenAI) software to determine whether a given task involved a new category by asking, "Does task A involve using category B?" We validate our approach by comparing the GenAI's binary classifications with the expert-provided labels for existing HSTCs. Notably, we utilize Ollama [4], a locally hosted GenAI tool that does not require cloud access, thus safeguarding NASA's proprietary data from unauthorized exposure. This study demonstrates the feasibility of leveraging cutting-edge AI tools to advance research, paving the way for automation and rapid decision-making in space exploration.

Mona Matar↗
DOE OSTI2025

DriveSense: A Noise-Resilient Framework for Driving Mode Identification

Accurate drive mode classification is essential for enhancing the reliability and predictive maintenance of heavy-duty electric trucks. This study proposes a novel fuzzy logic-based framework, DriveSense, for real-time drive mode classification, addressing key challenges such as sensor noise, transitional behaviors, and computational efficiency. The proposed approach integrates a two-stage filtering pipeline, combining adaptive outlier removal and a dynamic Kalman filter to enhance data quality. A fuzzy inference system with smoothened trapezoidal membership functions is then applied to classify driving modes into standstill, constant speed, acceleration, and deceleration while mitigating the effects of noise and edge cases. Performance evaluation using real-world and simulated drive cycles demonstrates significant improvements in classification accuracy (up to 97.8%), F1-score (up to 0.97), and robustness against noise, while reducing false positives. Comparative analysis against baseline models, demonstrates DriveSense’s superior accuracy and generalizability across diverse driving patterns. The framework’s lightweight and interpretable fuzzy inference engine operates with low computational latency, ensuring compatibility with real-time embedded systems typical of heavy-duty electric trucks. Moreover, DriveSense models transitional behaviors through overlapping fuzzy sets and adaptive borderline classification logic, enabling smooth identification of subtle shifts such as rolling stops or gradual deceleration. These results highlight DriveSense’s potential to enhance predictive maintenance strategies, reduce downtime, and support scalable, fleet-wide diagnostics.

Kumar, Praveen [Oak Ridge National Laboratory (ORN↗
NASA NTRS1982

The optimal design of involute gear teeth with unequal addenda

The design of a gear mesh is treated with the objective of minimizing the gear size for a given gear ratio, pinion torque, pressure angle, and allowable tooth lengths. Tooth strengths considered include scoring, pitting fatigue, and bending fatigue. Kinematic involute interference is avoided. The design variation on standard spur gear teeth called the long and short addendum system, is considered. In this system the mesh center distance and pressure angle are maintained as is the ability to manufacture the teeth with standard tooling. However, the pinion and gear tooth proportions are altered in order to obtain fewer teeth numbers for the same ratio as standard gears without kinematic involute interference. The effect of this nonstandard gearing geometry with on tooth strengths and gear mesh size are studied. For a 2:1 gearing ratio, the optimal nonstandard gear design is compared with the optimal standard gear design.

Savage, M.↗
NASA NTRS2001

Examination of Automation-Induced Complacency and Individual Difference Variates

Automation-induced complacency has been documented as a cause or contributing factor in many airplane accidents throughout the last two decades. It is surmised that the condition results when a crew is working in highly reliable automated environments in which they serve as supervisory controllers monitoring system states for occasional automation failures. Although many reports have discussed the dangers of complacency, little empirical research has been produced to substantiate its harmful effects on performance as well as what factors produce complacency. There have been some suggestions, however, that individual characteristics could serve as possible predictors of performance in automated systems. The present study examined relationship between the individual differences of complacency potential, boredom proneness, and cognitive failure, automation-induced complacency. Workload and boredom scores were also collected and analyzed in relation to the three individual differences. The results of the study demonstrated that there are personality individual differences that are related to whether an individual will succumb to automation-induced complacency. Theoretical and practical implications are discussed.

Prinzel, Lawrence J., III↗
NASA NTRS1984

The Impact of Pictorial Display on Operator Learning and Performance

The effects of pictorially displayed information on human learning and performance of a simple control task were investigated. The controlled system was a harmonic oscillator and the system response was displayed to subjects as either an animated pendulum or a horizontally moving dot. Results indicated that the pendulum display did not effect performance scores but did significantly effect the learning processes of individual operators. The subjects with the pendulum display demonstrated more vertical internal models early in the experiment and the manner in which their internal models were tuned with practice showed increased variability between subjects.

Miller, R. A.↗
NASA NTRS2007

Feasibility of Turing-Style Tests for Autonomous Aerial Vehicle "Intelligence"

A new approach is suggested to define and evaluate key metrics as to autonomous aerial vehicle performance. This approach entails the conceptual definition of a "Turing Test" for UAVs. Such a "UAV Turing test" would be conducted by means of mission simulations and/or tailored flight demonstrations of vehicles under the guidance of their autonomous system software. These autonomous vehicle mission simulations and flight demonstrations would also have to be benchmarked against missions "flown" with pilots/human-operators in the loop. In turn, scoring criteria for such testing could be based upon both quantitative mission success metrics (unique to each mission) and by turning to analog "handling quality" metrics similar to the well-known Cooper-Harper pilot ratings used for manned aircraft. Autonomous aerial vehicles would be considered to have successfully passed this "UAV Turing Test" if the aggregate mission success metrics and handling qualities for the autonomous aerial vehicle matched or exceeded the equivalent metrics for missions conducted with pilots/human-operators in the loop. Alternatively, an independent, knowledgeable observer could provide the "UAV Turing Test" ratings of whether a vehicle is autonomous or "piloted." This observer ideally would, in the more sophisticated mission simulations, also have the enhanced capability of being able to override the scripted mission scenario and instigate failure modes and change of flight profile/plans. If a majority of mission tasks are rated as "piloted" by the observer, when in reality the vehicle/simulation is fully- or semi- autonomously controlled, then the vehicle/simulation "passes" the "UAV Turing Test." In this regards, this second "UAV Turing Test" approach is more consistent with Turing s original "imitation game" proposal. The overall feasibility, and important considerations and limitations, of such an approach for judging/evaluating autonomous aerial vehicle "intelligence" will be discussed from a theoretical perspective.

Young, Larry A.↗
NASA NTRS2014

Modeling of Blood Lead Levels in Astronauts Exposed to Lead from Microgravity-Accelerated Bone Loss

Human exposure to lead has been associated with toxicity to multiple organ systems. Studies of various population groups with relatively low blood lead concentrations (<10 μg/dL) have indicated associations of blood lead level with lower cognitive test scores in children, later onset of puberty in girls, and increased blood pressure and cardiovascular mortality rates in adults. Cognitive effects are considered by regulatory agencies to be the most sensitive endpoint at low doses. Although ~95% of the body burden of lead is stored in the bones, the adverse effects of lead correlate with the concentration of lead in the blood better than with that in the bones. NASA has found that prolonged exposure to microgravity during spaceflight results in a significant loss of bone minerals, the extent of which varies from individual to individual and from bone to bone, but generally averages about 0.5% per month. During such bone loss, lead that had been stored in bones would be released along with calcium. The effects on the concentration of lead in the blood (PbB) of various concentrations of lead in drinking water (PbW) and of lead released from bones due to accelerated osteoporosis in microgravity, as well as changes in exposure to environmental lead before, during, and after spaceflight were evaluated using a physiologically based pharmacokinetic (PBPK) model that incorporated exposure to environmental lead both on earth and in flight and included temporarily increased rates of osteoporosis during spaceflight.

Garcia, H.↗