Search NASA⌕ Search

SEARCH · Search NASA

Results for “Score function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A Practical Probabilistic Benchmark for AI Weather Models

Since the weather is chaotic, it is necessary to forecast an ensemble of future states. Recently, multiple AI weather models have emerged claiming breakthroughs in deterministic skill. Unfortunately, it is hard to fairly compare ensembles of AI forecasts because variations in ensembling methodology become confounding and the baseline data volume is immense. We address this by scoring lagged initial condition ensembles—whereby an ensemble can be constructed from a library of deterministic hindcasts. This allows the first parameter‐free intercomparison of leading AI weather models' probabilistic skill against an operational baseline. Lagged ensembles of the two leading AI weather models, GraphCast and Pangu, perform similarly even though the former outperforms the latter in deterministic scoring. These results are elaborated upon by sensitivity tests showing that commonly used multiple time‐step loss functions damage ensemble calibration.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Structural aging of human neurons is opposite of the changes in schizophrenia

Human mentality develops with age and is altered in psychiatric disorders, though their underlying mechanism is unknown. In this study, we analyzed nanometer-scale three-dimensional structures of brain tissues of the anterior cingulate cortex from eight schizophrenia and eight control cases. The distribution profiles of neurite curvature of the control cases showed a trend depending on their age, resulting in an age-correlated decrease in the standard deviation of neurite curvature (Pearson’s r = -0.80, p = 0.018). In contrast to the control cases, the schizophrenia cases deviate upward from this correlation, exhibiting a 60% higher neurite curvature compared with the controls (p = 7.8 × 10 -4 ). The neurite curvature also showed a correlation with a hallucination score (Pearson’s r = 0.80, p = 1.8 × 10 -4 ), indicating that neurite structure is relevant to brain function. This report is based on our 3D analysis of human brain tissues over a decade and is unprecedented in terms of the number of cases. We suggest that neurite curvature plays a pivotal role in brain aging and can be used as a hallmark to exploit a novel treatment of schizophrenia.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Decoding Cancer Variants of Unknown Significance for Helicase–Nuclease–RPA Complexes Orchestrating DNA Repair During Transcription and Replication

All tumors have DNA mutations, and a predictive understanding of those mutations could inform clinical treatments. However, 40% of the mutations are variants of unknown significance (VUS), with the challenge being to objectively predict whether a VUS is pathogenic and supports the tumor or whether it is benign. To objectively decode VUS, we mapped cancer sequence data and evolutionary trace (ET) scores onto crystallography and cryo-electron microscopy structures with variant impacts quantitated by evolutionary action (EA) measures. As tumors depend on helicases and nucleases to deal with transcription/replication stress, we targeted helicase–nuclease–RPA complexes: (1) XPB-XPD (within TFIIH), XPF-ERCC1, XPG, and RPA for transcription and nucleotide excision repair pathways and (2) BLM, EXO5, and RPA plus DNA2 for stalled replication fork restart. As validation, EA scoring predicts severe effects for most disease mutations, but disease mutants with low ET scores not only are likely destabilizing but also disrupt sophisticated allosteric mechanisms. For sites of disease mutations and VUS predicted to be severe, we found strong co-localization to ordered regions. Rare discrepancies highlighted the different survival requirements between disease and tumor mutations, as well as the value of examining proteins within complexes. In a genome-wide analysis of 33 cancer types, we found correlation between the number of mutations in each tumor and which pathways or functional processes in which the mutations occur, revealing different mutagenic routes to tumorigenesis. We also found upregulation of ancient genes including BLM, which supports a non-random and concerted cancer process: reversion to a unicellular, proliferation-uncontrolled, status by breaking multicellular constraints on cell division. Together, these genes and global analyses challenge the binary “driver” and “passenger” mutation paradigm, support a gradient impact as revealed by EA scoring from moderate to severe at a single gene level, and indicate reduced regulation as well as activity. The objective quantitative assessment of VUS scoring and gene overexpression in the context of functional interactions and pathways provides insights for biology, oncology, and precision medicine.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

A Pattern Dictionary Method for Anomaly Detection

In this paper, we propose a compression-based anomaly detection method for time series and sequence data using a pattern dictionary. The proposed method is capable of learning complex patterns in a training data sequence, using these learned patterns to detect potentially anomalous patterns in a test data sequence. The proposed pattern dictionary method uses a measure of complexity of the test sequence as an anomaly score that can be used to perform stand-alone anomaly detection. We also show that when combined with a universal source coder, the proposed pattern dictionary yields a powerful atypicality detector that is equally applicable to anomaly detection. The pattern dictionary-based atypicality detector uses an anomaly score defined as the difference between the complexity of the test sequence data encoded by the trained pattern dictionary (typical) encoder and the universal (atypical) encoder, respectively. We consider two complexity measures: the number of parsed phrases in the sequence, and the length of the encoded sequence (codelength). Specializing to a particular type of universal encoder, the Tree-Structured Lempel–Ziv (LZ78), we obtain a novel non-asymptotic upper bound, in terms of the Lambert W function, on the number of distinct phrases resulting from the LZ78 parser. This non-asymptotic bound determines the range of anomaly score. As a concrete application, we illustrate the pattern dictionary framework for constructing a baseline of health against which anomalous deviations can be detected.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1984

Performance Measures for Aircraft Landings as a Function of Aircraft Dynamics

A theory of performance measurement for operator controlled systems is presented. The theory permits synthesis of a system performance measure which scores performance on successive data samples based on the impact of the sampled performance on the overall summary of performance. The development of measures for aircraft carrier landings for the glide path and angle of attack control channels is documented. While developing the performance measures, the measures used previously were evaluated and were found to lack the necessary discrimination capability. The previously used measure, the RMS of deviations from the glide path, can, for instance, provide identical scores for both satisfactory and unsatisfactory flight paths. Two types of performance measures developed for aircraft landings are described. Also, an argument is given for the need to test performance measures prior to their use. A suggested test of the measures is offered.

Connelly, E. M.↗
DOE OSTI2025

Functional protein mining with conformal guarantees

Molecular structure prediction and homology detection offer promising paths to discovering protein function and evolutionary relationships. However, current approaches lack statistical reliability assurances, limiting their practical utility for selecting proteins for further experimental and in-silico characterization. To address this challenge, we introduce a statistically principled approach to protein search leveraging principles from conformal prediction, offering a framework that ensures statistical guarantees with user-specified risk and provides calibrated probabilities (rather than raw ML scores) for any protein search model. Our method (1) lets users select many biologically-relevant loss metrics (i.e. false discovery rate) and assigns reliable functional probabilities for annotating genes of unknown function; (2) achieves state-of-the-art performance in enzyme classification without training new models; and (3) robustly and rapidly pre-filters proteins for computationally intensive structural alignment algorithms. Our framework enhances the reliability of protein homology detection and enables the discovery of uncharacterized proteins with likely desirable functional properties.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Dimensionality reduction using elastic measures

With the recent surge in big data analytics for hyperdimensional data, there is a renewed interest in dimensionality reduction techniques. In order for these methods to improve performance gains and understanding of the underlying data, a proper metric needs to be identified. This step is often overlooked, and metrics are typically chosen without consideration of the underlying geometry of the data. Here, in this paper, we present a method for incorporating elastic metrics into the t-distributed stochastic neighbour embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP). We apply our method to functional data, which is uniquely characterized by rotations, parameterization and scale. If these properties are ignored, they can lead to incorrect analysis and poor classification performance. Through our method, we demonstrate improved performance on shape identification tasks for three benchmark data sets (MPEG-7, Car data set and Plane data set of Thankoor), where we achieve 0.77, 0.95 and 1.00 F1 score, respectively.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2010

Implementation and Comparison of Acoustic Travel-Time Measurement Procedures for the Solar Dynamics Observatory-Helioseismic and Magnetic Imager Time-Distance Helioseismology Pipeline

The Helioseismic and Magnetic Imager (HMI) instrument onboard the Solar Dynamics Observatory (SDO) satellite is designed to produce high-resolution Doppler-velocity maps of oscillations at the solar surface with high temporal cadence. To take advantage of these high-quality oscillation data, a time - distance helioseismology pipeline (Zhao et al., Solar Phys. submitted, 2010) has been implemented at the Joint Science Operations Center (JSOC) at Stanford University. The aim of this pipeline is to generate maps of acoustic travel times from oscillations on the solar surface, and to infer subsurface 3D flow velocities and sound-speed perturbations. The wave travel times are measured from cross-covariances of the observed solar oscillation signals. For implementation into the pipeline we have investigated three different travel-time definitions developed in time - distance helioseismology: a Gabor-wavelet fitting (Kosovichev and Duvall, SCORE'96: Solar Convection and Oscillations and Their Relationship, ASSL, Dordrecht, 241, 1997), a minimization relative to a reference cross-covariance function (Gizon and Birch, Astrophys. J. 571, 966, 2002), and a linearized version of the minimization method (Gizon and Birch, Astrophys. J. 614, 472, 2004). Using Doppler-velocity data from the Michelson Doppler Imager (MDI) instrument onboard SOHO, we tested and compared these definitions for the mean and difference traveltime perturbations measured from reciprocal signals. Although all three procedures return similar travel times in a quiet-Sun region, the method of Gizon and Birch (Astrophys. J. 614, 472, 2004) gives travel times that are significantly different from the others in a magnetic (active) region. Thus, for the pipeline implementation we chose the procedures of Kosovichev and Duvall (SCORE'96: Solar Convection and Oscillations and Their Relationship, ASSL, Dordrecht, 241, 1997) and Gizon and Birch (Astrophys. J. 571, 966, 2002). We investigated the relationships among these three travel-time definitions, their sensitivities to fitting parameters, and estimated the random errors that they produce.

HMI↗
DOE OSTI2025

FL‐ADS: Federated learning anomaly detection system for distributed energy resource networks

Abstract With the ongoing development of Distributed Energy Resources (DER) communication networks, the imperative for strong cybersecurity and data privacy safeguards is increasingly evident. DER networks, which rely on protocols such as Distributed Network Protocol 3 and Modbus, are susceptible to cyberattacks such as data integrity breaches and denial of service due to their inherent security vulnerabilities. This paper introduces an innovative Federated Learning (FL)‐based anomaly detection system designed to enhance the security of DER networks while preserving data privacy. Our models leverage Vertical and Horizontal Federated Learning to enable collaborative learning while preserving data privacy, exchanging only non‐sensitive information, such as model parameters, and maintaining the privacy of DER clients' raw data. The effectiveness of the models is demonstrated through its evaluation on datasets representative of real‐world DER scenarios, showcasing significant improvements in accuracy and F1‐score across all clients compared to the traditional baseline model. Additionally, this work demonstrates a consistent reduction in loss function over multiple FL rounds, further validating its efficacy and offering a robust solution that balances effective anomaly detection with stringent data privacy needs.

Purohit, Shaurya [Iowa State University Ames Iowa ↗
DOE OSTI2023

Hydropower Evaluation Framework for Wildfire Resilient Microgrids

This paper presents a framework for characterizing hydropower plants based on plant and site attributes, assessing their feasibility to function within microgrids during periods of wildfire-induced outages. A set of nine metrics is developed to assign scores to hydropower plants, considering their suitability for forming wildfire-resilient microgrids, their capability to operate within such microgrids, and their simulated performance during wildfire events. A comprehensive database is constructed, comparing hydropower plants according to their suitability and capability metrics. To evaluate the performance score of a specific hydropower plant, namely the Hills Creek plant situated in a wildfire-prone region of Oregon, a case study is conducted, simulating a wildfire scenario. The results of steady-state and dynamic simulations demonstrate that the Hills Creek hydropower plant can effectively provide crucial microgrid services and power nearby communities during a prolonged wildfire-related outages.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS2019

Sensor Analysis, Modeling, and Test for Robust Propulsion System Autonomy

An approach is presented supporting analysis, modeling, and test validation of operational flight instrumentation (OFI) that facilitates critical functions for the Space Launch System (SLS) main propulsion system (MPS). Certain types of OFI sensors were shown to exhibit highly nonlinear and non-gaussian noise characteristics during acceptance testing, motivating the development of advanced modeling and simulation (M&S) capability to support algorithm verification and flight certification. Hardware model and algorithm simulation fidelity was informed by a risk scoring metric; redesign of high-risk algorithms using test-validated sensor models significantly improved their expected performance as evaluated using Monte Carlo acceptance sampling methods. Autonomous functions include closed-loop ullage pressure regulation, pressurant leak detection, and fault isolation for automated safing and crew caution and warning (C&W).

Orr, Jeb S.↗
NASA NTRS1990

Users manual for the Automated Performance Test System (APTS)

The characteristics of and the user information for the Essex Automated Performance Test System (APTS) computer-based portable performance assessment battery are given. The battery was developed to provide a menu of performance test tapping the widest possible variety of human cognitive and motor functions, implemented on a portable computer system suitable for use in both laboratory and field settings for studying the effects of toxic agents and other stressors. The manual gives guidance in selecting, administering and scoring tests from the battery, and reviews the data and studies underlying the development of the battery. Its main emphasis is on the users of the battery - the scientists, researchers and technicians who wish to examine changes in human performance across time or as a function of changes in the conditions under which test data are obtained. First the how to information needed to make decisions about where and how to use the battery is given, followed by the research background supporting the battery development. Further, the development history of the battery focuses largely on the logical framework within which tests were evaluated.

Lane, N. E.↗
DOE OSTI2021

Influence of geometry on columnar to equiaxed transition during electron beam powder bed fusion of IN718

Correlation between spot-melt scan parameters (linear spot-density aka areal energy density), build geometry, and solidification microstructure evolution (columnar vs equiaxed) in a powder bed fusion technology is investigated. It is shown that to maintain the equiaxed solidification microstructure evolution in electron beam powder bed additive manufacturing (AM), the areal energy density per layer needs to be scaled with respect to the 2D cross-sectional area of the layer being melted. Samples with two different cross-sectional areas (40 × 40 mm and 20 × 20 mm) have been fabricated with varying areal energy densities. For a given square cross-section (20 × 20 mm), increasing the areal energy density (4.8 MJ/sq.m to 14.7 MJ/sq.m) transitioned the solidification microstructure from columnar to equiaxed. The observed microstructure data (Electron Back Scattered Diffraction - EBSD) is quantified by calculating the principal component (PC) score using a spatial statistics methodology. The sample with equiaxed grains is found to have a low PC score while the sample with columnar grain had a high PC score. A semi-analytical model is used to simulate the heat transfer and the local solidification conditions as a function of processing parameters (linear spot-density). The result from the heat transfer model is correlated with previously quantified microstructure data. Space-Time analysis of the melt pattern is done and correlated with the observed microstructure. In addition, from the findings, appropriate parameters have been used to additively manufacture a turbine blade with site-specific or hybrid solidification microstructure (traditional fabrication possible via a patented method of localized cold working and heat treatment).

36 MATERIALS SCIENCE↗
DOE OSTI2022

Monitoring and prediction of porosity in laser powder bed fusion using physics-informed meltpool signatures and machine learning

In this work we accomplished the monitoring and prediction of porosity in laser powder bed fusion (LPBF) additive manufacturing process. This objective was realized by extracting physics-informed meltpool signatures from an in-situ dual-wavelength imaging pyrometer, and subsequently, analyzing these signatures via computationally tractable machine learning approaches. Porosity in LPBF occurs despite extensive optimization of processing conditions due to stochastic causes. Hence, it is essential to continually monitor the process with in-situ sensors for detecting and mitigating incipient pore formation. In this work a tall cuboid-shaped part (10 mm × 10 mm × 137 mm, material ATI 718Plus) was built with controlled porosity by varying laser power and scanning speed. This test caused various types of porosity, such as lack-of-fusion and keyhole formation, with varying degrees of severity in the part. The meltpool was continuously monitored using a dual-wavelength imaging pyrometer installed in the machine. Physically intuitive process signatures, such as meltpool length, temperature distribution, and ejecta (spatter) characteristics, were extracted from the meltpool images. Subsequently, relatively simple machine learning models, e.g., K-Nearest Neighbors, were trained to predict both the severity and type of porosity as a function of these physics-informed meltpool signatures. These models resulted in a prediction accuracy exceeding 95% (statistical F1-score). The same analysis was carried out with a complex, black-box deep learning convolutional neural network which directly used the meltpool images instead of physics-informed features. The convolutional neural network produced a comparable F1-score in the range of 89–97%. Finally, these results demonstrate that using pragmatic, physics-informed meltpool signatures within a simple machine learning model is as effective for flaw prediction in LPBF as using a complex and computationally demanding black-box deep learning model.

36 MATERIALS SCIENCE↗
NASA NTRS1996

Visual Vestibular Interaction in the Dynamic Visual Acuity Test during Voluntary Head Rotation

Although intact vestibular function is essential in maintaining spatial orientation, no good screening tests of vestibular function are available to the aviation community. High frequency voluntary head rotation was selected as a vestibular stimulus to isolate the vestibulo-ocular reflex (VOR) from visual influence. A dynamic visual acuity test that incorporates voluntary head rotation was evaluated as a potential vestibular function screening tool. Twenty-seven normal subjects performed voluntary sinusoidal head rotation at frequencies from 0.7-4.0 Hz under three different visual conditions: visually-enhanced VOR, normal VOR, and visually suppressed VOR. Standardized Baily-Lovie chart letters were presented on a computer monitor in front of the subject, who then was asked to read the letters while rotating his head horizontally. The electro-oculogram and dynamic visual acuity score were recorded and analyzed. There were no significant differences in gain or phase shift among three visual conditions in the frequency range of 2.8 to 4.0 Hz. The dynamic visual acuity score shifted less than 0.3 logMAR at frequencies under 2.0 Hz. The dynamic visual acuity test at frequencies a round 2.0 Hz can be recommended for evaluating vestibular function.

VISUAL ACUITY↗
DOE OSTI2022

A deep learning framework for layer-wise porosity prediction in metal powder bed fusion using thermal signatures

Abstract Part quality manufactured by the laser powder bed fusion process is significantly affected by porosity. Existing works of process–property relationships for porosity prediction require many experiments or computationally expensive simulations without considering environmental variations. While efforts that adopt real-time monitoring sensors can only detect porosity after its occurrence rather than predicting it ahead of time. In this study, a novel porosity detection-prediction framework is proposed based on deep learning that predicts porosity in the next layer based on thermal signatures of the previous layers. The proposed framework is validated in terms of its ability to accurately predict lack of fusion porosity using computerized tomography (CT) scans, which achieves a F1-score of 0.75. The framework presented in this work can be effectively applied to quality control in additive manufacturing. As a function of the predicted porosity positions, laser process parameters in the next layer can be adjusted to avoid more part porosity in the future or the existing porosity could be filled. If the predicted part porosity is not acceptable regardless of laser parameters, the building process can be stopped to minimize the loss.

42 ENGINEERING↗
DOE OSTI2024

Efficient learning of accurate surrogates for simulations of complex systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse or dynamic data. Here, we introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data, and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation-of-state model can be reliably autogenerated from expensive calculations using few model evaluations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Efficient Learning of Accurate Surrogates for Simulations of Complex Systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse, or dynamic data. We introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation of state model can be reliably auto-generated from expensive calculations using few model evaluations.

97 MATHEMATICS AND COMPUTING↗