Search NASA⌕ Search

SEARCH · Search NASA

Results for “sequential training method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

50 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Performance of Pulsed Thermal Tomography Imaging with Machine Learning-Based Classification of Defects in Additively Manufactured Structures

Additive manufacturing (AM) is an emerging method for cost-efficient fabrication of complex topology nuclear reactor parts from high-strength corrosion resistance alloys, such as stainless steel and Inconel. AM of metallic structures for nuclear energy applications is currently based on laser powder bed fusion (LPBF) process. Some of the challenges with using LPBF method for nuclear manufacturing include the possibility of introducing pores into metallic structures. Integrity of AM structures needs to be evaluated nondestructively because material flaws could lead to premature failures in high temperature nuclear reactor environment. Currently, there exist limited capabilities to evaluate actual AM structures non-destructively. Pulsed Thermal Tomography Imaging (PTT) provides a capability for non-destructive evaluation (NDE) of subsurface defects in arbitrary size structures. The PTT method is based on recording material surface temperature transients with infrared (IR) camera following thermal pulse delivered on material surface with flash light. The PTT method has advantages for NDE of actual AM structures because the method involves one-sided non-contact measurements and fast processing of large sample areas captured in one image. Following initial qualification of an AM component for deployment in a nuclear reactor, a PTT system can also be used for in-service nondestructive evaluation (NDE) applications. In this report, we describe recent progress in enhancing PTT capabilities in detecting and visualizing microscopic defects in metallic specimens. The thermal tomography (TT) algorithm obtains depth reconstructions of spatial effusivity from the data cube of sequentially recorded surface temperatures. However, interpretation of TT images is non-trivial because of blurring of images with increasing depth. To address this challenge, we have developed a deep learning convolutional neural network (CNN) to classify size and orientation subsurface defects in simulated TT images. CNN is trained on a database of TT images created for a set of simulated metallic structures with elliptical subsurface voids. Test of CNN performance demonstrate the ability to classify radii and angular orientation of subsurface defects in TT images. In addition, we have shown that CNN trained on elliptical defects is capable of classifying irregular-shaped defects obtained from scanning electron microscopy (SEM) of stainless steel sections printed with LPBF.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Physics-informed State-space Neural Networks for transport phenomena

This work introduces Physics -informed State -space neural network Models (PSMs), a novel solution to achieving real-time optimization, flexibility, and fault tolerance in autonomous systems, particularly in transportdominated systems such as chemical, biomedical, and power plants. Traditional data -driven methods fall short due to a lack of physical constraints like mass conservation; PSMs address this issue by training deep neural networks with sensor data and physics -informing using components' Partial Differential Equations (PDEs), resulting in a physics -constrained, end -to -end differentiable forward dynamics model. Further, through two in silico experiments - a heated channel and a cooling system loop - we demonstrate that PSMs offer a more accurate approach than a purely data -driven model. In the former experiment, PSMs demonstrated significantly lower average root -mean -square errors across test datasets compared to a purely data -driven neural network, with reductions of 44 %, 48 %, and 94 % in predicting pressure, velocity, and temperature, respectively. Beyond accuracy, PSMs demonstrate a compelling multitask capability, making them highly versatile. In this work, we showcase two: supervisory control of a nonlinear system through a sequentially updated state -space representation and the proposal of a diagnostic algorithm using residuals from each of the PDEs. The former demonstrates PSMs' ability to handle constant and time -dependent constraints, while the latter illustrates their value in system diagnostics and fault detection.

42 ENGINEERING↗
DOE OSTI2021

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Here, based on challenging speech, neuromorphic speech, and neuromorphic image datasets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

iPNHOT: a knowledge-based approach for identifying protein-nucleic acid interaction hot spots

The interaction between proteins and nucleic acids plays pivotal roles in various biological processes such as transcription, translation, and gene regulation. Hot spots are a small set of residues that contribute most to the binding affinity of a protein-nucleic acid interaction. Compared to the extensive studies of the hot spots on protein-protein interfaces, the hot spot residues within protein-nucleic acids interfaces remain less well-studied, in part because mutagenesis data for protein-nucleic acids interaction are not as abundant as that for protein-protein interactions. In this study, we built a new computational model, iPNHOT, to effectively predict hot spot residues on protein-nucleic acids interfaces. One training data set and an independent test set were collected from dbAMEPNI and some recent literature, respectively. To build our model, we generated 97 different sequential and structural features and used a two-step strategy to select the relevant features. The final model was built based only on 7 features using a support vector machine (SVM). The features include two unique features such as ΔSASsa 1/2 and esp3, which are newly proposed in this study. Based on the cross validation results, our model gave F1 score and AUROC as 0.725 and 0.807 on the subset collected from ProNIT, respectively, compared to 0.407 and 0.670 of mCSM-NA, a state-of-the art model to predict the thermodynamic effects of protein-nucleic acid interaction. The iPNHOT model was further tested on the independent test set, which showed that our model outperformed other methods. Here, by collecting data from a recently published database dbAMEPNI, we proposed a new model, iPNHOT, to predict hotspots on both protein-DNA and protein-RNA interfaces. The results show that our model outperforms the existing state-of-art models. Our model is available for users through a webserver: http://zhulab.ahu.edu.cn/iPNHOT/.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Building thermal load prediction through shallow machine learning and deep learning

Building thermal load prediction informs the optimization of cooling plant and thermal energy storage. Physics-based prediction models of building thermal load are constrained by the model and input complexity. In this study, we developed 12 data-driven models (7 shallow learning, 2 deep learning, and 3 heuristic methods) to predict building thermal load and compared shallow machine learning and deep learning. The 12 prediction models were compared with the measured cooling demand. It was found XGBoost (Extreme Gradient Boost) and LSTM (Long Short Term Memory) provided the most accurate load prediction in the shallow and deep learning category, and both outperformed the best baseline model, which uses the previous day's data for prediction. Then, we discussed how the prediction horizon and input uncertainty would influence the load prediction accuracy. Major conclusions are twofold: first, LSTM performs well in short-term prediction (1 h ahead) but not in long term prediction (24 h ahead), because the sequential information becomes less relevant and accordingly not so useful when the prediction horizon is long. Second, the presence of weather forecast uncertainty deteriorates XGBoost's accuracy and favors LSTM, because the sequential information makes the model more robust to input uncertainty. Training the model with the uncertain rather than accurate weather data could enhance the model's robustness. Finally, our findings have two implications for practice. First, LSTM is recommended for short-term load prediction given that weather forecast uncertainty is unavoidable. Second, XGBoost is recommended for long term prediction, and the model should be trained with the presence of input uncertainty.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗
DOE OSTI2020

Robust Online Sequential RVFLNs for Data Modeling of Dynamic Time-Varying Systems with Application of an Ironmaking Blast Furnace

In a world where the increasing complexity of modern industrial processes brings difficulties for accurate mathematical modeling, taking advantage of data has become an efficient solution to complex dynamic process modeling issue. In this paper, we develop a novel robust online sequential version of random vector functional-link networks (RVFLNs) for data-driven modeling of dynamic time-varying system and applied it in a blast furnace (BF) ironmaking process. First, to overcome the time-varying dynamics of process and to enable the RVFLNs to learn online with avoiding data saturation, an improved online sequential version of RVFLNs (OS-RFVLNs) is first presented by online sequential learning with forgetting factor. This improved OS-RVFLNs algorithm is not only suitable for the real-time and large data transfer situation, but also can adjust the sensitivity of the algorithm to different samples with the help of the introduced forgetting factor. Second, since the output weights of the improved OS-RVFLNs as well as other RVFLNs algorithms are obtained by the least squares approach, a robustness problem may occur when the training dataset is contaminated with various outliers. To solve this problem, a Cauchy distribution weighted M-estimator is introduced to improve the robustness of the improved OS- RVFLNs. For this proposed robust OS-RVFLNs (R-OS- RVFLNs), since the weights of different outlier data are properly determined by the Cauchy distribution function, their corresponding contribution on modeling can be properly distinguished. Thus robust and better modeling results can be achieved. Experiments using actual industrial data of BF ironmaking process and comparative studies have demonstrated that the proposed method produces a better estimation accuracy and stronger robustness than other methods.

Blast furnace (BF), Modelling, Dynamic systems↗
DOE OSTI2024

Sequential Inference of Hospitalization Electronic Health Records Using Probabilistic Models

In the dynamic hospital setting, decision support can be a valuable tool for improving patient outcomes. Data-driven inference of future outcomes is challenging in this dynamic setting, where long sequences such as laboratory tests and medications are updated frequently. This is due in part to heterogeneity of data types and mixed-sequence types contained in variable length sequences. In this work we design a probabilistic unsupervised model for multiple arbitrary-length sequences contained in hospitalization Electronic Health Record (EHR) data. The model uses a latent variable structure and captures complex relationships between medications, diagnoses, laboratory tests, neurological assessments, and medications. It can be trained on original data, without requiring any lossy transformations or time binning. Inference algorithms are derived that use partial data to infer properties of the complete sequences, including their length and presence of specific values. We train this model on data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The results are evaluated against held-out data for predicting the length of sequences and presence of Intensive Care Unit (ICU) in hospitalization bed sequences. Our method outperforms a baseline approach, showing that in these experiments the trained model captures information in the sequences that is informative of their future values.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

An Approach to Bayesian Optimization for Design Feasibility Check on Discontinuous Black-Box Functions

The paper presents a novel approach to applying Bayesian Optimization (BO) in predicting an unknown constraint boundary, also representing the discontinuity of an unknown function, for a feasibility check on the design space, thereby representing a classification tool to discern between a feasible and infeasible region. Bayesian optimization is a low-cost black-box global optimization tool in the Sequential Design Methods where one learns and updates knowledge from prior evaluated designs, and proceeds to the selection of new designs for future evaluation. However, BO is best suited to problems with the assumption of a continuous objective function and does not guarantee true convergence when having a discontinuous design space. This is because of the insufficient knowledge of the BO about the nature of the discontinuity of the unknown true function. In this paper, we have proposed to predict the location of the discontinuity using a BO algorithm on an artificially projected continuous design space from the original discontinuous design space. The proposed approach has been implemented in a thin tube design with the risk of creep-fatigue failure under constant loading of temperature and pressure. The stated risk depends on the location of the designs in terms of safe and unsafe regions, where the discontinuities lie at the transition between those regions; therefore, the discontinuity has also been treated as an unknown creep-fatigue failure constraint. The proposed BO algorithm has been trained to maximize sampling toward the unknown transition region, to act as a high accuracy classifier between safe and unsafe designs with minimal training cost. The converged solution has been validated for different design parameters with classification error rate and function evaluations at an average of <1% and ~150, respectively. Finally, the performance of our proposed approach in terms of training cost and classification accuracy of thin tube design is shown to be better than the existing machine learning (ML) algorithms such as Support Vector Machine (SVM), Random Forest (RF), and Boosting.

Engineering↗
DOE OSTI2025

Modeling the Behavior of Complex Aqueous Electrolytes Using Machine Learning Interatomic Potentials: The Case of Sodium Sulfate

Understanding the structure and thermodynamics of solvated ions is essential for advancing applications in electrochemistry, water treatment, and energy storage. While ab initio molecular dynamics methods are highly accurate, they are limited by short accessible time and length scales whereas classical force fields struggle with accuracy. Herein, we explore the structure and thermodynamics of complex monovalent-divalent ion pairs using Na 2 SO 4 (aq) as a case study by applying a machine learning interatomic potential (MLIP) trained on density functional theory (DFT) data. Our MLIP-based approach reproduces key bulk properties such as density and radial distribution functions of water. We provide the hydration structure of the sodium and sulfate ions in the 0.1–2 M concentration range and the one-dimensional and two-dimensional potentials of mean force for the sodium–sulfate ion pairing at the low concentration limit (0.1 M), which are inaccessible to DFT. At low concentrations, the sulfate ion is strongly solvated, leading to the stabilization of solvent-separated ion pairs over contact ion pairs. Minimum energy pathway analysis revealed that coordinating two sodium ions with a sulfate ion is a multistep process whereby the sodium ions coordinate to the sulfate ion sequentially. Finally, we demonstrate that MLIPs allow the study of solvated ions beyond simple monovalent pairs with DFT-level accuracy in their low concentration limit (0.1 M) via statistically converged properties from ns-long simulations.

anions↗
DOE OSTI2023

Extending Conformal Prediction to Hidden Markov Models with Exact Validity via de Finetti’s Theorem for Markov Chains

Conformal prediction is a widely used method to quantify uncertainty in settings where the data is independent and identically distributed (IID), or more generally, exchangeable. Conformal prediction takes in a pre-trained classifier and a calibration dataset as inputs, and returns a function which maps feature vectors to subsets of classes. The output of the returned function for a new feature vector is guaranteed to contain the true class with a pre-specified confidence. Despite its success and usefulness in IID settings, extending conformal prediction to non-exchangeable (e.g., Markovian) data in a manner that provably preserves all desirable theoretical properties has largely remained an open problem. As a solution, we extend conformal prediction to the setting of a Hidden Markov Model (HMM) with unknown parameters. The key idea behind the proposed method is to partition the non-exchangeable Markovian data from the HMM into exchangeable blocks by exploiting the de Finetti’s Theorem for Markov Chains discovered by Diaconis and Freedman (1980). The permutations of the exchangeable blocks are then viewed as randomizations of the observed Markovian data from the HMM. The proposed method provably retains all desirable theoretical guarantees offered by the classical conformal prediction framework and is general enough to be useful in many sequential prediction problems.

Nettasinghe, Don Buddhika Wijayantha↗
DOE OSTI2024

Prognostic analysis of high-flow nasal cannula therapy and non-invasive ventilation in mild to moderate hypoxemia patients and construction of a machine learning model for 48-h intubation prediction—a retrospective analysis of the MIMIC database

Background This study aims to investigate the clinical outcome between high-flow nasal cannula (HFNC) and non-invasive ventilation (NIV) therapy in mild to moderate hypoxemic patients on the first ICU day and to develop a predictive model of 48-h intubation. Methods The study included adult patients from the MIMIC III and IV databases who first initiated HFNC or NIV therapy due to mild to moderate hypoxemia (100 < PaO2/FiO2 ≤ 300). The 48-h and 30-day intubation rates were compared using cross-sectional and survival analysis. Nine machine learning and six ensemble algorithms were deployed to construct the 48-h intubation predictive models, of which the optimal model was determined by its prediction accuracy. The top 10 risk and protective factors were identified using the Shapley interpretation algorithm. Result A total of 123,042 patients were screened, of which, 673 were from the MIMIC IV database for ventilation therapy comparison (HFNC n = 363, NIV n = 310) and 48-h intubation predictive model construction (training dataset n = 471, internal validation set n = 202) and 408 were from the MIMIC III database for external validation. The NIV group had a lower intubation rate (23.1% vs. 16.1%, p = 0.001), ICU 28-day mortality (18.5% vs. 11.6%, p = 0.014), and in-hospital mortality (19.6% vs. 11.9%, p = 0.007) compared to the HFNC group. Survival analysis showed that the total and 48-h intubation rates were not significantly different. The ensemble AdaBoost decision tree model (internal and external validation set AUROC 0.878, 0.726) had the best predictive accuracy performance. The model Shapley algorithm showed Sequential Organ Failure Assessment (SOFA), acute physiology scores (APSIII), the minimum and maximum lactate value as risk factors for early failure and age, the maximum PaCO 2 and PH value, Glasgow Coma Scale (GCS), the minimum PaO 2 /FiO 2 ratio, and PaO 2 value as protective factors. Conclusion NIV was associated with lower intubation rate and ICU 28-day and in-hospital mortality. Further survival analysis reinforced that the effect of NIV on the intubation rate might partly be attributed to the other impact factors. The ensemble AdaBoost decision tree model may assist clinicians in making clinical decisions, and early organ function support to improve patients’ SOFA, APSIII, GCS, PaCO 2 , PaO 2 , PH, PaO 2 /FiO 2 ratio, and lactate values can reduce the early failure rate and improve patient prognosis.

Fu, Wei↗
DOE OSTI2025

Neural chaos: A spectral stochastic neural operator

Building surrogate models for operators with uncertainty quantification capabilities is essential for many engineering applications where randomness–such as variability in material properties, boundary conditions, and initial conditions–is unavoidable. Polynomial Chaos Expansion (PCE) is widely recognized as a go-to method for constructing stochastic surrogates in both intrusive and non-intrusive ways, and it has recently been used in the context of operator learning. However, its application becomes challenging for complex or high-dimensional processes, as achieving accuracy requires higher-order polynomials, which can increase computational demand and/or the risk of overfitting. Furthermore, PCE requires specialized treatments to manage random variables that are not independent, and these treatments may be problem-dependent or may fail with increasing complexity. Here, in this work, we adopt the same formalism as the spectral expansion used in PCE; however, we replace the classical polynomial basis functions with neural network (NN) basis functions to leverage their expressivity. To achieve this, we propose an algorithm that identifies NN-parameterized basis functions in a purely data-driven manner, without any prior assumptions about the joint distribution of the random variables involved, whether independent or dependent, or about their marginal distributions. The proposed algorithm identifies each NN-parameterized basis function sequentially, ensuring they are orthogonal with respect to the data distribution. The basis functions are constructed directly on the joint stochastic variables without requiring a tensor product structure or assuming independence of the random variables. This approach may offer greater flexibility for complex stochastic models, while simplifying implementation compared to the tensor product structures typically used in PCE to handle random vectors. This is particularly advantageous given the current state of open-source packages, where building and training neural networks can be done with just a few lines of code and extensive community support. We demonstrate the effectiveness of the proposed scheme through several numerical examples of varying complexity and provide comparisons with classical PCE.

Polynomial chaos expansion↗
DOE OSTI2025

PDF Entity Annotation Tool (PEAT)

While different text mining approaches – including the use of Artificial Intelligence (AI) and other machine based methods - continue to expand at a rapid pace, the tools used by researchers to create the labeled datasets required for training, modeling, and evaluation remain rudimentary. Labeled datasets contain the target attributes the machine is going to learn; for example, training an algorithm to delineate between images of a car or truck would generally require a set of images with a quantitative description of the underlying features of each vehicle type. Development of labeled textual data that can be used to build natural language machine learning models for scientific literature is not currently integrated into existing manual workflows used by domain experts. Published literature is rich with important information, such as different types of embedded text, plots, and tables that can all be used as inputs to train ML/natural language processing (NLP) models, when extracted and prepared in machine readable formats. Currently, both normalized data extraction of use to domain experts and extraction to support development of ML/NLP models are labor intensive and cumbersome manual processes. Automatic extraction of data and information from formats such as PDFs that are optimized for layout and human readability, not machine readability. The PDF (Portable Document Format) Entity Annotation Tool (PEAT) was developed with the goal of allowing users to annotate publications within their current print format, while also allowing those annotations to be captured in a machine-readable format. One of the main issues with traditional annotation tools is that they require transforming the PDF into plain text to facilitate the annotation process. While doing so lessens the technical challenges of annotating data, the user loses all structure and provenance that was inherent in the underlying PDF. Also, textual data extraction from PDFs can be an error prone process. Challenges include identifying sequential blocks of text and a multitude of document formats (multiple columns, font encodings, etc.). As a result of these challenges, using existing tools for development of NLP/ML models directly from PDFs is difficult because the generated outputs are not interoperable. We created a system that allows annotations to be completed on the original PDF document structure, with no plain text extraction. The result is an application that allows for easier and more accurate annotations. In addition, by including a feature that grants the user the ability to easily create a schema, we have developed a system that can be used to annotate text for different domain-centric schemas of relevance to subject matter experts. Different knowledge domains require distinct schemas and annotation tags to support machine learning.

97 MATHEMATICS AND COMPUTING↗