Development of multilayer perceptron networks for isothermal time temperature transformation prediction of U-Mo-X alloys
Not Available
Johns, Jesse M. (ORCID:0000000346239802)↗
SEARCH · Search NASA
Results for “Multilayer perceptron”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Multi-channel low-cost light spectrum measurement using a multilayer perceptron
Light is one of the most important elements for residential and work spaces, which affects visual performance, comfort, productivity and well-being. The measures that quantify the characteristics of a light source are derived directly from the spectral power distribution (SPD). In addition the SPD is an important factor influencing the quality of a light source. However, measuring light source spectrum with traditional spectrometers is expensive, difficult to adapt to normal spaces, and hard to integrate with other systems. To address these challenges, a low-cost spectrometer was developed using an artificial neural network, with a resolution of 5 nm in the visible spectrum. The reconstructed SPD has an error lower than 2% and allows the derivation of measurements to characterize the colour quality of light sources. Additionally, the device has wireless communication (Bluetooth and Wi-Fi) in real-time, which allows integration into lighting control applications and other Internet of things (IoT) applications.
Botero-Valencia, Juan Sebastián↗
Fast Approximation of the Gauss--Newton Hessian Matrix for the Multilayer Perceptron
Not provided.
Mathematics↗
CMOS-Free Multilayer Perceptron Enabled by Four-Terminal MTJ Device.
Abstract not provided.
Brigner, Wesley H.↗
A mathematical discussion concerning the performance of multilayer perceptron-type artificial neural networks through use of orthogonal bipolar vectors
This work proposes the use of orthogonal bipolar vectors (OBV) as targets for multilayer perceptron (MLP) Artificial neural networks pertaining to the pattern recognition field. Implications involving this proposal are supported by a proper mathematical discussion. This study shows that the use of OBV as targets improves the class identification performance of MLP networks for pattern recognition with more than two classes when compared to the widely used conventional target bipolar vector (CBV). In addition to the overall performance improvement, it is possible to obtain high recognition rates with fewer training epochs, while keeping a feasible generalization with other data. This behavior is related to the Euclidean distance of the points in the output space, which is much higher in the case of OBVs. Therefore, with a greater distance between the points in the output space, the MLP network becomes less prone to associate trained outputs with incorrect targets. The mathematical discussion in this study is based on the backpropagation training algorithm. In addition to the mathematical discussion, results from experiments with real data are presented. These results follow those obtained by mathematical derivations. Real data sets used in the experiments are available from: (a) the Semeion Handwritten Digit of Machine Learning Repository, international repository; (b) Iris Image Database from the Chinese Academy of Sciences—CASIA; (c) Australian Sign Language, signs of Machine Learning Repository, international repository. Experiments were conducted to measure the mean Euclidean distance between the MLP network output subjected to a given input pattern and outputs related to other pattern classifications. Experimental results show that MLP networks trained with OBVs were able to ensure a Euclidean distance between the output and incorrect targets greater than MLP networks trained with CBVs (by up to 4 times for handwritten, by up to 12 times for iris images and by up to 10 times for Auslan signs language). As such, the adoption of OBVs as target vectors for MLPs reduces the computational effort for training the network.
97 MATHEMATICS AND COMPUTING↗
Improvement of parallelization efficiency of batch pattern BP training algorithm using Open MPI
The use of tuned collective's module of Open MPI to improve a parallelization efficiency of parallel batch pattern back propagation training algorithm of a multilayer perceptron is considered in this paper. The multilayer perceptron model and the usual sequential batch pattern training algorithm are theoretically described. An algorithmic description of a parallel version of the batch pattern training method is introduced. The obtained parallelization efficiency results using Open MPI tuned collective's module and MPICH2 are compared. Our results show that (i) Open MPI tuned collective's module outperforms MPICH2 implementation both on SMP computer and computational cluster and (ii) different internal algorithms of MPI_Allreduce() collective operation give better results on different scenarios and different parallel systems. Therefore the properties of the communication network and user application should be taken into account when a specific collective algorithm is used.
97 MATHEMATICS AND COMPUTING↗
Learning-based CO 2 concentration prediction: Application to indoor air quality control using demand-controlled ventilation
There have been increasing concerns over the air quality inside buildings as high levels of bio-effluents can cause nausea, dizziness, headaches, and fatigue to the people working in those spaces. First published in 2004 as Standard 62.1, ASHRAE Standard 62.2-2019 requires highly occupied spaces to implement heating, ventilation, and air conditioning (HVAC) that can dilute contaminants produced by occupants. In this regard, occupant-centric ventilation control has been regarded as an effective practice to maintain a satisfactory indoor air quality (IAQ) when dealing with highly variable occupancy environments. However, few established models in current literature and practice consider dynamic occupancy behavior and adaptive IAQ control. To address this gap, a dynamic indoor CO2 model is constructed using machine learning algorithms to forecast CO2concentrations across a range of forecasting horizons. Herein, we tuned and compared six state-of-the-algorithms—including Support Vector Machine, Ada Boost, Random Forest, Gradient Boosting, Logistic Regression, and Multilayer Perceptron. The algorithms’ performances are validated using CO 2 and historical meteorological data collected from a campus classroom with a variable occupancy rate. Simulation results showed that Multilayer Perceptron can strongly predict the volatile CO 2 behavior and also outperforms other algorithms in terms of accuracy. Furthermore, a control strategy capable of modeling and detecting dynamic patterns of CO 2 level is utilized to modulate the ventilation rate in real-time and also reduce the energy consumption. The proposed controller reduced the HVAC fan’s energy consumption by 51.4% and provide ventilation as needed per the ASHRAE standards.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Exploring Randomly Wired Neural Networks for Climate Model Emulation
Exploring the climate impacts of various anthropogenic emissions scenarios is key to making informed decisions for climate change mitigation and adaptation. State-of-the-art Earth system models can provide detailed insight into these impacts but have a large associated computational cost on a per-scenario basis. This large computational burden has driven recent interest in developing cheap machine learning models for the task of climate model emulation. In this paper, we explore the efficacy of randomly wired neural networks for this task. We describe how they can be constructed and compare them with their standard feedforward counterparts using the ClimateBench dataset. Specifically, we replace the serially connected dense layers in multilayer perceptrons, convolutional neural networks, and convolutional long short-term memory networks with randomly wired dense layers and assess the impact on model performance for models with 1 million and 10 million parameters. We find that models with less-complex architectures see the greatest performance improvement with the addition of random wiring (up to 30.4% for multilayer perceptrons). Furthermore, of 24 different model architecture, parameter count, and prediction task combinations, only one had a statistically significant performance deficit in randomly wired networks relative to their standard counterparts, with 14 cases showing statistically significant improvement. We also find no significant difference in prediction speed between networks with standard feedforward dense layers and those with randomly wired layers. These findings indicate that randomly wired neural networks may be suitable direct replacements for traditional dense layers in many standard models.
54 ENVIRONMENTAL SCIENCES↗
Comparison of ANN (MLP), ANFIS, SVM, and RF models for the online classification of heating value of burning municipal solid waste in circulating fluidized bed incinerators
Highlights: • A rapid, cost-effective, and comparative methodology was proposed to evaluate the HVs of burning MSW online. • Models ranked in descending order of performance are ANFIS, RF, SVM, MLP. • A well trained model is capable of characterizing the variation trend of HVs accurately. - Abstract: The heating values, particularly lower heating values of burning municipal solid waste are critically important parameters in operating circulating fluidized bed incineration systems. However, the heating values change widely and frequently, while there is no reliable real-time instrument to measure heating values in the process of incinerating municipal solid waste. A rapid, cost-effective, and comparative methodology was proposed to evaluate the heating values of burning MSW online based on prior knowledge, expert experience, and data-mining techniques. First, selecting the input variables of the model by analyzing the operational mechanism of circulating fluidized bed incinerators, and the corresponding heating value was classified into one of nine fuzzy expressions according to expert advice. Development of prediction models by employing four different nonlinear models was undertaken, including a multilayer perceptron neural network, a support vector machine, an adaptive neuro-fuzzy inference system, and a random forest; a series of optimization schemes were implemented simultaneously in order to improve the performance of each model. Finally, a comprehensive comparison study was carried out to evaluate the performance of the models. Results indicate that the adaptive neuro-fuzzy inference system model outperforms the other three models, with the random forest model performing second-best, and the multilayer perceptron model performing at the worst level. A model with sufficient accuracy would contribute adequately to the control of circulating fluidized bed incinerator operation and provide reliable heating value signals for an automatic combustion control system.
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
Improving Primary Frequency Response in Networked Microgrid Operations using Multilayer Perceptron-Driven Reinforcement Learning
Individual microgrids can improve the reliability of power grids during extreme events, and networked microgrids can further improve efficiency through resource sharing and increase the resilience of critical end-use loads. However, networked microgrid operations are subject to large switching transients, which can cause dynamic instability and lead to system collapse. These transients are especially prevalent in microgrids with high penetrations of inverter-connected renewable energy resources, which do not provide the system inertia needed to mitigate the transients. Existing generator controls can be modified to invoke a drop in terminal voltage in response to a frequency deviation, thereby reducing load and improving frequency response. This paper investigates the use of a reinforcement-learning--based controller trained over several switching transient scenarios to modify generator controls during large frequency deviations. Compared to previously used proportional integral controllers, the proposed controller can improve primary frequency response while adapting to changes in system topologies and events.
Radhakrishnan, Nikitha↗
Development of models for predicting Torsade de Pointes cardiac arrhythmias using perceptron neural networks
Background: Blockage of some ion channels and in particular, the hERG (human Ether-a’-go-go-Related Gene) cardiac potassium channel delays cardiac repolarization and can induce arrhythmia. In some cases it leads to a potentially life-threatening arrhythmia known as Torsade de Pointes (TdP). Therefore recognizing drugs with TdP risk is essential. Candidate drugs that are determined not to cause cardiac ion channel blockage are more likely to pass successfully through clinical phases II and III trials (and preclinical work) and not be withdrawn even later from the marketplace due to cardiotoxic effects. The objective of the present study is to develop an SAR (Structure-Activity Relationship) model that can be used as an early screen for torsadogenic (causing TdP arrhythmias) potential in drug candidates. The method is performed using descriptors comprised of atomic NMR chemical shifts (13C and 15N NMR) and corresponding interatomic distances which are combined into a 3D abstract space matrix. The method is called 3D-SDAR (3-dimensional spectral data-activity relationship) and can be interrogated to identify molecular features responsible for the activity, which can in turn yield simplified hERG toxicophores. A dataset of 55 hERG potassium channel inhibitors collected from Kramer et al. consisting of 32 drugs with TdP risk and 23 with no TdP risk was used for training the 3D-SDAR model. Results: An artificial neural network (ANN) with multilayer perceptron was used to define collinearities among the independent 3D-SDAR features. A composite model from 200 random iterations with 25% of the molecules in each case yielded the following figures of merit: training, 99.2%; internal test sets, 66.7%; external (blind validation) test set, 68. 4%. In the external test set, 70.3% of positive TdP drugs were correctly predicted. Moreover, toxicophores were generated from TdP drugs. Conclusion: A 3D-SDAR was successfully used to build a predictive model for drug-induced torsadogenic and non-torsadogenic drugs based on 55 compounds. The model was tested in 38 external drugs.
59 BASIC BIOLOGICAL SCIENCES↗
Comparative Study of Wind Energy Potential Estimation Methods for Wind Sites in Togo and Benin (West Sub-Saharan Africa)
The characterization of wind speed distribution and the optimal assessment of wind energy potential are critical factors in selecting a suitable site for wind power plants (WPP). The Weibull distribution law has been used extensively to analyze the wind characteristics of candidate WPP sites, and to estimate the available and deliverable energy. This paper presents a comparative study of five wind energy resource assessment methods as they applied to the context of wind sites in West Sub-Saharan Africa. We investigated three numerical approaches, namely, the adaptive neuro-fuzzy inference system (ANFIS), the multilayer perceptron method (MLP), and support vector regression (SVR), to derive the distribution law of wind speeds and to optimally quantify the corresponding wind energy potential. Next, we compared these three approaches to two well-known Weibull distribution law-based methods: the empirical method of Justus (EMJ) and the maximum likelihood method (MLM). Case study results indicated that the neural network-based methods, ANFIS and MLP, yielded the most accurate distribution fits and wind energy potential estimates, and consequently, are the most recommended methods for the wind sites in Togo and Benin. The orders of magnitude of the root mean squared error (RMSE) in estimating the recoverable energy using ANFIS were, respectively, 10-4 and 10-5 for Lomé and Cotonou, while MLP achieved an RMSE order of magnitude of 10-3 for both sites.
17 WIND ENERGY↗
From atomistic models to machine learning: Predictive design of nanocarbons under extreme conditions
The formation of technologically valuable nanocarbon structures under extreme conditions, such as those produced during high-explosive detonations, remains poorly understood but holds significant potential for the development of controlled synthesis pathways. While detonation shockwaves provide the high-pressure, high-temperature environment required for nanodiamond formation, subsequent cooling and decompression dictate whether the diamond phase is preserved or transformed into other nanocarbon structures. Here, in this study, we employ GPU-accelerated reactive molecular dynamics (ReaxFF) simulations to investigate the graphitization and structural remodeling of detonation nanodiamond under nonlinear quench and pressure-release trajectories. We further investigate how the initial nanodiamond morphology; cuboctahedral, octahedral, or hexagonal prism influences the resulting transformation products. Evolution of nanostructure, allotrope (via simulated x-ray diffraction), carbon hybridization, and ring statistics are tracked during a two-stage quench from 5000 K to 60 GPa. Rapid cooling combined with slow decompression optimizes cubic diamond retention, whereas slow cooling with rapid pressure release promotes surface-to-core graphitization, producing concentric sp 2 -hybridized layers and hollowed inner shells. Octahedral nanodiamonds evolve into carbon nano-onions, initially forming bucky diamonds that progressively transform into fully sp 2 -hybridized structures, while hexagonal prisms preferentially form parallel-stacked graphite layers resembling carbon dots. Transient hexagonal diamond (lonsdaleite) emerges as an interfacial phase, suggesting potential reversibility in the shock-induced graphite-to-diamond transformation pathway transformation route. To extend predictive capabilities, we trained machine learning (ML) regressors on over 10 5 node-hours of molecular dynamics (MD) trajectories. A multilayer perceptron (MLP) model reliably predicts the number of graphitized layers from temperature–pressure trajectories with a coefficient of determination (R 2 ) exceeding 0.90. This high predictive fidelity enables efficient, high-throughput mapping of the synthesis parameter space for optimized graphitization outcomes. Collectively, morphological control combined with optimized quench–decompression conditions promote the selective synthesis of nanocarbon allotropes. This work establishes a data-driven framework for the rational, a priori design of carbon nanomaterials for applications in energy storage, sensing, and biomedicine.
Detonation nanodiamond remodeling↗
Machine Learning-Based Classification of Lignocellulosic Biomass from Pyrolysis-Molecular Beam Mass Spectrometry Data
High-throughput analysis of biomass is necessary to ensure consistent and uniform feedstocks for agricultural and bioenergy applications and is needed to inform genomics and systems biology models. Pyrolysis followed by mass spectrometry such as molecular beam mass spectrometry (py-MBMS) analyses are becoming increasingly popular for the rapid analysis of biomass cell wall composition and typically require the use of different data analysis tools depending on the need and application. Here, the authors report the py-MBMS analysis of several types of lignocellulosic biomass to gain an understanding of spectral patterns and variation with associated biomass composition and use machine learning approaches to classify, differentiate, and predict biomass types on the basis of py-MBMS spectra. Py-MBMS spectra were also corrected for instrumental variance using generalized linear modeling (GLM) based on the use of select ions relative abundances as spike-in controls. Machine learning classification algorithms e.g., random forest, k-nearest neighbor, decision tree, Gaussian Naïve Bayes, gradient boosting, and multilayer perceptron classifiers were used. The k-nearest neighbors (k-NN) classifier generally performed the best for classifications using raw spectral data, and the decision tree classifier performed the worst. After normalization of spectra to account for instrumental variance, all the classifiers had comparable and generally acceptable performance for predicting the biomass types, although the k-NN and decision tree classifiers were not as accurate for prediction of specific sample types. Gaussian Naïve Bayes (GNB) and extreme gradient boosting (XGB) classifiers performed better than the k-NN and the decision tree classifiers for the prediction of biomass mixtures. The data analysis workflow reported here could be applied and extended for comparison of biomass samples of varying types, species, phenotypes, and/or genotypes or subjected to different treatments, environments, etc. to further elucidate the sources of spectral variance, patterns, and to infer compositional information based on spectral analysis, particularly for analysis of data without a priori knowledge of the feedstock composition or identity.
59 BASIC BIOLOGICAL SCIENCES↗
Comparison of Machine Learning-Based Predictive Models of the Nutrient Loads Delivered from the Mississippi/Atchafalaya River Basin to the Gulf of Mexico
Predicting nutrient loads is essential to understanding and managing one of the environmental issues faced by the northern Gulf of Mexico hypoxic zone, which poses a severe threat to the Gulf’s healthy ecosystem and economy. The development of hypoxia in the Gulf of Mexico is strongly associated with the eutrophication process initiated by excessive nutrient loads. Due to the complexities in the excessive nutrient loads to the Gulf of Mexico, it is challenging to understand and predict the underlying temporal variation of nutrient loads. The study was aimed at identifying an optimal predictive machine learning model to capture and predict nonlinear behavior of the nutrient loads delivered from the Mississippi/Atchafalaya River Basin (MARB) to the Gulf of Mexico. For this purpose, monthly nutrient loads (N and P) in tons were collected from US Geological Survey (USGS) monitoring station 07373420 from 1980 to 2020. Machine learning models—including autoregressive integrated moving average (ARIMA), gaussian process regression (GPR), single-layer multilayer perceptron (MLP), and a long short-term memory (LSTM) with the single hidden layer—were developed to predict the monthly nutrient loads, and model performances were evaluated by standard assessment metrics—Root Mean Square Error (RMSE) and Correlation Coefficient (R). The residuals of predictive models were examined by the Durbin–Watson statistic. The results showed that MLP and LSTM persistently achieved better accuracy in predicting monthly TN and TP loads compared to GPR and ARIMA. In addition, GPR models achieved slightly better test RMSE score than ARIMA models while their correlation coefficients are much lower than ARIMA models. Moreover, MLP performed slightly better than LSTM in predicting monthly TP loads while LSTM slightly outperformed for TN loads. Furthermore, it was found that the optimizer and number of inputs didn’t show effects on the LSTM performance while they exhibited impacts on MLP outcomes. This study explores the capability of machine learning models to accurately predict nonlinearly fluctuating nutrient loads delivered to the Gulf of Mexico. Further efforts focus on improving the accuracy of forecasting using hybrid models which combine several machine learning models with superior predictive performance for nutrient fluxes throughout the MARB.
54 ENVIRONMENTAL SCIENCES↗
Estimation of the Surface Fluxes for Heat and Momentum in Unstable Conditions with Machine Learning and Similarity Approaches for the LAFE Data Set
Abstract Measurements of three flux towers operated during the land atmosphere feedback experiment (LAFE) are used to investigate relationships between surface fluxes and variables of the land–atmosphere system. We study these relations by means of two machine learning (ML) techniques: multilayer perceptrons (MLP) and extreme gradient boosting (XGB). We compare their flux derivation performance with Monin–Obukhov similarity theory (MOST) and a similarity relationship using the bulk Richardson number (BRN). The ML approaches outperform MOST and BRN. Best agreement with the observations is achieved for the friction velocity. For the sensible heat flux and even more so for the latent heat flux, MOST and BRN deviate from the observations while MLP and XGB yield more accurate predictions. Using MOST and BRN for latent heat flux, the root mean square errors (RMSE) are 107 Wm $$^{-2}$$ - 2 and 121 Wm $$^{-2}$$ - 2 , respectively, as well as the intercepts of the regression lines are $$\approx 110$$ ≈ 110 Wm $$^{-2}$$ - 2 . For the ML methods, the RMSEs reduce to 31 Wm $$^{-2}$$ - 2 for MLP and 33 Wm $$^{-2}$$ - 2 for XGB as well as the intercepts to just 4 Wm $$^{-2}$$ - 2 for MLP and $$-1$$ - 1 Wm $$^{-2}$$ - 2 for XGB with slopes of the regression lines close to 1, respectively. These results indicate significant deficiencies of MOST and BRN, particularly for the derivation of the latent heat flux. In fact, in contrast to the established theories, feature importance weighting demonstrates that the ML methods base their improved derivations on net radiation, the incoming and outgoing shortwave radiations, the air temperature gradient, and the available water contents, but not on the water vapor gradient. The results imply that further studies of surface fluxes and other turbulent variables with ML techniques provide great promise for deriving advanced flux parameterizations and their implementation in land–atmosphere system models.
54 ENVIRONMENTAL SCIENCES↗
A Study of Physics-Informed Deep Learning for System Fluid Dynamics Closures
The sub-grid-scale (SGS) physics models, or so-called closure relations (CR), are essential in thermal-fluid modeling and simulation codes, ranging from large-eddy simulation to system-level thermo-fluid dynamic problems. While details vary, all thermo-fluid codes solve coupled mass-momentum-energy conservation partial differential equations (PDE). Traditionally, the analytical form of closure models has to be known, and then we calibrate the model using data from a set of applicable experiments. Characteristically, it takes long effort to gain insights and mechanistic understanding needed for developing closure models. Such a model is a compact form of data. The long time required for new model development limits the pace of model application while dealing with new geometries, new coolants or system conditions, particularly in newly designed systems. Alternative to mechanistic and semi-analytical models, it is possible to discover the pattern of closure relations by applying the modern machine learning methods to the body of relevant experimental data. Deep learning (DL) is a relatively new branch in the machine learning family, and a major breakthrough has been achieved since Hinton introduces a fast algorithm to efficiently train a feed-forward neural network (NN) with multilayer perceptrons. Generally, any NN with more than two layers is considered as deep learning, and it contains a large number of hyper-parameters to achieve pattern recognition or regression for complex datasets. Hornik shows that multilayer NN is compatible with universal approximation theorem. There is no constraint for a multilayer NN to capture the properties of any measurable information. This capability makes DL attractive for closure models in a system thermal-hydraulic code. As system geometry and/or operating conditions change, new physical experiments (increasingly sophisticated) are conducted to support evaluation and revision of closure models. Such a so-called data-driven modeling relaxes the structural form of system codes to directly assimilate data instead of hard-wiring the model to pre-calibrated correlations. The objective of this research is to enable cost-effective development of a DL-based closure that reflects the data. The previous work on applying machine learning techniques to fluid dynamics focused on statistical learning with basic NN from direct numerical simulations (DNS) dataset. In the present work, we show that DL-based closure models can cause the fluid calculation to diverge if multilayer NN is used. To address the issue, we investigate conditions by which the DL-based closure models work compatibly, stably, and effectively with PDE-constrained forward prediction problems. Optimally meeting the compatibility conditions constitutes a physics-informed deep learning (PIDL) strategy. We refer to this process as PDE-constrained machine learning or PDE-ML. To develop the technical basis for PDE-ML using DL-based fluid dynamics closure, numerical experiments are performed for manufactured problems of increasing complexity. (authors)
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
WavPool: A New Block for Deep Neural Networks
Modern deep neural networks comprise many operational layers, such as dense or convolutional layers, which are often collected into blocks. In this work, we introduce a new, wavelet-transform-based network architecture that we call the multi-resolution perceptron: by adding a pooling layer, we create a new network block, the WavPool. The first step of the multi-resolution perceptron is transforming the data into its multi-resolution decomposition form by convolving the input data with filters of fixed coefficients but increasing size. Following image processing techniques, we are able to make scale and spatial information simultaneously accessible to the network without increasing the size of the data vector. WavPool outperforms a similar multilayer perceptron while using fewer parameters, and outperforms a comparable convolutional neural network by ~ 10% on relative accuracy on CIFAR-10.
McDermott, Samuel D.↗