Search NASA⌕ Search

SEARCH · Search NASA

Results for “Long Short-term memory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Building thermal load prediction through shallow machine learning and deep learning

Building thermal load prediction informs the optimization of cooling plant and thermal energy storage. Physics-based prediction models of building thermal load are constrained by the model and input complexity. In this study, we developed 12 data-driven models (7 shallow learning, 2 deep learning, and 3 heuristic methods) to predict building thermal load and compared shallow machine learning and deep learning. The 12 prediction models were compared with the measured cooling demand. It was found XGBoost (Extreme Gradient Boost) and LSTM (Long Short Term Memory) provided the most accurate load prediction in the shallow and deep learning category, and both outperformed the best baseline model, which uses the previous day's data for prediction. Then, we discussed how the prediction horizon and input uncertainty would influence the load prediction accuracy. Major conclusions are twofold: first, LSTM performs well in short-term prediction (1 h ahead) but not in long term prediction (24 h ahead), because the sequential information becomes less relevant and accordingly not so useful when the prediction horizon is long. Second, the presence of weather forecast uncertainty deteriorates XGBoost's accuracy and favors LSTM, because the sequential information makes the model more robust to input uncertainty. Training the model with the uncertain rather than accurate weather data could enhance the model's robustness. Finally, our findings have two implications for practice. First, LSTM is recommended for short-term load prediction given that weather forecast uncertainty is unavoidable. Second, XGBoost is recommended for long term prediction, and the model should be trained with the presence of input uncertainty.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Short-term culture with IL-2 is beneficial for potent memory chimeric antigen receptor T cell production

Interleukin-2 (IL-2) has been extensively used to boost the body's immune cells, especially T cells. IL-2 is a cytokine that for many years was used to activate and amplify T cells. Due to its potent T cell growth-inducing functions in vitro, for many years, IL-2 was used for the culture and expansion of various T cell products, including tumor-infiltrating lymphocytes (TIL), T cell receptors T cells (TCR T), or genetically engineered cells with chimeric antigen receptors T cells (CAR T). Despite its positive effect on T cell production, the side-effect is not well studied. Here, we reported that long-term culture with IL-2 promotes terminal differentiation and impairs rather than boosts the function of chimeric antigen receptor T cells. However, short-term culture with IL-2 predominantly generates memory CAR T cell favorable for cancer treatment.

60 APPLIED LIFE SCIENCES↗

Short-Term Electric Load Forecasting for a Residential Household in Alaska

Accurate short-term load forecasting at a fine scale is essential for demand response programs, peak shaving, and load-shedding strategies [1]. While traditionally, only aggregate short-term consumption data was available, advanced metering infrastructure (AMI) now provides data at the individual consumer level [1]. There is increasing interest in utilizing this data for short-term load forecasting (from an hour to a few days) to optimize grid operations. Electricity consumption in individual households is highly influenced by residents’ personal behaviors [2]. As a result, unlike aggregate loads, electrical power usage in single households often shows significant volatility, making meter-level load forecasting for individual users particularly challenging [3], [4]. Deep learning methods, with their strong ability to model nonlinear data, have become popular for improving the accuracy of household electricity consumption forecasting [4]. Notably, the Long ShortTerm Memory (LSTM) has attracted significant attention [5], [6].

42 ENGINEERING↗

Evidence for long-term potentiation in phospholipid membranes

Biological supramolecular assemblies, such as phospholipid bilayer membranes, have been used to demonstrate signal processing via short-term synaptic plasticity (STP) in the form of paired pulse facilitation and depression, emulating the brain’s efficiency and flexible cognitive capabilities. However, STP memory in lipid bilayers is volatile and cannot be stored or accessed over relevant periods of time, a key requirement for learning. Using droplet interface bilayers (DIBs) composed of lipids, water and hexadecane, and an electrical stimulation training protocol featuring repetitive sinusoidal voltage cycling, we show that DIBs displaying memcapacitive properties can also exhibit persistent synaptic plasticity in the form of long-term potentiation (LTP) associated with capacitive energy storage in the phospholipid bilayer. The time scales for the physical changes associated with the LTP range between minutes and hours, and are substantially longer than previous STP studies, where stored energy dissipated after only a few seconds. STP behavior is the result of reversible changes in bilayer area and thickness. On the other hand, LTP is the result of additional molecular and structural changes to the zwitterionic lipid headgroups and the dielectric properties of the lipid bilayer that result from the buildup of an increasingly asymmetric charge distribution at the bilayer interfaces.

59 BASIC BIOLOGICAL SCIENCES↗

A sequential DNN based Baseline Energy Prediction Framework with Long term Error Mitigation

In this paper,we aim to present a novel sequential framework containing two deep network architectures for baseline energy prediction, applied to a simulated commercial building dataset. We have observed that this sequential framework helps to improve long term prediction accuracy, there by mitigating prediction error accumulation over time. The proposed framework utilizes convolution layers to extract features from the input data space without changing spatial relations between variables. These features are memorized by tensor train based gated recurrent units for an accurate long-term prediction. Although, architecture comprising the amalgamation of convolution layers and memory cell have shown promising results in domains such as social media analysis, language modeling, video frame prediction and image recognition, this paper extends its scope to the context of energy applications. Furthermore, the addition of tensor train based gated recurrent units is motivated by the necessity of computational time reduction during the training process, there by making this framework more suitable for future in-field deployment. Lastly, results show the current framework outperforms other existing machine learning based methods, in both short-term and long-term prediction categories.

Chakraborty, Indrasis↗

A cross-disciplinary response to improve test activities: The corporate memory capitalization in Ariane4 test domain

The Assembly, Integration, Test, and Validation (AIT/AIV) of the Ariane4 Vehicle Equipment Bay was held at Matra Marconi Space (MMS) site of Toulouse for several years. For this activity, incident interpretation necessitates a great deal of different knowledge. When complex faults occur, particularly those appearing during overall control tests, experts of various domains (EGSE, software, on-board equipment) have to join for investigation sessions. Thus, an assistance tool for the identification of faulty equipment will improve the efficiency of diagnosis and the overall productivity of test activities. As a solution, the Aramiihs laboratory proposed considering the opportunity of a knowledge based system intended to assist the tester in diagnosis. This knowledge based system is, in fact, a short-term achievement of a long-term goal which is the capitalization of corporate memory in the Ariane4 test domain. Aramiihs is a research unit where engineers from MMS and researchers from the IRIT-CNRS cooperate on problems concerning new types of man-system interaction.

Vo, Dinh Phuoc↗

An algebraic method for constructing stable and consistent autoregressive filters

In this paper, we introduce an algebraic method to construct stable and consistent univariate autoregressive (AR) models of low order for filtering and predicting nonlinear turbulent signals with memory depth. By stable, we refer to the classical stability condition for the AR model. By consistent, we refer to the classical consistency constraints of Adams–Bashforth methods of order-two. One attractive feature of this algebraic method is that the model parameters can be obtained without directly knowing any training data set as opposed to many standard, regression-based parameterization methods. It takes only long-time average statistics as inputs. The proposed method provides a discretization time step interval which guarantees the existence of stable and consistent AR model and simultaneously produces the parameters for the AR models. In our numerical examples with two chaotic time series with different characteristics of decaying time scales, we find that the proposed AR models produce significantly more accurate short-term predictive skill and comparable filtering skill relative to the linear regression-based AR models. These encouraging results are robust across wide ranges of discretization times, observation times, and observation noise variances. Finally, we also find that the proposed model produces an improved short-time prediction relative to the linear regression-based AR-models in forecasting a data set that characterizes the variability of the Madden–Julian Oscillation, a dominant tropical atmospheric wave pattern.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Mixed Delay/Nondelay Embeddings Based Neuromorphic Computing with Patterned Nanomagnet Arrays

Patterned nanomagnet arrays (PNAs) have been shown to exhibit a strong geometrically frustrated dipole interaction. Some PNAs have also shown emergent domain wall dynamics. Previous works have demonstrated methods to physically probe these magnetization dynamics of PNAs to realize neuromorphic reservoir systems that exhibit chaotic dynamical behavior and high-dimensional nonlinearity. These PNA reservoir systems from prior works leverage echo state properties and linear/nonlinear short-term memory of component reservoir nodes to map and preserve the dynamical information of the input time-series data into nondelay spatial embeddings. Such mappings enable these PNA reservoir systems to imitate and predict/forecast the input time series data. However, these prior PNA reservoir systems are based solely on the nondelay spatial embeddings obtained at component reservoir nodes. As a result, they require a massive number of component reservoir nodes, or a very large spatial embedding (i.e., high-dimensional spatial embedding) per reservoir node, or both, to achieve acceptable imitation and prediction accuracy. These requirements reduce the practical feasibility of such PNA reservoir systems. To address this shortcoming, we present a mixed delay/nondelay embeddings-based PNA reservoir system. Our system uses a single PNA reservoir node with the ability to obtain a mixture of delay/nondelay embeddings of the dynamical information of the time-series data applied at the input of a single PNA reservoir node. Our analysis shows that when these mixed delay/nondelay embeddings are used to train a perceptron at the output layer, our reservoir system outperforms existing PNA-based reservoir systems for the imitation of NARMA 2, NARMA 5, NARMA 7, and NARMA 10 time series data, and for the short-term and long-term prediction of the Mackey Glass time series data.

Ti, Changpeng↗

Boost of the Bio-memristor Performance for Artificial Electronic Synapses by Surface Reconstruction

Biomaterial-based memristors (bio-memristors) are often adopted to emulate biological synapse functions and applied to construct neural computing networks in brain-inspired chip systems. However, the randomness of conductive filament formation in bio-memristors inhibits their switching performance by causing the dispersion of the device-switching parameters. In this case, a facile porous silk fibroin (p-SF) memristor was obtained through a protein surface reconstruction strategy, in which the size of the hole can be adjusted by the density of hybrid nanoseeds. Here, the porous SF memristors exhibit greatly enhanced electrical characteristics, including uniform I-V cycles, centralized distribution of the switching voltages, and both high and low resistances, compared to devices without pores. The results of three-dimensional (3D) simulations based on classical density functional theory (cDFT) suggest that the reconstructed pores in the SF layers guide the formation and fracture of Ag filaments under an electric field and enhance the overall conductivity by separating Ag + ion and electron diffusion pathways. Ag + ions are predicted to preferentially diffuse through pores, whereas electrons diffuse through the SF network. Interestingly, the device conductance can be bidirectionally modulated gradually by positive and negative voltages, can faithfully simulate short-term and long-term plasticity, and can even realize the triplet-spike-timing-dependent plasticity (triplet-STDP) rule, which can be used for pattern recognition in biological systems. The simulation results reveal that a memristor network of this type has an accuracy of ~95.78% in memory learning and the capability of pattern learning. This work provides a facile technology route to improve the performance of bionic-material memristors.

36 MATERIALS SCIENCE↗

Hybrid stochastic synapses enabled by scaled ferroelectric field-effect transistors

Achieving brain-like density and performance in neuromorphic computers necessitates scaling down the size of nanodevices emulating neuro-synaptic functionalities. However, scaling nanodevices results in reduction of programming resolution and emergence of stochastic non-idealities. While prior work has mainly focused on binary transitions, in this work, we leverage the stochastic switching of a three-state ferroelectric field-effect transistor to implement a long-term and short-term two-tier stochastic synaptic memory with a single device. Experimental measurements are performed on a scaled 28 nm high-k metal gate technology-based device to develop a probabilistic model of the hybrid stochastic synapse. In addition to the advantage of ultra-low programming energies afforded by scaling, our hardware–algorithm co-design analysis reveals the efficacy of the two-tier memory in comparison to binary stochastic synapses in on-chip learning tasks—paving the way for algorithms exploiting multi-state devices with probabilistic transitions beyond deterministic ones.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Wattile: Probabilistic Deep Learning-based Forecasting of Building Energy Consumption [SWR-20-94]

Accurate energy forecasting is becoming critical due to many reasons: i ) optimal distributed energy resources operations and dispatch, ii) fault detection and diagnostics, and iii) meeting operational energy efficiency targets. Wattile uses deep learning (DL) for the building's short-term load forecasting application. Two specific types of neural networks called, Long Short Term Memory (LSTM) and Sequence-to-Sequence (S2S) models are used to make predictions. Forecasting models are trained using online historical weather and occupancy indicator data streams from the Intelligent Campus Program's data acquisition systems at the National Renewable Energy Laboratory (NREL) for main meters and sub-meters of multiple building types. These models use probabilistic methods to provide quantile-based forecasts in addition to nominal conditional median predictions of electricity consumption.

Frank, Stephen↗

Multi-Frequency Analysis of Simulated versus Observed Variability in Tropospheric Temperature

Studies seeking to identify a human-caused global warming signal generally rely on climate model estimates of the “noise” of intrinsic natural variability. Assessing the reliability of these noise estimates is of critical importance. We evaluate here the statistical significance of differences between climate model and observational natural variability spectra for global-mean mid- to upper-tropospheric temperature (TMT). We use TMT information from satellites and large multimodel ensembles of forced and unforced simulations. Our main goal is to explore the sensitivity of model-versus-data spectral comparisons to a wide range of subjective decisions. These include the choice of satellite and climate model TMT datasets, the method for separating signal and noise, the frequency range considered, and the statistical model used to represent observed natural variability. Of particular interest is the amplitude of the interdecadal noise against which an anthropogenic tropospheric warming signal must be detected. We find that on time scales of 5–20 years, observed TMT variability is (on average) overestimated by the last two generations of climate models participating in the Coupled Model Intercomparison Project. This result is relatively insensitive to different plausible analyst choices, enhancing confidence in previous claims of detectable anthropogenic warming of the troposphere and indicating that these claims may be conservative. A further key finding is that two commonly used statistical models of short-term and long-term memory have deficiencies in their ability to capture the complex shape of observed TMT spectra.

54 ENVIRONMENTAL SCIENCES↗

Heterosynaptic plasticity in biomembrane memristors controlled by pH

Abstract In biology, heterosynaptic plasticity maintains homeostasis in synaptic inputs during associative learning and memory, and initiates long-term changes in synaptic strengths that nonspecifically modulate different synapse types. In bioinspired neuromorphic circuits, heterosynaptic plasticity may be used to extend the functionality of two-terminal, biomimetic memristors. In this article, we explore how changes in the pH of droplet interface bilayer aqueous solutions modulate the memristive responses of a lipid bilayer membrane in the pH range 4.97–7.40. Surprisingly, we did not find conclusive evidence for pH-dependent shifts in the voltage thresholds ( V* ) needed for alamethicin ion channel formation in the membrane. However, we did observe a clear modulation in the dynamics of pore formation with pH in time-dependent, pulsed voltage experiments. Moreover, at the same voltage, lowering the pH resulted in higher steady-state currents because of increased numbers of conductive peptide ion channels in the membrane. This was due to increased partitioning of alamethicin monomers into the membrane at pH 4.97, which is below the pKa (~5.3–5.7) of carboxylate groups on the glutamate residues of the peptide, making the monomers more hydrophobic. Neutralization of the negative charges on these residues, under acidic conditions, increased the concentration of peptide monomers in the membrane, shifting the equilibrium concentrations of peptide aggregate assemblies in the membrane to favor greater numbers of larger, increasingly more conductive pores. It also increased the relaxation time constants for pore formation and decay, and enhanced short-term facilitation and depression of the switching characteristics of the device. Modulating these thresholds globally and independently of alamethicin concentration and applied voltage will enable the assembly of neuromorphic computational circuitry with enhanced functionality. Impact statement We describe how to use pH as a modulatory “interneuron” that changes the voltage-dependent memristance of alamethicin ion channels in lipid bilayers by changing the structure and dynamical properties of the bilayer. Having the ability to independently control the threshold levels for pore conduction from voltage or ion channel concentration enables additional levels of programmability in a neuromorphic system. In this article, we note that barriers to conduction from membrane-bound ion channels can be lowered by reducing solution pH, resulting in higher currents, and enhanced short-term learning behavior in the form of paired-pulse facilitation. Tuning threshold values with environmental variables, such as pH, provide additional training and learning algorithms that can be used to elicit complex functionality within spiking neural networks. Graphical abstract

36 MATERIALS SCIENCE↗

Short-Term Rainfall Prediction Based on Radar Echo Using an Improved Self-Attention PredRNN Deep Learning Model

Accurate short-term precipitation forecast is extremely important for urban flood warning and natural disaster prevention. In this paper, we present an innovative deep learning model named ISA-PredRNN (improved self-attention PredRNN) for precipitation nowcasting based on radar echoes on the basis of the advanced PredRNN-V2. We introduce the self-attention mechanism and the long-term memory state into the model and design a new set of gating mechanisms. To better capture different intensities of precipitation, the loss function with weights was designed. We further train the model using a combination of reverse scheduled sampling and scheduled sampling to learn the long-term dynamics from the radar echo sequences. Experimental results show that the new model (ISA-PredRNN) can effectively extract the spatiotemporal features of radar echo maps and obtain radar echo prediction results with a small gap from the ground truths. From the comparison with the other six models, the new ISA-PredRNN model has the most accurate prediction results with a critical success index (CSI) of 0.7001, 0.5812 and 0.3052 under the radar echo thresholds of 10 dBZ, 20 dBZ and 30 dBZ, respectively.

Wu, Dali (ORCID:0000000231177074)↗

On the design of a postprocessor for a search for extraterrestrial intelligence /SETI/ system

The design of an on-line postprocessor for a search for extraterrestrial intelligence (SETI) system is described. Signal processing tasks of the postprocessor include: (1) analysis of power level, phase coherence, and state of polarization of single-channel signals in a search for significant signals; (2) grouping or aggregation of adjacent channel data, time averaging of data; and (3) the detection of drifting and modulated signals. Control functions include multichannel spectrum analyzer frequency and clock control, system calibration and selfdiagnostic, control of data flow to and from short-term and long-term (archival) memories, and operation of detection subsystems, such as a visual display and a tunable receiver.

Healy, T. J.↗

Analytical design of intelligent machines

The problem of designing 'intelligent machines' to operate in uncertain environments with minimum supervision or interaction with a human operator is examined. The structure of an 'intelligent machine' is defined to be the structure of a Hierarchically Intelligent Control System, composed of three levels hierarchically ordered according to the principle of 'increasing precision with decreasing intelligence', namely: the organizational level, performing general information processing tasks in association with a long-term memory; the coordination level, dealing with specific information processing tasks with a short-term memory; and the control level, which performs the execution of various tasks through hardware using feedback control methods. The behavior of such a machine may be managed by controls with special considerations and its 'intelligence' is directly related to the derivation of a compatible measure that associates the intelligence of the higher levels with the concept of entropy, which is a sufficient analytic measure that unifies the treatment of all the levels of an 'intelligent machine' as the mathematical problem of finding the right sequence of internal decisions and controls for a system structured in the order of intelligence and inverse order of precision such that it minimizes its total entropy. A case study on the automatic maintenance of a nuclear plant illustrates the proposed approach.

Saridis, George N.↗

The Effect of Spaceflight on the Ultrastructure of the Cerebellum

In weightlessness, astronauts and cosmonauts may experience postural illusions as well as motion sickness symptoms known as the space adaptation syndrome. Upon return to Earth, they have irregularities in posture and balance. The adaptation to microgravity and subsequent re-adaptation to Earth occurs over several days. At the cellular level, a process called neuronal plasticity may mediate this adaptation. The term plasticity refers to the flexibility and modifiability in the architecture and functions of the nervous system. In fact, plastic changes are thought to underlie not just behavioral adaptation, but also the more generalized phenomena of learning and memory. The goal of this experiment was to identify some of the structural alterations that occur in the rat brain during the sensory and motor adaptation to microgravity. One brain region where plasticity has been studied extensively is the cerebellar cortex-a structure thought to be critical for motor control, coordination, the timing of movements, and, most relevant to the present experiment, motor learning. Also, there are direct as well as indirect connections between projections from the gravity-sensing otolith organs and several subregions of the cerebellum. We tested the hypothesis that alterations in the ultrastructural (the structure within the cell) architecture of rat cerebellar cortex occur during the early period of adaptation to microgravity, as the cerebellum adapts to the absence of the usual gravitational inputs. The results show ultrastructural evidence for neuronal plasticity in the central nervous system of adult rats after 24 hours of spaceflight. Qualitative studies conducted on tissue from the cerebellar cortex (specifically, the nodulus of the cerebellum) indicate that ultrastructural signs of plasticity are present in the cerebellar zones that receive input from the gravity-sensing organs in the inner ear (the otoliths). These changes are not observed in this region in cagematched ground control animals. The specific changes include the formation of lamellar bodies, profoundly enlarged Purkinje cell mitochondria, the presence of inter-neuronal cellular protrusions in the molecular layer, and signs of degeneration in the distal dendrites of the Purkinje cells. Since these morphologic signs are not apparent in the control animals, they are not likely to be due to caging or tissue processing effects. The particular nature of the structural alterations in the nodulus, most notably the formation of lamellar bodies and the presence of degeneration, further suggests that excitotoxicity (damaging overstimulation of neurons) may play a role in the short-term neural response to spaceflight. These findings suggest a structural basis for the neuronal and synaptic plasticity accompanying the central nervous system response to altered gravity and help identify the cellular bases underlying the vestibular abnormalities experienced by astronauts during periods of adaptation and re-adaptation to different gravitational forces. Also, since the short- and long-term changes in neural structure occurring during such periods of adaptation resemble the neuronal alterations that occur in some neurologic disorders such as stroke, these findings may offer guidance in the development of strategies for rehabilitation and treatment of such disorders.

Holstein, Gay R.↗

Working and strategic memory deficits in schizophrenia

Working memory and its contribution to performance on strategic memory tests in schizophrenia were studied. Patients (n = 18) and control participants (n = 15), all men, received tests of immediate memory (forward digit span), working memory (listening, computation, and backward digit span), and long-term strategic (free recall, temporal order, and self-ordered pointing) and nonstrategic (recognition) memory. Schizophrenia patients performed worse on all tests. Education, verbal intelligence, and immediate memory capacity did not account for deficits in working memory in schizophrenia patients. Reduced working memory capacity accounted for group differences in strategic memory but not in recognition memory. Working memory impairment may be central to the profile of impaired cognitive performance in schizophrenia and is consistent with hypothesized frontal lobe dysfunction associated with this disease. Additional medial-temporal dysfunction may account for the recognition memory deficit.

Memory Disorders/etiology/physiopathology↗