Search NASA⌕ Search

SEARCH · Search NASA

Results for “temporal learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Newly reconstructed Arctic surface air temperatures for 1979–2021 with deep learning method

A precise Arctic surface air temperature (SAT) dataset, that is regularly updated, has more complete spatial and temporal coverage, and is based on instrumental observations, is critically important for timely monitoring and improving understanding of the rapid change in the Arctic climate. In this study, a new monthly gridded Arctic SAT dataset dated back to 1979 was reconstructed with a deep learning method by combining surface air temperatures from multiple data sources. The source data include the observations from land station of GHCN (Global Historical Climatology Network), ICOADS (International Comprehensive Ocean-Atmosphere Data Set) over the oceans, drifting ice station of Russian NP (North Pole), and buoys of IABP (International Arctic Buoy Programme). The last two are crucial for improving the representation of the in-situ observed temperatures within the Arctic. The newly reconstructed dataset includes monthly Arctic SAT beginning in 1979 and daily Arctic SAT beginning in 2011. This dataset would represent a new improvement in developing observational temperature datasets and can be used for a variety of applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Temporal Cyber Attack Detection.

Rigorous characterization of the performance and generalization ability of cyber defense systems is extremely difficult, making it hard to gauge uncertainty, and thus, confidence. This difficulty largely stems from a lack of labeled attack data that fully explores the potential adversarial space. Currently, performance of cyber defense systems is typically evaluated in a qualitative manner by manually inspecting the results of the system on live data and adjusting as needed. Additionally, machine learning has shown promise in deriving models that automatically learn indicators of compromise that are more robust than analyst-derived detectors. However, to generate these models, most algorithms require large amounts of labeled data (i.e., examples of attacks). Algorithms that do not require annotated data to derive models are similarly at a disadvantage, because labeled data is still necessary when evaluating performance. In this work, we explore the use of temporal generative models to learn cyber attack graph representations and automatically generate data for experimentation and evaluation. Training and evaluating cyber systems and machine learning models requires significant, annotated data, which is typically collected and labeled by hand for one-off experiments. Automatically generating such data helps derive/evaluate detection models and ensures reproducibility of results. Experimentally, we demonstrate the efficacy of generative sequence analysis techniques on learning the structure of attack graphs, based on a realistic example. These derived models can then be used to generate more data. Additionally, we provide a roadmap for future research efforts in this area.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Machine Learning Application for Improving Cloud Detection and Phase Determination Over Sunglint Regions for Geostationary Satellites

Cloud detection and phase determination over sunglint regions has been a challenge, especially for geostationary (GEO) satellites. Sunglint is observed when the sunlight specular reflection is at the same viewing angle of the satellite sensor. This intense reflection in the visible channels (VIS) is often comparable to that from optically thick clouds. It also contaminates the shortwave infrared channels (SWIR). Consequently, VIS and SWIR channels become less useful - or not useful- when they are saturated, hampering the detection of cloudy and clear-sky pixels. Sunglint contamination happens frequently and exists nearly in every daytime GEO full disk satellite images. However, sunglint intensity and region are difficult to model due to variable viewing geometry and ocean surface conditions. Moreover, existing physical models do not meet the accuracy required for operational GEO satellite cloud detection. We developed a machine learning algorithm to improve cloud detection in sunglint conditions for the NASA Langley’s Satellite ClOud and radiation Property retrieval System (SatCORPS). This poster presents our recent progress in the algorithm development, validation and applications. The algorithm is validated using collocated SatCORPS GOES-East and GOES-West cloud products. We demonstrate that the machine learning cloud detection in sunglint regions is superior to the traditional approach by improving temporal consistency between sunglint and non-sunglint conditions.

Machine Learning, Cloud detection, Sunglint, SatCO↗
DOE OSTI2021

Machine learning and artificial intelligence for wildfire prediction

Wildfire ignition, intensity, and spread rates are tightly linked with water cycle extremes. The science of wildfire prediction has traditionally encompassed the use of physical and empirical models to quantify the direction and speed of fire spread, plume injection and fire-aerosol impacts on atmospheric composition, predictions of fire season severity on subseasonal-to-seasonal (S2S) time scales, and assessment of the spatial and temporal patterns of fire risk across landscapes. Together with expanding observation networks, machine learning and artificial intelligence (AI) have the potential to revolutionize the application of such models for fire science, saving lives, protecting critical infrastructure, and providing more accurate estimates of wildfire-climate feedbacks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Gap-filling eddy covariance methane fluxes: Comparison of machine learning model predictions and uncertainties at FLUXNET-CH4 wetlands

Time series of methane fluxes measured by eddy-covariance require gap-filling to estimate annual emissions. Gap-filling methane fluxes is challenging because of high variability and complex responses to multiple drivers. To date, there is no widely established gap-filling standard for methane, with regards both to the best model algorithms and predictors. In this study, we address the need for standardization by synthesizing results of gap-filling methods applied at 17 wetland sites spanning boreal to tropical regions including all major wetlands classes and two rice paddies. We introduce new procedures for: 1) creating realistic artificial gap scenarios, 2) training and evaluating gap-filling models without overstating performance, and 3) predicting half-hourly methane fluxes and annual emissions with robust uncertainty estimates. We tested a conventional method (marginal distribution sampling) and four machine learning algorithms - penalized linear regression, artificial neural networks, random forests, and boosted decision trees - and four predictor sets, including temporal, meteorological, ecosystem carbon and energy flux, and soil predictors. We find that the conventional method can achieve similar median performance to the machine learning models but is worse than the best machine learning models and relatively insensitive to predictor choices. Of the machine learning models, decision tree algorithms performed the best in cross-validation experiments, even with a baseline predictor set, and artificial neural networks showed comparable performance when using all predictors. Soil temperature was frequently the most important predictor whilst water table depth was important at sites with substantial water table fluctuations, highlighting the value of data on soil conditions. Raw gap-filling uncertainties from the machine learning models were underestimated and we propose a method to calibrate uncertainties to observations. Finally, we gap-fill and provide summary evaluation metrics for all 81 sites in the FLUXNET-CH4 community dataset and publicly release the python code for model development, evaluation, and uncertainty estimation.

42 ENGINEERING↗
DOE OSTI2020

Real-time Intrusion Detection for High-bandwidth Research Networks using Unsupervised Deep Learning

Nowadays, data generated by large-scale scientific experiments are on the scale of petabytes per month. These data are transferred through dedicated high-bandwidth networks across distributed collaboration sites for processing, storage, and analysis. Like general purpose networks, research networks experience attacks. However, monitoring traffic in such high-bandwidth network environments is challenging with conventional deep packet inspection (DPI)- based network intrusion detection systems (NIDS). Recently, machine learning (ML) algorithms have shown potential in detecting anomalous patterns with better generalizability and faster speed. Compared to the general purpose networks, traffic movement in research network is less volatile and more likely to benefit from the ML-based anomaly analysis. We present ThunderSecure, an efficient AI-powered system for detecting anomalies in 100G research networks. ThunderSecure implements a high-throughput packet processing infrastructure using multi-cores and GPUs. It takes time-series statistics from a variety of network measurements and uses 2D Convolutional Neural Network (CNN) to explore both spatial and temporal dependencies in network data stream. Patterns of science data traffic are learned with a one-class neural network, which blends an Adversarial Autoencoder (AAE) with Gaussian mixture density estimation. Testing traffic flows exhibiting significant deviations from the learned baseline of normality are marked as anomalies. We trained ThunderSecure on hundreds of billions of science data packets recorded from a 100G research network at Fermi National Accelerator Laboratory. The detection performance was evaluated on traffic captured from the same research network days and weeks after the training with different types of attack flows injected. To the best of our knowledge, this is the first ML work in the area of network anomaly detection that has been validated on such extreme scale datasets. Results show that ThunderSecure can recognize science data traffic that are captured long after the training and make nearly certain detection on those with anomalous flows injected, even in the case when the anomaly-to-normal mixing ratio is 0.1%.

Gong, Qian↗
NASA NTRS2023

Fine particulate concentrations over East Asia derived from aerosols measured by the Advanced Himawari Imager using machine learning

Fine particulate matter with a diameter below 2.5 μm (PM 2.5 ) is deleterious to the cardiovascular and respiratory systems. It is often difficult to assess the effects of PM 2.5 on human health over regions with limited ground monitoring sites, especially in East Asia. As an alternative, we estimated near-surface PM 2.5 concentrations by analyzing Advanced Himawari Imager (AHI) Yonsei Aerosol Retrieval (YAER) products. This study incorporates daytime data for East Asia covering the Korean Peninsula, China, Japan, Southeast Asia, and southern Mongolia. We collocated AHI YAER product pixels with meteorological, land-cover, and other ancillary data for the period from March 2018 to February 2019. To estimate PM 2.5 concentrations over wide areas spanning many countries displaying various relationships between aerosol optical depth and PM 2.5 , monthly models were developed by considering both the spatial and temporal characteristics of ground-based PM 2.5 measurements. Random forest machine learning model estimated ground-level mass concentrations of PM 2.5 ; subsequent 10-fold cross validation (CV) yielded a CV R 2 value of 0.81 and a CV root mean squared error (RMSE) of 12.3 μg m -3 . We investigated the spatial pattern of PM 2.5 concentrations over multiple countries and seasonal variation in PM 2.5 concentrations. Diurnal variation of a severe PM 2.5 event in the Korean Peninsula was investigated as a case study. The model captured the extremely heterogeneous spatial distribution of PM 2.5 concentrations peaked around local noon. To measure the capability of the developed model to estimate PM 2.5 concentrations in areas with few in-situ data, its predictive performance was evaluated using a dataset independent of the training process with an R 2 of 0.60 and RMSE of 8.18 μg m −3 . This study demonstrates the potential for satellite-based PM 2.5 estimation for areas with insufficient measuring stations.

Pm2.5↗
NASA NTRS1991

An architecture for designing fuzzy logic controllers using neural networks

Described here is an architecture for designing fuzzy controllers through a hierarchical process of control rule acquisition and by using special classes of neural network learning techniques. A new method for learning to refine a fuzzy logic controller is introduced. A reinforcement learning technique is used in conjunction with a multi-layer neural network model of a fuzzy controller. The model learns by updating its prediction of the plant's behavior and is related to the Sutton's Temporal Difference (TD) method. The method proposed here has the advantage of using the control knowledge of an experienced operator and fine-tuning it through the process of learning. The approach is applied to a cart-pole balancing system.

Berenji, Hamid R.↗
NASA NTRS2023

A Global Land Cover Training Dataset From 1984 to 2020

State-of-the-art cloud computing platforms such as Google Earth Engine (GEE) enable regional-to-global land cover and land cover change mapping with machine learning algorithms. However, collection of high-quality training data, which is necessary for accurate land cover mapping, remains costly and labor-intensive. To address this need, we created a global database of nearly 2 million training units spanning the period from 1984 to 2020 for seven primary and nine secondary land cover classes. Our training data collection approach leveraged GEE and machine learning algorithms to ensure data quality and biogeographic representation. We sampled the spectral-temporal feature space from Landsat imagery to efficiently allocate training data across global ecoregions and incorporated publicly available and collaborator-provided datasets to our database. To reflect the underlying regional class distribution and post-disturbance landscapes, we strategically augmented the database. We used a machine learning-based cross-validation procedure to remove potentially mis-labeled training units. Our training database is relevant for a wide array of studies such as land cover change, agriculture, forestry, hydrology, urban development, among many others.

Radost Stanimirova↗
DOE OSTI2023

Improving Subsurface Stress Characterization for Carbon Dioxide Storage Projects by Incorporating Machine Learning Techniques

The overall objective of this project is to develop a framework for reliable characterization and prediction of the state of stress in the overburden and underburden (including the basement) in CO 2 storage reservoirs using machine learning and integrated geomechanics and geophysical methods. Specifically, we propose to develop workflow encompassing of technologies and/or methods to predict stress and pressure changes due to CO 2 injection in an active tertiary recovery site and their impacts on subtle fault activation, fractures and occurrence of microseismic events and compare responses to field observations. In this project, we anticipate using dataset from the Farnsworth field Unit (FWU) which is operated by Purdure Petroleum. A novel elastic-waveform VSP inversion technique will be used to estimate high-resolution spatial and temporal changes of elastic moduli in CO 2 storage reservoirs, which will be combined with velocity-stress relationship derived from laboratory tests to obtain subsurface pressure and stress. Clustered microseismic data will be jointly inverted for improved focal mechanisms. Least-squares reverse-time migration of microseismic waveform data will be performed to directly image fracture/fault zones. Additionally, a deep neural network machine learning technique with convolutional and recurrent layers will be used for learning the spectro-temporal structures in microseismic waveforms. The results of this geotechnical data analysis will be integrated to develop a high-resolution 3D mechanical earth model extending from the overburden sealing formations to the underburden including the basement. Mechanical properties will be derived through integration of mechanical logs, tests, available results from chemo-mechanical laboratory tests, and elastic inversion of seismic data using a combination of Bayesian and stochastic methods as well as machine learning technique. Failure features (faults/fractures) will be represented and/or modeled based on seismic and core data analysis. A transient hydrodynamic-geomechanical model will be developed through coupling with the calibrated FWU reservoir simulation model. The full physics coupled model will be used to train a reduced order proxy model using machine learning algorithm for estimating stress which will then be used with appropriate constitutive relationships and forward seismological models to simulate pressure changes and induced microseismicity. An advanced optimization framework will be developed to perform a history match to minimize error between field observations and simulated. The history matched proxy model will be verified against the full-physics equivalent. The field observations that will be used in the coupled model calibration process include pressure/stress inverted from VSP, moment magnitude from microseismic analysis, real time downhole pressure measurements, production and injection data. Parameter sensitivity and uncertainty analysis will be performed to characterize the impact of model parameter uncertainty on stress estimates. The proposed project will have significant impact on future field implementation of the proposed technology. Because the project field site is an ongoing CO 2 EOR development, the value of the new technology will be demonstrated in an operational context and evaluated as a viable risk mitigation strategy. Cost/benefit will be evaluated together with the various commercial incentives for CO 2 sequestration available to oil and gas operators. The extensive available dataset and ongoing data acquisition under the SWP Phase III work plan provides flexibility for investigation of multiple approaches and reduces technical risk.

58 GEOSCIENCES↗
DOE OSTI2024

Using Knowledge-Guided Machine Learning To Assess Patterns of Areal Change in Waterbodies across the Contiguous United States

Lake and reservoir surface areas are an important proxy for freshwater availability. Advancements in machine learning (ML) techniques and increased accessibility of remote sensing data products have enabled the analysis of waterbody surface area dynamics on broad spatial scales. However, interpreting the ML results remains a challenge. While ML provides important tools for identifying patterns, the resultant models do not include mechanisms. Thus, the “black-box” nature of ML techniques often lacks ecological meaning. Using ML, we characterized temporal patterns in lake and reservoir surface area change from 1984 to 2016 for 103,930 waterbodies in the contiguous United States. We then employed knowledge-guided machine learning (KGML) to classify all waterbodies into seven ecologically interpretable groups representing distinct patterns of surface area change over time. Many waterbodies were classified as having “no change” (43%), whereas the remaining 57% of waterbodies fell into other groups representing both linear and nonlinear patterns. This analysis demonstrates the potential of KGML not only for identifying ecologically relevant patterns of change across time but also for unraveling complex processes that underpin those changes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Using Knowledge-Guided Machine Learning To Assess Patterns of Areal Change in Waterbodies across the Contiguous United States

Lake and reservoir surface areas are an important proxy for freshwater availability. Advancements in machine learning (ML) techniques and increased accessibility of remote sensing data products have enabled the analysis of waterbody surface area dynamics on broad spatial scales. However, interpreting the ML results remains a challenge. While ML provides important tools for identifying patterns, the resultant models do not include mechanisms. Thus, the “black-box” nature of ML techniques often lacks ecological meaning. Using ML, we characterized temporal patterns in lake and reservoir surface area change from 1984 to 2016 for 103,930 waterbodies in the contiguous United States. We then employed knowledge-guided machine learning (KGML) to classify all waterbodies into seven ecologically interpretable groups representing distinct patterns of surface area change over time. Many waterbodies were classified as having “no change” (43%), whereas the remaining 57% of waterbodies fell into other groups representing both linear and nonlinear patterns. This analysis demonstrates the potential of KGML not only for identifying ecologically relevant patterns of change across time but also for unraveling complex processes that underpin those changes.

58 GEOSCIENCES↗
NASA NTRSDate not supplied

Communicating Metrics of Land Surface Temperature Variability Using Multi-sensor Machine Learning

Land surface temperature (LST) is a key climate observable used to detect changes in the Earth’s surface energy budget that influence carbon and water cycles. Land surface temperature exhibits strong diurnal variability, which geostationary satellites can observe at scale thanks to their temporal resolution. Due to anthropogenic climate and land use changes, the surface energy balance has been considerably modified and may be described by changes in diurnal temperature range and extremes. Using high performance computing and datasets from the NASA Earth Exchange, we exploit co-located, co-temporal observations from low-earth orbit (LEO) and geostationary (GEO) sensors to develop a deep learning-based method for LEO-to-GEO algorithm emulation. Our model is trained to predict MODIS Terra LST from GOES-16 thermal bands and achieves validation error <2K. Application of the model to unseen times of day (observed by MODIS Aqua) and a new GEO sensor (Himawari-8) observing an unseen spatial domain, demonstrate the generalization of the deep learning model across space, time and spectra. Further, time series clustering approaches are examined with the objective of identifying key indicators of change in diurnal cycling and extremes on a continental scale. Communicating LST variability observed by geostationary satellites can have impacts in multiple disciplines, from understanding of snow, vegetation and soil dynamics, to recognizing trends in heat events relevant to human health.

Kate Duffy↗
DOE OSTI2022

Enhanced deep neural networks with transfer learning for distribution LMP considering load and PV uncertainties

As the flexibility of generation and demand increases in distribution systems, the residential loads are emerging as a promising means to participate in demand response and the transactive energy market. Market pricing is an instrumental mechanism for the distribution system operator to exploit the full potential of the flexible resources. The distribution locational marginal price (DLMP) can be used to guide the residential load consumption. This type of market signal helps the distribution system operator to optimize the scheduling of all resources while satisfying related network constraints through a day-ahead market. However, solving the optimization problem for large-scale systems can be computationally expensive. To address the scalability and practicability limitations of the DLMP framework, a learning-based approach is proposed in this paper to complement the day-ahead distribution market framework. Here, the proposed approach combines long short-term memory and transfer learning to develop deep neural network that can capture the spatial–temporal correlation of the input data. The model can determine the optimal DLMP for each node in a distribution system without the system parameters required to formulate the optimization problem. Testing results on IEEE 33-bus and 123-bus systems show that the proposed approach can generate a comparable DLMP against the optimization solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1991

Parameter estimation in space systems using recurrent neural networks

The identification of time-varying parameters encountered in space systems is addressed, using artificial neural systems. A hybrid feedforward/feedback neural network, namely a recurrent multilayer perception, is used as the model structure in the nonlinear system identification. The feedforward portion of the network architecture provides its well-known interpolation property, while through recurrency and cross-talk, the local information feedback enables representation of temporal variations in the system nonlinearities. The standard back-propagation-learning algorithm is modified and it is used for both the off-line and on-line supervised training of the proposed hybrid network. The performance of recurrent multilayer perceptron networks in identifying parameters of nonlinear dynamic systems is investigated by estimating the mass properties of a representative large spacecraft. The changes in the spacecraft inertia are predicted using a trained neural network, during two configurations corresponding to the early and late stages of the spacecraft on-orbit assembly sequence. The proposed on-line mass properties estimation capability offers encouraging results, though, further research is warranted for training and testing the predictive capabilities of these networks beyond nominal spacecraft operations.

Parlos, Alexander G.↗
DOE OSTI2022

Frame invariant neural network closures for Kraichnan turbulence

Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in developing parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we put forth a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work opens up a possibility of connecting physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models.

42 ENGINEERING↗
DOE OSTI2023

Controls and variability of soil respiration temperature sensitivity across China

Understanding the temperature sensitivity (Q 10 ) of soil respiration is critical for benchmarking the potential intensity of regional and global terrestrial soil carbon fluxes-climate feedbacks. Although field observations have demonstrated the strong spatial heterogeneity of Q 10 , a significant knowledge gap still exists regarding to the factors driving spatial and temporal variabilities of Q 10 at regional scales. Here, therefore, we used a machine learning approach to predict Q 10 from 1994 to 2016 with a spatial resolution of 1 km across China from 515 field observations at 5 cm soil depth using climate, soil and vegetation variables. Predicted Q 10 varied from 1.54 to 4.17, with an area-weighted average of 2.52. There was no significant temporal trend for Q 10 (p = 0.32), but annual vegetation production (indicated by normalized difference vegetation index, NDVI) was positively correlated to it (p < 0.01). Spatially, soil organic carbon (SOC) was the most important driving factor in 62 % of the land area across China, and varied greatly, demonstrating soil controls on the spatial pattern of Q 10 . These findings highlighted different environmental controls on the spatial and temporal pattern of soil respiration Q 10 , which should be considered to improve global biogeochemical models used to predict the spatial and temporal patterns of soil carbon fluxes to ongoing climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A robust and interpretable machine learning approach using multimodal biological data to predict future pathological tau accumulation

The early stages of Alzheimer’s disease (AD) involve interactions between multiple pathophysiological processes. Although these processes are well studied, we still lack robust tools to predict individualised trajectories of disease progression. Here, we employ a robust and interpretable machine learning approach to combine multimodal biological data and predict future pathological tau accumulation. In particular, we use machine learning to quantify interactions between key pathological markers (β-amyloid, medial temporal lobe atrophy, tau and APOE 4) at mildly impaired and asymptomatic stages of AD. Using baseline non-tau markers we derive a prognostic index that: (a) stratifies patients based on future pathological tau accumulation, (b) predicts individualised regional future rate of tau accumulation, and (c) translates predictions from deep phenotyping patient cohorts to cognitively normal individuals. Our results propose a robust approach for fine scale stratification and prognostication with translation impact for clinical trial design targeting the earliest stages of AD.

60 APPLIED LIFE SCIENCES↗