SEARCH · Search NASA

Results for “stream networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments

In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R 2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

FFTSF: Revisiting Sub-Seasonal Streamflow Forecasting with Simple Feedforward Network

Accurate short-to-subseasonal streamflow forecasts are vital for water management, including flood preparedness, drought mitigation, hydropower scheduling, and ecosystem protection. However, extending a forecast beyond a few days remains challenging due to complexity of hydrological processes. While recent self-attention based transformer architectures such as iTransformer have gained traction in time-series forecasting, these models suffer from several critical limitations: (1) significant computational overhead that scales quadratically with sequence length, (2) vulnerability to overfitting on limited hydrological datasets, (3) degraded performance on long-horizon forecasts due to attention decay, and (4) excessive architectural complexity that hampers interpretability and operational deployment. In this study, we propose a simple Feedforward Time Series Forecasting (FFTSF) network that directly addresses these limitations through its lightweight architecture and long-range forecasting capabilities. We evaluate FFTSF across 178 USGS stream gauges spanning diverse climate regimes by forecasting lead times of 1-, 7-, 14-, and 30-days. Our results demonstrate that FFTSF achieves competitive performance at short lead times (NSE of 0.778 for 1-day forecasts) while substantially outperforming complex baselines at longer forecast period, achieving the highest NSE (0.271) at 30-day forecasts with greater robustness and stability. For 30-day forecasts, FFTSF achieves a 71% improvement over NLinear, 57% improvement over DLinear and 12% improvement over the computationally intensive iTransformer while requiring fewer computational resources. Our findings reveal that architectural complexity is not necessary for hydrological forecasting, demonstrating that well-designed simple models can outperform attention mechanisms for subseasonal streamflow forecasting. The computational efficiency and consistent long-range performance of FFTSF make it suitable for water management applications where reliable extended forecasts are essential.

Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗

NASA NTRS2021

Accessible Telemetry Streams using a Zero Trust Architecture for the Flight Operations Directorate

As a result of information technology based work becoming increasingly distributed, unique challenges have been presented within the realm of defined network perimeters, namely with respect to secure access to resources. Historically, and from a simplistic abstract perspective, the common approach has been to adopt the, so-called, moat model whereby a physical network perimeter (or interconnected perimeters) is defined to encapsulate resources behind a boundary protected by a firewall. Users are provisioned access through a virtual private network (VPN) and may be further constrained to resources through specific firewall allow and disallow rulesets. Virtual Private Networks and firewall rulesets lead to common problems, particularly at scale and, as a result, perimeter-less architectures provided over the public internet are increasingly becoming prevalent, particularly with its more popular implementation, the Zero Trust Architecture. We present a proposed implementation of the Zero Trust Architecture with a particular concrete example utilizing a de-perimeterized network that requires authentication and authorization for each action between nodes and does not operate within an implicit trust boundary. It should be noted that this paper is not an attempt at providing comprehensive resolutions for the specific problem space with respect to perimeter based security and is more directed at providing information with regard to our proposed implementation of a Zero Trust Architecture for the Flight Operations Directorate. We direct the reader to our Introduction and Background section for more details on specific documentation and where it can be located as it relates to de-perimeterization and Zero Trust.

Paul Shoemaker↗

DOE OSTI2023

AmeriFlux FLUXNET-1F US-xDL NEON Dead Lake (DELA)

This is the AmeriFlux Management Project (AMP) created FLUXNET-1F version of the carbon flux data for the site US-xDL NEON Dead Lake (DELA). This is the FLUXNET version of the carbon flux data for the site US-xDL NEON Dead Lake (DELA) produced by applying the standard ONEFlux (1F) software. Site Description - The Dead Lake site is just northeast of Demopolis, AL near the Black Warrior River. The site is in Major Land Resource Area (MLRA) 133A–Southern Coastal Plains, near the boundary of MLRA 135A–Alabama and Mississippi Blackland Prairie. Surrounded by woody wetlands and evergreen forest, DELA sits on the flood plain steps and stream terraces of an inside meander of the Black Warrior River in Greene Co., AL.

Network), NEON (National Ecological Observatory↗

DOE OSTI2026

A Brief Survey of Data Streaming Technologies

Streaming data is data that is emitted at variable volumes in a continuous, incremental manner with the goal of low-latency processing often at a different physical location. Network infrastructure is used to facilitate the connection between data sources and sinks, and must be robust to handle the requirements of the workflow. The U.S. Department of Energy Office of Science (DOE SC) a federal agency supporting fundamental scientific research for energy and the Nation’s largest supporter of basic research in the physical sciences. DOE SC has the responsibility for operating $\mathbf{1 0}$ National Laboratories, and 28 scientific user facilities supporting advanced supercomputers, particle accelerators, large x-ray light sources, neutron scattering sources, and other specialized facilities for nanoscience and genomics. This paper investigates the state of streaming data workfows, and details some of the approaches to this challenging problem.

Kissel, Ezra↗

NASA NTRSDate not supplied

Some Aeronautical Communications Experiments

Classically there has existed an asymmetry between the computing and communicating sides of aerospace systems. Over the past few decades, this asymmetry has shifted to favoring communication link technologies, meaning that advancements in available central processing units (CPUs), storage devices, and internal data buses have stagnated. Indeed, the increased emphasis placed on refining subsystem performance such as with antenna bandwidth in phased arrays, amplifier power efficiency, software defined radio (SDR) flexibility and encoding for data compression and error correction has given rise to successful debuts of multi-gigabit-per-second data return from long space-link distances. These accomplishments are easily quantifiable through link budgets and illustrate what is possible, but also reveal the deficiencies in overall communications capabilities. In particular, the ever-accelerating presence of aerospace vehicles gives rise to newer and larger classes of challenges to address the needs of 21st century systems. Furthermore remote sensing and imaging capabilities have far outpaced our ability to transmit their products to the ground, so we are increasingly dependent on pre-processing and downselection to contend with the communications bottleneck. No longer may we depend upon the constrained logistics in delivering end-to-end data delivery through manual reconfigurations, static event scheduling and execution on a per-vehicle basis, for these methods do not scale and therefore must give way to dynamic, networked approaches with an overall systems view in mind. Emerging mission requirements exhibit a trend toward multiple smaller-scale vehicles working together to perform dissimilar observations. Such operations necessitate sensor fusion across a constellation, and where data processing may be distributed throughout a fairly disconnected network whose topology changes over time in non-deterministic manners. Individual communications link performance is still very relevant to deploying an effective communications system, but now must be embedded within a greater architecture of capability to optimally utilize the bandwidth available from each link to generate an ultimate end-to-end quality of service. The deleterious effects of timing uncertainty across the arrangement presents a challenge to measurement synchronization and delivery, so a successful deployed system needs to be tolerant to the delays inherent in time-of-light between elements and digital processing latencies existing at each node. In this presentation we share the flight test results from a high performance Gbps laser communications terminal evaluated with a suite of store and forward capabilities called High-rate Delay Tolerant Networking (HDTN). The communications payload is operated over Lake Erie across a range of configurations including several convergence layers, and is evaluated to determine recovery time after link disruptions, information loss, efficiency and speed. The effectiveness of utilizing a flying laboratory to increase the Technology Readiness Level (TRL) of an integrated system in relevant environments is discussed, as well as the value of conducting aeronautics experiments to retire risk for technology infusion into space missions. Upcoming flight campaigns will be presented, including opportunities to demonstrate secure command and control, data intensive hyperspectral imaging, quantum link characterization, 4k High Definition (HD) video streaming and internetworked space-ground-aero relay operations. These experiments will pave the way for future missions which will depend upon interoperability across disparate government and privately owned networks, involve contention with uncertain and dynamic timing, and require agility to autonomously configure optimal parameters across networks of ever-increasing size and complexity to ensure data delivery. https://www1.grc.nasa.gov/space/scan/acs/tech-studies/dtn/

Daniel Raible↗

DOE OSTI2022

Analysis of the Source Physics Experiment Dry Alluvium Geology Ball Drop Sources

Two balls were dropped approximately 150 m down an open borehole as part of an investigation into the aftershocks from a 51 ton TNT-equivalent chemical explosion. The ball drop sources, a concrete-filled basketball and a bowling ball, were intended to mimic falling debris within an underground explosion borehole and cavity. Falling debris or sloughing of the borehole and cavity walls is thought to be one of the sources of explosion aftershocks. The signals from the ball drop sources were recorded by the same seismic network that monitored the aftershocks of the chemical explosion, so the ball drop recordings were used as templates to search through the continuous stream of aftershocks and detect correlated signals as a proportion of all aftershocks. The search resulted in only four similar event recordings out of the thousands of detected aftershocks. A synthetic test showed that the small, high-frequency ball drop templates were limited to producing highly correlated detections for events occurring in a small region around the impact location at 150 m depth. The high-frequency templates were necessary since the small ball drop signals were only above the noise at frequencies greater than 10 Hz. Unfortunately, the impact location is where the borehole was filled with sand in preparation for the subsequent chemical explosion in the test series, making it 150 m above the explosion created cavity centered at 300 m depth. This puts the ball drop templates 150 m away from the cavity, where many more events from falling debris would be expected. Further synthetic tests confirmed events emanating from the cavity 150 m below the impact site would have correlated with the lower frequency template and been detected had the template been usable below 10 Hz. Therefore, we conclude that the four detected events were most probably falling debris within the open borehole that impacted the floor at 150 m depth just as the ball drop sources had, and that this type of explosion aftershock mechanism comprises less than 1% of all explosion aftershocks.

58 GEOSCIENCES↗

DOE OSTI2023

Vehicle Localization in 3D World Coordinates Using Single Camera at Traffic Intersection

Optimizing traffic control systems at traffic intersections can reduce the network-wide fuel consumption, as well as emissions of conventional fuel-powered vehicles. While traffic signals have been controlled based on predetermined schedules, various adaptive signal control systems have recently been developed using advanced sensors such as cameras, radars, and LiDARs. Among these sensors, cameras can provide a cost-effective way to determine the number, location, type, and speed of the vehicles for better-informed decision-making at traffic intersections. In this research, a new approach for accurately determining vehicle locations near traffic intersections using a single camera is presented. For that purpose, a well-known object detection algorithm called YOLO is used to determine vehicle locations in video images captured by a traffic camera. YOLO draws a bounding box around each detected vehicle, and the vehicle location in the image coordinates is converted to the world coordinates using camera calibration data. During this process, a significant error between the center of a vehicle’s bounding box and the real center of the vehicle in the world coordinates is generated due to the angled view of the vehicles by a camera installed on a traffic light pole. As a means of mitigating this vehicle localization error, two different types of regression models are trained and applied to the centers of the bounding boxes of the camera-detected vehicles. The accuracy of the proposed approach is validated using both static camera images and live-streamed traffic video. Based on the improved vehicle localization, it is expected that more accurate traffic signal control can be made to improve the overall network-wide energy efficiency and traffic flow at traffic intersections.

47 OTHER INSTRUMENTATION↗

NASA NTRS2011

An Analysis of Database Replication Technologies with Regard to Deep Space Network Application Requirements

The Deep Space Network (DSN) has three communication facilities which handle telemetry, commands, and other data relating to spacecraft missions. The network requires these three sites to share data with each other and with the Jet Propulsion Laboratory for processing and distribution. Many database management systems have replication capabilities built in, which means that data updates made at one location will be automatically propagated to other locations. This project examines multiple replication solutions, looking for stability, automation, flexibility, performance, and cost. After comparing these features, Oracle Streams is chosen for closer analysis. Two Streams environments are configured - one with a Master/Slave architecture, in which a single server is the source for all data updates, and the second with a Multi-Master architecture, in which updates originating from any of the servers will be propagated to all of the others. These environments are tested for data type support, conflict resolution, performance, changes to the data structure, and behavior during and after network or server outages. Through this experimentation, it is determined which requirements of the DSN can be met by Oracle Streams and which cannot.

Oracle Database File System↗

DOE OSTI2021

Anomaly Detection, Localization and Classification using Drifting Synchrophasor Data Streams

With ongoing automation and digitization of the electric power system, several Phasor Measurement Units(PMUs) have been deployed for monitoring and control. PMU data can have multiple anomalies, and many of the researchers in the past have concentrated on training machine/deep learning algorithms offline for anomaly detection over PMU data (i.e., not in real time). These machine/deep learning algorithms, when trained offline on a sample rather than a population of the dataset, fail to consider the dynamic behavior of the power grid in real-time, resulting in low accuracy. Considering the dynamic behavior of the power grid (e.g., change in load, generation, distributed energy resources (DERs) switching, network, controls), the definition of data anomalies varies in time and requires online training. A fundamental challenge is to enable online (i.e., real-time) training of machine/deep learning algorithms for anomaly detection over streaming PMU data. While machine/deep learning is often desirable to manage data streams, training a deep learning algorithm over streaming PMU data is nontrivial due to changes in data statistics caused by dynamic streaming data. This paper proposes PMUNET: a novel device-level deep learning-based data-driven approach for anomaly detection, localization, and classification over streaming PMU data, using online learning and multivariate data-drift detection algorithm .Two variants of PMUNET, Dynamic data Change Driven Learning (DCDL) and Continuity Driven Learning (CDL), are proposed and compared. DCDL aims to train the deep learning algorithm whenever the definition of anomaly changes due to the power grid dynamics. On the other hand, CDL continuously trains the deep learning algorithm over the PMU data-stream. The experimental results verify that DCDL outperforms CDL and other efficient anomaly detection methods over multiple events such as faults and load/ generator/capacitor/DERs variations/switching for IEEE 14 and 39 Bus test system as well as real PMU industrial data. The result verifies that DCDL variant of PMUNET improves over existing approach with a gain of 2% - 10% in terms of accuracy, false-positive rate, and false-negative rate.

adversarial deep learning↗

NASA NTRS1981

Spatial reasoning in remotely sensed data

Photointerpreters employ a variety of implicit spatial models to provide interpretations from remotely sensed aerial or satellite imagery. In this paper one application is illustrated: how ridges and valleys can be automatically interpreted from Landsat imagery of a mountainous area, and how a relative elevation terrain model can be constructed from this interpretation. How to examine valleys for the possible presence of streams or rivers is shown, and how a spatial relational model can be set up to make a final interpretation of the river drainage network is explored.

Campbell, J.↗

NASA NTRS2014

Communication System and Method

A communication system for communicating over high-latency, low bandwidth networks includes a communications processor configured to receive a collection of data from a local system, and a transceiver in communication with the communications processor. The transceiver is configured to transmit and receive data over a network according to a plurality of communication parameters. The communications processor is configured to divide the collection of data into a plurality of data streams; assign a priority level to each of the respective data streams, where the priority level reflects the criticality of the respective data stream; and modify a communication parameter of at least one of the plurality of data streams according to the priority of the at least one data stream.

Sanders, Adam M.↗

DOE OSTI2020

My vehicle is a data mine

In this talk we explore how analysis of vehicle data provides information of individual vehicle behaviors, information of other vehicles in the flow of traffic, and insights into the behavior of drivers. Over the last two decades traditional passenger vehicles have been transformed from integrated two-port electrical nodes to cyber-physical systems of communicating computational nodes whose individual state and control variables are shared on a standard controller area network (CAN) bus. As driver assistance systems have crept into vehicles as safety features, driver behaviors can be observed through analysis of the data streams on the CAN bus as these new nodes communicate with one another. The properties of these data streams, as well as architectures and approaches to gather the data, are important to consider when drawing conclusions on the relevance of the data in making decisions at varying levels of the information hierarchy. We will demonstrate several technical challenges associated with these data collection processes, as well as preliminary results that demonstrate application relevance of the data to behavior, traffic, and systems domains.

42 ENGINEERING↗

DOE OSTI2023

Real-Time Monitoring of Fracture Dynamics with a Contrast Agent-Assisted Electromagnetic Method

In collaboration with the Advanced Energy Consortium, our team has previously demonstrated that the placement of electrically active proppants (EAPs) in a hydraulic fracture surveyed by electromagnetic (EM) methods can enhance the imaging of the stimulated reservoir volumes during hydraulic fracturing. That work culminated in constructing a well-characterized EAP-filled fracture anomaly at the Devine field pilot site (DFPS). In subsequent laboratory studies, we observed that the electrical conductivity of our EAP correlates with changes in pressure, salinity, and flow. Thus, we postulated that the EAP could be used as an in-situ sensor for the remote monitoring of these changes in previously EAP-filled fractures. This paper presents our latest field data from the DFPS to demonstrate such correlations at an intermediate pilot scale. We conducted surface-based EM surveys during freshwater (200 ppm) and saltwater (2,500 ppm) slug injections while running surfaced-based EM surveys. Simultaneously, we measured the following: 1) bottomhole pressure and salinity in five monitoring wells; 2) injection rate using high-precision data loggers; 3) distributed acoustic sensors in four monitoring wells; and 4) tiltmeter data on the survey area. We demonstrated that injections into an EAP-filled fracture could be successfully coupled with real-time electric field measurements on the surface, leading to remote monitoring of dynamic changes within the EAP-filled fracture. Furthermore, by comparing the electrical field traces with the bottomhole pressure, flow rate, and salinity, we concluded that the observed electric field in our study is influenced by fracture dilation and flow rate. Salinity effect was observed when saltwater was injected. EM simulations solely based on assumptions of fracture conductivity changes during injection did not reproduce all of the measured electric field magnitudes. Preliminary estimates showed that including streaming potential in our geophysical model may be needed to reduce the simulation mismatch. The methods developed and demonstrated during this study will lead to a better understanding of the extent of fracture networks, formation stress states, fluid leakoff and invasion, characterizations of engineered fracture systems, and other applications where monitoring subsurface flow tracking is deemed important.

02 PETROLEUM↗

NASA NTRS2017

Framework for Processing Citizens Science Data for Applications to NASA Earth Science Missions

Citizen science (or crowdsourcing) has drawn much high-level recent and ongoing interest and support. It is poised to be applied, beyond the by-now fairly familiar use of, e.g., Twitter for natural hazards monitoring, to science research, such as augmenting the validation of NASA earth science mission data. This interest and support is seen in the 2014 National Plan for Civil Earth Observations, the 2015 White House forum on citizen science and crowdsourcing, the ongoing Senate Bill 2013 (Crowdsourcing and Citizen Science Act of 2015), the recent (August 2016) Open Geospatial Consortium (OGC) call for public participation in its newly-established Citizen Science Domain Working Group, and NASA's initiation of a new Citizen Science for Earth Systems Program (along with its first citizen science-focused solicitation for proposals). Over the past several years, we have been exploring the feasibility of extracting from the Twitter data stream useful information for application to NASA precipitation research, with both "passive" and "active" participation by the twitterers. The Twitter database, which recently passed its tenth anniversary, is potentially a rich source of real-time and historical global information for science applications. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. Mining the Twitter stream could augment these validation programs and, potentially, help tune existing algorithms. Our ongoing work, though exploratory, has resulted in key components for processing and managing tweets, including the capabilities to filter the Twitter stream in real time, to extract location information, to filter for exact phrases, and to plot tweet distributions. The key step is to process the "precipitation" tweets to be compatible with satellite-retrieved precipitation data. These key components for processing and managing "precipitation" tweets (and additional ones to be developed) are not limited to precipitation, nor are they limited to the Twitter social medium. Indeed, to maximize the value of our work for NASA earth science programs, these components should be generalized and be part of an overall framework for processing citizen science data for science research. In this paper, we outline such a framework.

earth science satellite data↗

NASA NTRS2012

Global Network of Slow Solar Wind

The streamer belt region surrounding the heliospheric current sheet (HCS) is generally treated as the primary or sole source of the slow solar wind. Synoptic maps of solar wind speed predicted by the Wang-Sheeley-Arge model during selected periods of solar cycle 23, however, show many areas of slow wind displaced from the streamer belt. These areas commonly have the form of an arc that is connected to the streamer belt at both ends. The arcs mark the boundaries between fields emanating from different coronal holes of the same polarity and thus trace the paths of belts of pseudostreamers, i.e., unipolar streamers that form over double arcades and lack current sheets. The arc pattern is consistent with the predicted topological mapping of the narrow open corridor or singular separator line that must connect the holes and, thus, consistent with the separatrix-web model of the slow solar wind. Near solar maximum, pseudostreamer belts stray far from the HCS-associated streamer belt and, together with it, form a global-wide web of slow wind. Recognition of pseudostreamer belts as prominent sources of slow wind provides a new template for understanding solar wind stream structure, especially near solar maximum.

solar wind↗

NASA NTRS2021

The role of analysis error in the convergence of reanalysis production streams in MERRA-2

Due to production time constraints, most reanalyses are produced in multiple parallel streams instead of a single continuous one. These streams cover separate segments of the reanalysis time period with short overlaps to allow reconstruction of the official record. A fundamental assumption justifying this approach is that the streams will be assimilating the same observations during the periods where they overlap, and so will eventually converge to a similar atmospheric state, making discontinuities at stream junctions negligible. This assumption is revisited in this work by examining the impact of analysis error on the differences between MERRA-2 overlapping streams in three historical periods. Comparison results are shown in terms of standard deviations of stream differences as well as the spectral decomposition of the variance of their differences. Residual differences were found at the end of each year of overlap, with larger values observed in the earlier segments of the presatellite era. By drawing parallels with analysis error statistics estimated from the GMAO OSSE system, these differences are shown to reflect the varying constraint of data with the varying observing network, and to further carry the imprint of errors that the data assimilation process is not able to mitigate. As such, they are unlikely to be reduced by longer spinup periods. The ability of data assimilation to ensure continuity in the parallel streams is put into question when the observing system coverage is inadequate or simply when the data assimilation system as a whole is suboptimal.

Amal El Akkraoui↗

DOE OSTI2021

Mitigating Catastrophic Forgetting in Deep Learning in a Streaming Setting Using Historical Summary

Recent advancements in scientific equipment and the adaptation of electronics and the Internet of Things (IoT) in our everyday lives resulted in large and complex data production at a high rate. Making meaningful and timely knowledge discovery at a modest cost from this big data is difficult for computing power and storage limitations. Training deep learning models incrementally in a streaming setting can help us with overcoming these limitations. However, in a well-known phenomenon named catastrophic forgetting, incrementally trained models increasingly perform poorly on the past data. To mitigate catastrophic forgetting in training in a streaming setting, we propose constructing a historical summary over time and use the summary with newly arrived data during incremental training. We propose various data summarization techniques such as random sampling, micro clustering, coreset computation, and Auto Encoders to counteract catastrophic forgetting. We built a pipeline for incremental training with a historical summary for training deep learning models for streaming data. We demonstrate the effectiveness of historical summary in mitigating catastrophic forgetting using three case studies involving three different deep learning applications: an Artificial Neural Network (ANN) for classification task on MNIST dataset, a language model (RNN-LM) on the WikiText2 dataset, and a Convolutional Neural Network (CNN), ResNet50 to classify the ImageNet dataset. Through the training of the models, we observe that catastrophic forgetting is evident in ANN and CNN but not in an RNN. For the first task, our method recovers up to 47.9% lost accuracy due to catastrophic forgetting. For the third task, the historical summary recovers classification accuracy by up to 25%. For the second task, though there is not proof of catastrophic forgetting, the training performance (PPL) improves by up to 26% with historical summary.

Dash, Sajal↗