Search NASA⌕ Search

SEARCH · Search NASA

Results for “stream networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Ultrafast Bragg coherent diffraction imaging of epitaxial thin films using deep complex-valued neural networks

Abstract Domain wall structures form spontaneously due to epitaxial misfit during thin film growth. Imaging the dynamics of domains and domain walls at ultrafast timescales can provide fundamental clues to features that impact electrical transport in electronic devices. Recently, deep learning based methods showed promising phase retrieval (PR) performance, allowing intensity-only measurements to be transformed into snapshot real space images. While the Fourier imaging model involves complex-valued quantities, most existing deep learning based methods solve the PR problem with real-valued based models, where the connection between amplitude and phase is ignored. To this end, we involve complex numbers operation in the neural network to preserve the amplitude and phase connection. Therefore, we employ the complex-valued neural network for solving the PR problem and evaluate it on Bragg coherent diffraction data streams collected from an epitaxial La 2-x Sr x CuO 4 (LSCO) thin film using an X-ray Free Electron Laser (XFEL). Our proposed complex-valued neural network based approach outperforms the traditional real-valued neural network methods in both supervised and unsupervised learning manner. Phase domains are also observed from the LSCO thin film at an ultrafast timescale using the complex-valued neural network.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

Simple and Scalable Streaming: The GRETA Data Pipeline

The Gamma Ray Energy Tracking Array (GRETA) is a state of the art gamma-ray spectrometer being built at Lawrence Berkeley National Laboratory to be first sited at the Facility for Rare Isotope Beams (FRIB) at Michigan State University. A key design requirement for the spectrometer is to perform gamma-ray tracking in near real time. To meet this requirement we have used an inline, streaming approach to signal processing in the GRETA data acquisition system, using a GPU-equipped computing cluster. The data stream will reach 480 thousand events per second at an aggregate data rate of 4 gigabytes per second at full design capacity. We have been able to simplify the architecture of the streaming system greatly by interfacing the FPGA-based detector electronics with the computing cluster using standard network technology. A set of highperformance software components to implement queuing, flow control, event processing and event building have been developed, all in a streaming environment which matches detector performance. Prototypes of all high-performance components have been completed and meet design specifications.

Cromaz, Mario↗
DOE OSTI2023

Visible Light Communications: A Survey on Recent High-Capacity Demonstrations and Digital Modulation Techniques

In order to deal with the increasing number of mobile devices and with their demand for Internet services, particularly social media platforms, streaming video, and online gaming, Radio-Frequency (RF) wireless networks have been pushed to their capacity limits. In addition to this, 80% of the total data traffic is carried out by users inside buildings. Therefore, new technologies have started to be considered for indoor wireless communications. Visible Light Communications (VLC) can provide both illumination and communications, appearing as an alternative or complement to RF wireless networks. VLC offers high bandwidth and immunity to interference from electromagnetic sources. This manuscript reviews recent high-capacity VLC demonstrations. The main focus of this work is to present digital-signal-processing techniques used in VLC systems. Different modulation formats are analyzed, which can be divided into two large groups, namely single-carrier and multi-carrier modulation schemes. Finally, some recently proposed capacity-achieving strategies are presented. We discuss how to implement these techniques and how they will be useful for the continued development of VLC systems.

Loureiro, Pedro A. (ORCID:0000000283546185)↗
DOE OSTI2020

Contaminant Transport Modeling for Technology Evaluation and Long-Term Monitoring in the Tims Branch Testbed, SC - 20343

Studies conducted at U.S. DOE sites have shown the presence of heavy metals, radionuclides, and volatile organic compounds in surface water, groundwater, and soil as a result of nuclear activity in the Cold War era. Since the 1990's, innovative cleanup methods have been implemented in the Tims Branch watershed at Savannah River Site (SRS) to limit the contaminant flux to the stream that have reduced the contaminant concentrations to acceptable regulatory levels in the dissolved phase. A tin-based treatment which effectively eliminated all local anthropogenic mercury inputs to this ecosystem resulted in a known step function addition of inert tin oxide particles which now serve as a potential tracer for sedimentation and particle transport processes in the stream. The long-term effectiveness of this and other remediation techniques and the potential for remobilization of adsorbed contaminant in sediment during extreme hydrologic conditions however remains unclear. It is therefore important to understand not only the fate and transport of dissolved contaminants, but also the movement of sediment and the relevant interactions with dissolved contaminant. To narrow this knowledge gap, a study is being conducted using the Tims Branch watershed as a stream-scale ecosystem test-bed to identify the primary transport processes of major contaminants of concern (such as mercury, nickel and uranium) with an emphasis on interactions with sediment transport. This involves the development of a fully distributed hydrologic watershed model of the Tims Branch watershed to predict streamflow under extreme weather conditions, as well as the development of a comprehensive contaminant transport model that can properly account for coupled contaminant and sediment transport. Review of relevant research reports and peer-reviewed journals revealed that aside from advection-dispersion transport of dissolved contaminants, adsorption and desorption with suspended solids and bed sediment also play an important role in the transport of those contaminants of concern. To develop the fully distributed hydrologic watershed model, the MIKE SHE 2-dimensional (2D) land surface/3D groundwater model that simulates surface/subsurface hydrologic processes (such as overland flow, evapotranspiration, and infiltration) was coupled with a 1D streamflow model that accounts for stream water hydraulics (such as hydraulic structures, cross-sections, and network). To model contaminant transport, the MIKE 11 streamflow component was coupled with the MIKE 11 AD module that simulates solute transport through advection and dispersion, and MIKE ECO Lab module that accounts for both sediment transport and interactions with dissolved contaminant. At this stage, the development and optimization of the fully distributed hydrologic model has been completed, achieving satisfactory statistical results between observed and predicted discharge as indicated by a root mean square error (RMSE) of 0.039 cms and a Nash-Sutcliffe efficiency coefficient (NSE) of 0.764. The ongoing development of the contaminant transport model has also yielded realistic results from preliminary tests. Results from this study are a key to evaluating the effectiveness of tin (II)-based mercury treatment of wetlands at the SRS site, and are also relevant to evaluating the potential of using this type of novel remediation technology in other mercury-contaminated stream systems. Knowledge acquired from this research will also support interpretation of historical data on the trends of contaminant concentration distribution in Tims Branch, particularly considering the effect of extreme hydrological events on the stream flow and pollutant transport. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

WHONDRS laboratory time series moisture manipulative experiment from soil core layers across eastern contiguous US: time series aerobic respiration, geochemistry, and aggregates

This dataset supports a broader study examining the effects of wetting and drying on soil layers across the eastern contiguous United States (CONUS). The dataset provides data generated from a laboratory moisture manipulation experiment. The contents include time series aerobic respiration and moisture; dissolved oxygen; sediment geochemistry data; and field metadata. Samples were collected as part of a collaboration between WHONDRS (Worldwide Hydrobiogeochemistry Observation Network for Dynamic River Systems; https://whondrs.pnnl.gov) and MONet (Molecular Observation Network; https://www.emsl.pnnl.gov/monet). The field samples (soil cores) were labeled as MEL_##_COR and subsequent subsamples begin with MEL_##. Additional subsamples were taken for the laboratory experiment and were labeled as EL_##. The labels from the MEL field samples and the EL subsamples can be mapped directly based on the digits following the prefix and underscore (i.e., EL_01 is a subsample from MEL_01). See the critical details section below for more details on sample naming and experimental design.For details on how to navigate this data package, see this infographic from the River Corridor SFA https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About.In addition to this readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions.This dataset is comprised of (1) a folder containing environmental context photos; (2) file-level metadata; (3) data dictionary; (4) field metadata; (5) readme; (6) international generic sample number (IGSN) mapping file; and (7) a subfolder with soil sample data from field samples and the incubation experiment. The sample data subfolder contains (1) effect size; (2) gravimetric moisture from field samples and incubation experiment; (3) respiration rates, raw dissolved oxygen values, and plots; (4) specific conductance, pH, and temperature from the incubation; (5) soil aggregates; (6) a summary containing median values of each data type for each treatment (wet and dry) in the incubation; (7) a summary containing averages for each data type of each soil layer; and (8) methods codes. All files are .csv, .pdf, .jpeg, or .jpg.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Global River Topology (GRIT): A Bifurcating River Hydrography

Existing global river networks underpin a wide range of hydrological applications but do not represent channels with divergent river flows (bifurcations, multi‐threaded channels, canals), as these features defy the convergent flow assumption that elevation‐derived networks (e.g., HydroSHEDS, MERIT Hydro) are based on. Yet, bifurcations are important features of the global river drainage system, especially on large floodplains and river deltas, and are also often found in densely populated regions. Here we developed the first raster and vector‐based Global RIver Topology that not only represents the tributaries of the global drainage network but also the distributaries, including multi‐threaded rivers, canals and deltas. We achieve this by merging a 30 m Landsat‐based river mask with elevation‐generated streams to ensure a homogeneous drainage density outside of the river mask for rivers narrower than approximately 30 m. Crucially, we employ the new 30 m digital terrain model, FABDEM, based on TanDEM‐X, which shows greater accuracy over the traditionally used SRTM derivatives. After vectorization and pruning, directionality is assigned by a series of elevation, flow angle and continuity approaches. The new global network and its attributes are validated using gauging stations, comparison with existing networks, and randomized manual checks. The new network represents 19.6 million km of streams and rivers with drainage areas greater than 50 km 2 and includes 67,495 bifurcations. With the advent of hyper‐resolution modeling and artificial intelligence, GRIT is expected to greatly improve the accuracy of many river‐based applications such as flood forecasting, water availability and quality simulations, or riverine habitat mapping.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Decision-Making Framework to Evaluate Opportunities for Recovery of Rare Earth Elements and Critical Minerals in Produced Water Networks

Oil and gas development activities require a significant amount of water. Given the environmental impacts associated with high consumption and subsequent disposal of this water, there is a need for strategies to support effective reuse of, treatment of, and/or resource recovery from this produced water. These streams can contain appreciable concentrations of rare earth elements (REE) and critical minerals (CM) (e.g., Lithium), which are critical for many important applications in electronics, technology, manufacturing, energy, and medicine [1]. Along with REE/CM present in produced water, other waste streams such as fly ash from coal-fired power plants and acid mine discharge present other potential sources for recovery of REE/CM [2-4]. And with increases in demand for REE/CM, the need for recovery from wastewater streams is becoming even more important. There is a need to develop effective decision-making tools to evaluate the economic and environmental potential for REE/CM recovery from produced water networks while supporting the needs of drilling and hydraulic fracturing activities. In this work, we present our progress in developing a decision-making framework to efficiently design and operate produced water networks for REE/CM recovery. Our approach considers REE/CM recovery from produced water streams alongside other potential wastewater sources. The models consider the location and capacity of storage and decentralized treatment processes as well as the design and operation of pipeline and transportation networks to connect different sources and effectively schedule inventories, reuse, and recovery opportunities. We focus on the evaluation of opportunities for REE/CM recovery with treatment constraints on composition and flow. We demonstrate this framework with an illustrative case study that features diverse production sites, storage facilities, transportation, and treatment sites. This proposed framework will be deployed in an open-source software package that is compatible with the PARETO framework to deliver analysis and decision-making tools that allow stakeholders to quantitatively evaluate potential opportunities for REE/CM recovery in produced water networks.

Pulsipher, Joshua↗
DOE OSTI2023

Using Data-Driven Prediction of Downstream 1D River Flow to Overcome the Challenges of Hydrologic River Modeling

Methods for downstream river flow prediction can be categorized into physics-based and empirical approaches. Although based on well-studied physical relationships, physics-based models rely on numerous hydrologic variables characteristic of the specific river system that can be costly to acquire. Moreover, simulation is often computationally intensive. Conversely, empirical models require less information about the system being modeled and can capture a system’s interactions based on a smaller set of observed data. This article introduces two empirical methods to predict downstream hydraulic variables based on observed stream data: a linear programming (LP) model, and a convolutional neural network (CNN). We apply both empirical models within the Colorado River system to a site located on the Green River, downstream of the Yampa River confluence and Flaming Gorge Dam, and compare it to the physics-based model Streamflow Synthesis and Reservoir Regulation (SSARR) currently used by federal agencies. Results show that both proposed models significantly outperform the SSARR model. Moreover, the CNN model outperforms the LP model for hourly predictions whereas both perform similarly for daily predictions. Although less accurate than the CNN model at finer temporal resolution, the LP model is ideal for linear water scheduling tools.

13 HYDRO ENERGY↗
DOE OSTI2021

Modeling the Impact of Riparian Hollows on River Corridor Nitrogen Exports

Recent studies in snowmelt-dominated catchments have documented changes in nitrogen (N) retention over time, such as declines in watershed exports of N, though there is a limited understanding of the controlling processes driving these trends. Working in the mountainous headwater East River Colorado watershed, we explored the effects of riparian hollows as N-cycling hotspots and as important small-scale controls on observed watershed trends. Using a modeling-based approach informed by remote sensing and in situ observations, we simulated the N-retention capacity of riparian hollows with seasonal and yearly hydrobiogeochemical perturbations imposed as drivers. We then implemented a scaling approach to quantify the relative contribution of riparian hollows to the total river corridor N budget. We found that riparian hollows primarily serve as N sinks, with N-transformation rates significantly limited by periods of enhanced groundwater upwelling and promoted at the onset of rainfall events. Given these observed hydrologic controls, we expect that the nitrate ( NO 3 - ) sink capacity of riparian hollows will increase in magnitude with future climatic perturbations, specifically the shift to more frequent rainfall events and fewer snowmelt events, as projected for many mountainous headwater catchments. Our current estimates suggest that while riparian hollows provision ~5–20% of NO 3 - to the river network, they functionally act as inhibitors to upland NO 3 - reaching the stream. Our work linking transient hydrological conditions to numerical biogeochemical simulations is an important step in assessing N-retaining features relative to the watershed N budget and better understanding the role of small-scale features within watersheds.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Stream Temperature Prediction in a Shifting Environment: Explaining the Influence of Deep Learning Architecture

Stream temperature is a fundamental control on ecosystem health. Recent efforts incorporating process guidance into deep learning models for predicting stream temperature have been shown to outperform existing statistical and physical models. This performance is in part because deep learning architectures can actively learn spatiotemporal relationships that govern how water and energy propagate through a river network. However, exploration of how spatiotemporal awareness and process guidance influence a model's generalizability under shifting environmental conditions such as climate change is limited. Here, we use Explainable Artificial Intelligence (XAI) to interrogate how differing deep learning architectures affect a model's learned spatial and temporal dependencies, and how those learned dependencies affect a model's ability to maintain high accuracy when applied to unseen environmental conditions. Using the Delaware River Basin in the northeastern United States as a test case, we compare two spatiotemporally aware process–guided deep learning models for predicting stream temperature (a recurrent graph convolution network—RGCN, and a temporal convolution graph model—Graph WaveNet). Both models achieve equally high predictive performance when testing data are well represented in the training data (test root mean squared errors of 1.64°C and 1.65°C); however, Graph WaveNet significantly outperforms RGCN in 4 out of 5 experiments where test partitions represent different types of unseen environmental conditions. XAI results show that the architecture of Graph WaveNet leads to learned spatial relationships with greater fidelity to physical processes, and that this fidelity improves the generalizability of the model when applied to shifting and/or unseen environmental conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Streaming readout for next generation electron scattering experiments

Current and future experiments at the high-intensity frontier are expected to produce an enormous amount of data that needs to be collected and stored for offline analysis. Thanks to the continuous progress in computing and networking technology, it is now possible to replace the standard ‘triggered’ data acquisition systems with a new, simplified and outperforming scheme. ‘Streaming readout’ (SRO) DAQ aims to replace the hardware-based trigger with a much more powerful and flexible software-based one, that considers the whole detector information for efficient real-time data tagging and selection. Considering the crucial role of DAQ in an experiment, validation with on-field tests is required to demonstrate SRO performance. In this paper, we report results of the on-beam validation of the Jefferson Lab SRO framework. In this work, we exposed different detectors (PbWO-based electromagnetic calorimeters and a plastic scintillator hodoscope) to the Hall-D electron-positron secondary beam and to the Hall-B production electron beam, with increasingly complex experimental conditions. By comparing the data collected with the SRO system against the traditional DAQ, we demonstrate that the SRO performs as expected. Furthermore, we provide evidence of its superiority in implementing sophisticated AI-supported algorithms for real-time data analysis and reconstruction.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Analyzing Data Privacy for Edge Systems

Internet-of-Things (IoT)-based streaming applications are all around us. Currently, we are transitioning from IoT processing being performed on the cloud to the edge. While moving to the edge provides significant networking efficiency benefits, IoT edge computing creates significant data privacy concerns. We propose a methodology that can successfully privacy protect the continual data streams generated by sensors on the edge device. We implement local differential privacy on streaming data and incorporate Bayesian inference and Gaussian process to evaluate the privacy policy. We demonstrate our methodology on a real-world smart meter testbed and identify the optimal privacy protection settings.

Kotevska, Olivera↗
DOE OSTI2022

Machine learning moment closure models for the radiative transfer equation I: Directly learning a gradient based closure

In this paper, we take a data-driven approach and apply machine learning to the moment closure problem for the radiative transfer equation in slab geometry. Instead of learning the unclosed high order moment, we propose to directly learn the gradient of the high order moment using neural networks. This new approach is consistent with the exact closure we derive for the free streaming limit and also provides a natural output normalization. Finally, a variety of benchmark tests, including the variable scattering problem, the Gaussian source problem with both periodic and reflecting boundaries, and the two-material problem, show both good accuracy and generalizability of our machine learning closure model.

Computer science↗
DOE OSTI2022

Probing Electron Beam Induced Transformations on a Single-Defect Level via Automated Scanning Transmission Electron Microscopy

A robust approach for real-time analysis of the scanning transmission electron microscopy (STEM) data streams, based on ensemble learning and iterative training (ELIT) of deep convolutional neural networks, is implemented on an operational microscope, enabling the exploration of the dynamics of specific atomic configurations under electron beam irradiation via an automated experiment in STEM. Combined with beam control, this approach allows studying beam effects on selected atomic groups and chemical bonds in a fully automated mode. Here, in this study, we demonstrate atomically precise engineering of single vacancy lines in transition metal dichalcogenides and the creation and identification of topological defects in graphene. The ELIT-based approach facilitates direct on-the-fly analysis of the STEM data and engenders real-time feedback schemes for probing electron beam chemistry, atomic manipulation, and atom by atom assembly.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

The LAKE model input dataset for three Arctic lakes

This dataset contains meteorological data collected for three Arctic lakes and compiled to satisfy input requirements of the LAKE 2.0 model. The dataset was generated to act as a benchmarking dataset for future model-data inter-comparisons. The LAKE 2.0 model simulates temperatures within the water later and the sedimentary layer of a lake. The LAKE2.0. is an open-source code and available to download via this weblike http://tesla.parallel.ru/Viktor/LAKE/-/wikis/LAKE-model (last visit July 14, 2021). The meteorological data are required to simulate the surface energy balance at the surface of a lake. This dataset includes a compilation of the meteorological data pulled from multiple data streams, including National Oceanic and Atmospheric Administration (NOAA) climate data, Circumarctic Lakes Observation Network (CALON) data, and the United States Geological Survey (USGS) data. The data were compiled for three Arctic lakes: FoxDen (66.55877, -164.45670), Atqasuk (70.452497, -156.951984), and Toolik (68.63150, -149.60740). Each meteorological data is in comma-delimited format (file extension ‘.dat’) and includes eight columns: Temperature [K], Pressure [Pa], longwave downward radiation [W/m2], shortwave downward radiation [W/m2], “U” wind speed [m/s], ”V” wind speed [m/s], humidity [kg/kg], precipitation [m/s]. In addition to the meteorological data file, we included setup and driver files. The Toolik lake is the deepest out of three lakes and has inflowing and outflowing groundwater data. InflowOutflowREADME.txt has more information about inflow and outflow flies. The other two lakes are much shallower and modeled as a closed system (i.e. no water inflow or outflow).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Transportation Hub Infrastructure Expansion: Decision Support Under Uncertainty

The Athena project (www.athena-mobility.org) has worked to investigate the relationship between the Dallas-Fort Worth Airport (DFW) and the greater Dallas area in order to better understand and therefore better inform future decision-making regarding the critical infrastructure that influence mobility between the airport and the city. Through this work, infrastructure related to curbside pickup and drop-off, parking, public transit, and the road network congestion were identified as critical to the operation of the DFW transportation hub. The infrastructure analysis and expansion aspect of the Athena project is focused on the restructuring of the CTA curb as a hierarchical curb and the building or repurposing of parking infrastructure as the interplay between these two areas. Many sources of uncertainty exist that may impact future airport and transportation hub operations, such as passenger volume growth, population demographic changes over time, electric vehicle (EV) adoption rates, and autonomous vehicle (AV) adoption rates. Due to these sources of uncertainty, we have selected for our research a modeling framework that can capture various types of uncertainty and hedge against those uncertainties in the optimization process. We analyze road network and curb congestion, the rise of transportation networking companies, trends in parking usage, existing policies around this infrastructure, airport revenue streams, and other contributing factors to enable infrastructure decision making with less uncertainty. To accomplish this wholistic analysis, we have developed a novel multi-stage, multi-period stochastic optimization model which considers the airport's decisions from 2025-2045 under different possible future macro trajectories and day-to-day variations in operational conditions captured as "annual representation of operations" scenarios with respective probabilities. This model has also been designed to leverage the outputs of various efforts under the Athena project to create a combined decision framework for infrastructure decisions. These various efforts include the route optimization model, the ASPIRES simulation, the mode choice model, and the SUMO traffic simulation. Our computational experiments of this system at scale have resulted in a working version of our infrastructure model which enables the explicit representation and consideration of various sources of uncertainty in the decision process to enable robust, flexible decision-making. This model has been effectively run on NREL's HPC system, Eagle, with large numbers of stochastic scenarios and shows promise as a scalable tool for robust consideration of uncertainties in airport planning. We have tested our model using 30,240 operational circumstances in total, resulting in a problem with more 200 million variables. This model was solved in several different configurations, and a workflow to simulate the performance of the infrastructure model results was developed and deployed. In general, our results indicate that a combination of remote parking, remote curb infrastructure, and dynamic pricing can generate revenue, reduce emissions, accommodate emerging technologies such as AVs and EVs, and manage airport passenger growth over time. We note the success of the proposed strategy depends on the data collection and forecasting abilities of DFW. We have also seen that the AV adoption by TNCs might necessitate larger amounts of remote curb. The results of this work inform strategies for airport infrastructure decision making, as well as demonstrate the value of an adaptable model, but also indicate that there are avenues remaining where further research would be of value.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Energy Materials Chemistry Integrating Theory, Experiment and Data Science (Final Report)

The Energy Materials Chemistry Integrating Theory, Experiment and Data Science (EM-CITED) project is a multidisciplinary research effort focused on accelerating discovery of scientific knowledge via incorporation of data science and artificial intelligence in materials chemistry research. The project aims to advance materials chemistry-aware data science to unify theory and experiment knowledge streams. The work resulted in foundational AI frameworks for materials chemistry – Deep Reasoning Networks (DRNets), Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP), and Material-to-Spectrum (Mat2Spec) prediction – as well as a host of strategies for accelerated scientific discoveries through principled incorporation of data science in computational and experimental research.

36 MATERIALS SCIENCE↗
DOE OSTI2023

ESnet/JLab FPGA Accelerated Transport

To increase the science rate for high data rates/volumes, Thomas Jefferson National Accelerator Facility (JLab) has partnered with Energy Sciences Network (ESnet) to define an edge to data center traffic shaping / steering transport capability featuring data event aware network shaping and forwarding. The keystone of this ESnet+JLab FPGA Accelerated Transport (EJFAT) is the joint development of an AI/ML directed dynamic compute work Load Balancer (LB) of UDP streamed data. The LB is a suite consisting of a Field Programmable Gate Array (FPGA) executing the dynamically configurable, low fixed latency LB data plane featuring real-time packet redirection and high throughput, and a control plane running on the FPGA host computer that monitors network and compute farm telemetry in order to make dynamic AI/ML guided decisions for destination compute host redirection/load balancing and destination resource provisioning. The LB provides for three-tier horizontal scaling across LB suites, core compute hosts, and CPUs within a host. The LB effectively provides seamless integration of edge/core computing to support direct experimental data processing for immediate use by JLab science programs and others such as the EIC as well as data centers of the future requiring high throughput and low latency for both hot and cooled data for both running experiment data acquisition systems and data center use cases.

97 MATHEMATICS AND COMPUTING↗