Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Epi Archive: Automated Synthesis of Global Notifiable Disease Data

ObjectiveLANL has built software that automatically collects global notifiable disease data, synthesizes the data, and makes it available to humans and computers within the Biosurveillance Ecosystem (BSVE) as a novel data stream. These data have many applications including improving the prediction and early warning of disease events.IntroductionMost countries do not report national notifiable disease data in a machine-readable format. Data are often in the form of a file that contains text, tables and graphs summarizing weekly or monthly disease counts. This presents a problem when information is needed for more data intensive approaches to epidemiology, biosurveillance and public health.While most nations likely store incident data in a machine-readable format, governments are often hesitant to share data openly for a variety of reasons that include technical, political, economic, and motivational issues1.A survey conducted by LANL of notifiable disease data reporting in over fifty countries identified only a few websites that report data in a machine-readable format. The majority (>70%) produce reports as PDF files on a regular basis. The bulk of the PDF reports present data in a structured tabular format, while some report in natural language.The structure and format of PDF reports change often; this adds to the complexity of identifying and parsing the desired data. Not all websites publish in English, and it is common to find typos and clerical errors.LANL has developed a tool, Epi Archive, to collect global notifiable disease data automatically and continuously and make it uniform and readily accessible.MethodsWe conducted a survey of the national notifiable disease reporting systems notating how the data are reported and in what formats. We determined the minimal metadata that is required to contextualize incident counts properly, as well as optional metadata that is commonly found.The development of software to regularly ingest notifiable disease data and make it available involves three or four main steps: scraping, detecting, parsing and persisting.Scraping: we examine website design and determine reporting mechanisms for each country/website, as well as what varies across the reporting mechanisms. We then designed and wrote code to automate the downloading of the data for each country. We store all artifacts presented as files (PDF, XLSX, etc.) in their original form, along with appropriate metadata for parsing and data provenance.Detecting: This step is required when parsing structured non-machine-readable data such as tabular data in PDF files. We combined the Nurminen methodology of PDF table detection with in-house heuristics to find the desired data within PDF reports2.Parsing: We determined what to extract from each dataset and parsed these data into uniform data structures, correctly accommodating the variations in metadata (e.g., time interval definitions) and the various human languages.Persisting: We store the data in the Epi Archive database and make it available on the internet and through the BSVE. The data is persisted into a structured and normalized SQL database.ResultsThe Epi Archive tool currently contains national and/or subnational notifiable disease data from twenty nations. When a user accesses the Epi Archive site, they are prompted with four fields: country, subregion, disease of interest, and date duration. Upon form submission, a time series is generated from the users’ specifications. The generated graph can then be downloaded into a CSV file if a user is interested in performing personal analysis. Additionally, the data from Epi Archive can be reached through a REST API (Representational State Transfer Application Programming Interface).ConclusionsLANL, as part of a currently funded DTRA effort, is automatically and continually collecting global notifiable disease data. While 20 nations are in production, more are being brought online in the near future. These data are already being utilized and will have many applications including improving the prediction and early warning of disease events.References[1] van Panhuis WG, Paul P, Emerson C, et al. A systematic review of barriers to data sharing in public health. BMC Public Health. 2014. 14:1144. doi:10.1186/1471-2458-14-1144[2] Nurminen, Anssi. "Algorithmic extraction of data in tables in PDF documents." (2013).

Khalsa, Hari S.↗
DOE OSTI2019

Soft decision analyzer and method

A soft decision analyzer system is operable to interconnect soft decision communication equipment and analyze the operation thereof to detect symbol wise alignment between a test data stream and a reference data stream in a variety of operating conditions.

Steele, Glen F.↗
DOE OSTI2015

A MGy radiation-hardened sensor instrumentation link for nuclear reactor monitoring and remote handling

Decommissioning, dismantling and remote handling applications in nuclear facilities all require robotic solutions that are able to survive in radiation environments. Recently raised safety, radiation hardness and cost efficiency demands from both the nuclear regulatory and the society impose severe challenges in traditional methods. For example, in case of the dismantling of the Fukushima sites, solutions that survive accumulated doses higher than 1 MGy are mandatory. To allow remote operation of these tools in nuclear environments, electronics were used to be shielded with several centimeters of lead or even completely banned in these solutions. However, shielding electronics always leads to bulky and heavy solutions, which reduces the flexibility of robotic tools. It also requires longer repair time and produces extra waste further in a dismantling or decommissioning cycle. In addition, often in current reactor designs, due to size restrictions and the need to inspect very tight areas there are limitations to the use of shielding. A MGy radiation-hardened sensor instrumentation link developed by MAGyICS provides a solution to build a flexible, easy removable and small I and C module with MGy radiation tolerance without any shielding. Hereby it removes all these pains to implement electronics in robotic tools. The demonstrated solution in this poster is developed for ITER Remote Handling equipments operating in high radiation environments (>1 MGy) in and around the Tokamak. In order to obtain adequately accurate instrumentation and control information, as well as to ease the umbilical management, there is a need of front-end electronics that will have to be located close to those actuators and sensors on the remote handling tool. In particular, for diverter remote handling, it is estimated that these components will face gamma radiation up to 300 Gy/h (in-vessel) and a total dose of 1 MGy. The radiation-hardened sensor instrumentation link presented here, consists of multiple in-house developed, state-of-the-art building blocks. They have all been assessed and characterized up to a dose higher than 1 MGy. The first block in the signal chain is a smaller than 1 uV offset, low noise instrumentation amplifier that has a programmable gain between 8 and 256. This amplifier is followed by a high precision 16 bit ADC. Its main function is to digitize the amplified signal coming from the amplifier. The output of the ADC is a serial digital data stream. The next block is a 8 channel digital multiplexer. It converts the 8 digital data streams into 1 digital data stream. Hereby the instrumentation solution reduces the number of cables from 16 to 1 (8* 2 analogue differential signals). In addition, the multiplexer modules can be combined with other modules to reduce even further the number of cables. Every instrumentation solution requires a stable high precision voltage reference. Therefore also a bandgap reference has been developed and assessed under gamma irradiation. A low jitter, 10 MHz clock generator has been developed and qualified to clock the ADC and the multiplexer with high accuracy. Finally, an on-chip radiation-hard temperature sensor is also included. A complete remote, real-time test setup was prepared by MAGyICS in cooperation with Fusion for Energy to qualify the sensor instrumentation link at SCK-CEN. It is qualified by closely following the ESCC22900 space standard for electronics used in a radiation environment. The main benefit of the sensor instrumentation solution discussed here is that it can be directly employed in a MGy-level accumulated dose radiation environment, therefore it can digitize and multiplex sensor readout values early in the signal chain. Hereby the sensor values are not distorted by external interferences on the long transmission cable. Moreover it allows readout and digitize multiple low-bandwidth sensors ( pressure and temperature sensors, thermocouples, angular resolvers and LVDTs). Hence these instrumentation modules can be used to follow the trend of the digitalization in the nuclear industry. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

Improving Quality of Observational Streaming Medical Data by Using Long Short-Term Memory Networks (LSTMs)

We present an exploration of the encoder-decoder structured Long Short-Term Memory Network (LSTM) as a detector of the anomalous missing observations in streaming medical data by using the difference between the LSTM-reconstructed and observed values as the anomaly detector. We experiment with time-series data from bedside monitoring devices from the available Medical Information Mart for Intensive Care Database (MIMIC). Our results show that not only encoder-decoder LSTM approach works well for detecting the difference between anomalous and normal missing observations in streaming medical data, but also has an imputation potential for the missing data.

Bowie, Michael B.↗
DOE OSTI2019

Open Humans: A platform for participant-centered research and personal data exploration

Background: Many aspects of our lives are now digitized and connected to the internet. As a result, individuals are now creating and collecting more personal data than ever before. This offers an unprecedented chance for human-participant research ranging from the social sciences to precision medicine. With this potential wealth of data comes practical problems (e.g., how to merge data streams from various sources), as well as ethical problems (e.g., how best to balance risks and benefits when enabling personal data sharing by individuals). Results: To begin to address these problems in real time, we present Open Humans, a community-based platform that enables personal data collections across data streams, giving individuals more personal data access and control of sharing authorizations, and enabling academic research as well as patient-led projects. We showcase data streams that Open Humans combines (e.g., personal genetic data, wearable activity monitors, GPS location records, and continuous glucose monitor data), along with use cases of how the data facilitate various projects. Conclusions: Open Humans highlights how a community-centric ecosystem can be used to aggregate personal data from various sources, as well as how these data can be used by academic and citizen scientists through practical, iterative approaches to sharing that strive to balance considerations with participant autonomy, inclusion, and privacy.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series: Preprint

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.

changepoint detection↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.

changepoint detection↗
DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as time series data shifts. In this research, a changepoint detection (CPD) algorithm that automatically detects data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window across the labeled data sets. Pending approval, we plan to release the labeled data sets for this research on NREL's DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package. By supplying the training sets and algorithm, we hope to encourage further development in this research space.

data shift↗
DOE OSTI2022

Overview and How-to Tutorial Videos for Using NEWTS Data

Overview and How-to Tutorial Videos for Using NEWTS Data Video 1: Overview of NEWTS Database Video 2: How-to tutorial for EPA Flue Gas Desulfurization (FGD) Effluent NEWTS dataset Video 3: How-to tutorial for USGS Produced Waters NEWTS dataset Video 4: How-to tutorial for EPA Ash NEWTS dataset Video 5: How-to tutorial for Quillinan, et al 2018 DOE Geothermal Technology Office REE dataset Video 6: Tutorial video on navigating the NEWTS Dashboard, with an overview of NEWTS and navigating between the NEWTS Dashboard and Datasets (https://netl-doe.maps.arcgis.com/apps/dashboards/a5fa4192f7c6478dab3d6180d9c30b84) Video 7: Additional tutorial video on navigating the NEWTS Dashboard and investigating specific data points in the Dashboard and Datasets Video 8: Re-record of recent webinar giving an overview of the NEWTS Database and Dashboard, including interacting with the NEWTS Dashboard, locating specific data points, and finding the relevant streams in the NEWTS Database and datasets on EDX. Includes overview of the datasets, case studies, and steps for taking stream data from the database and modeling stream data in OLI Studio and Geochemist's Workbench. Note: Video 3 tutorial is also applicable to the USGS Brackish Water NEWTS dataset.

Aqueous Chemistry↗
DOE OSTI2025

A Data-Agnostic, Continuous Machine Learning Framework for Application in High Energy Physics and Beyond: Phase 1 Final Scientific/Technical Report

This Phase 1 effort has focused on the development of continual learning frameworks for use in machine learning, specifically in the applied context of High Energy Physics (HEP). Machine learning (ML) is a transformative technology by which computers, typically through the use of neural networks, are able to perform tasks with proficiency that rivals or surpasses that of human users. Model Degradation & Catastrophic Forgetting are two undesired phenomena which can occur in ML where the performance of a model degrades when either deployed on novel data streams, or trained on novel data which are sufficiently different than the data the models were initially trained on. A natural example where these sorts of effects can be observed is in the performance of detectors in harsh environments, where the detector signature may change over the lifetime of the detector as it ages and deteriorates — precisely what occurs in the experiments conducted in HEP. Real world HEP data is therefore an excellent test-ground and use-case for Continual Learning paradigms, which are techniques used in ML to counteract these problems. Ensemble learning is one such technique, where multiple smaller models are trained on subsets of the overall data and are ensembled together during inference. The intuition behind this technique is that, although there are shifts in the distributions which govern the incoming data streams, these shifts are not expected to be homogeneous or global. If a sufficient diversity in solutions within the various sub-models has been achieved, then at least one sub-model is expected to retain its performance within the overall ensemble. One further strength of this approach is that the architectures of the various models do not need to be identical, and in fact even different modalities of data can naturally be combined in this way. This work focused on applying ensemble learning techniques to derive results using two main datasets, anomaly detection in HEP data & time-series forecasting in semiconductor manufacturing data. Semiconductor manufacturing involves data with surprising similarity to that of HEP (e.g. wafer maps look very similar to digi-occupancy maps) and Cerium Lab’s prominence within the semiconductor industry makes semiconductor manufacturing a natural opportunity for commercialization of this work. Our efforts have led to two strong results. The first is that we evaluated the proposed ensembling techniques using previously proposed machine learning architectures for use in anomaly detection, namely AutoEncoder based models and their derivatives. We also developed new architectures which have not been evaluated in this context before. In fact, this work marks the first use of Vision Transformers for anomaly detection in HEP. Second, we demonstrated that ensemble learning significantly improves model performance in scenarios prone to degradation, validating its effectiveness across both HEP and semiconductor datasets. These results further support ensemble learning as a powerful strategy for mitigating catastrophic forgetting and maintaining robust performance in evolving data environments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

StOKeDMD: Streaming Occupation kernel dynamic mode decomposition

Dynamic mode decomposition (DMD) has become a common technique for constructing surrogate models for dynamical systems from observed system states. The Occupation Kernel DMD (OKDMD) method proposed in (Rosenfeld et al., 2022) and (Rosenfeld et al., 2024) is a Liouville operator based method that builds surrogate models from system state trajectories. Here, this paper proposes an extension of OKDMD to the case when the system states are observed in a streaming fashion, i.e., only a small fraction of the state trajectory is available at a given time. The developed method, Streaming Occupation Kernel DMD (StOKeDMD), accommodates the streaming data input by leveraging properties of specific choices of kernel functions and occupation kernels. We apply the StoKeDMD method as a compression method for streaming data, analyze the memory complexity, and demonstrate the performance of StoKeDMD in the compression of streaming data generated from a Lorenz system and a fluid flow simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2011

High Temperature Materials Interim Data Qualification Report FY 2011

Projects for the very high temperature reactor (VHTR) Technology Development Office provide data in support of Nuclear Regulatory Commission licensing of the VHTR. Fuel and materials to be used in the reactor are tested and characterized to quantify performance in high temperature and high fluence environments. The VHTR program has established the Next Generation Nuclear Plant (NGNP) Data Management and Analysis System (NDMAS) to ensure that VHTR data are qualified for use, stored in a readily accessible electronic form, and analyzed to extract useful results. This document focuses on the first NDMAS objective. It describes the High Temperature Materials characterization data stream, the processing of these data within NDMAS, and reports the interim fiscal year (FY) 2011 qualification status of the data. Data qualification activities within NDMAS for specific types of data are determined by the data qualification category assigned by the data generator. The High Temperature Materials data are being collected under the Nuclear Quality Assurance (NQA)-1 guidelines and will be qualified data. For NQA-1 qualified data, the qualification activities include: (1) capture testing to confirm that the data stored within NDMAS are identical to the raw data supplied, (2) accuracy testing to confirm that the data are an accurate representation of the system or object being measured, and (3) documenting that the data were collected under an NQA-1 or equivalent Quality Assurance program. Currently, data from seven test series within the High Temperature Materials data stream have been entered into the NDMAS vault, including tensile tests, creep tests, and cyclic tests. Of the 5,603,682 records currently in the vault, 4,480,444 have been capture passed, and capture testing is in process for the remaining 1,123,238.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2010

High Temperature Materials Interim Data Qualification Report

ABSTRACT Projects for the very high temperature reactor (VHTR) Technology Development Office provide data in support of Nuclear Regulatory Commission licensing of the VHTR. Fuel and materials to be used in the reactor are tested and characterized to quantify performance in high temperature and high fluence environments. The VHTR program has established the NGNP Data Management and Analysis System (NDMAS) to ensure that VHTR data are qualified for use, stored in a readily accessible electronic form, and analyzed to extract useful results. This document focuses on the first NDMAS objective. It describes the High Temperature Materials characterization data stream, the processing of these data within NDMAS, and reports the interim FY2010 qualification status of the data. Data qualification activities within NDMAS for specific types of data are determined by the data qualification category assigned by the data generator. The High Temperature Materials data are being collected under NQA-1 guidelines, and will be qualified data. For NQA-1 qualified data, the qualification activities include: (1) capture testing, to confirm that the data stored within NDMAS are identical to the raw data supplied, (2) accuracy testing to confirm that the data are an accurate representation of the system or object being measured, and (3) documenting that the data were collected under an NQA-1 or equivalent Quality Assurance program. Currently, data from two test series within the High Temperature Materials data stream have been entered into the NDMAS vault: 1. Tensile Tests for Sm (i.e., Allowable Stress) Confirmatory Testing – 1,403,994 records have been inserted into the NDMAS database. Capture testing is in process. 2. Creep-Fatigue Testing to Support Determination of Creep-Fatigue Interaction Diagram – 918,854 records have been processed and inserted into the NDMAS database. Capture testing is in process.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2018

Adverse Event Prediction Using Graph-Augmented Temporal Analysis (Final Report)

This report summarizes the work performed under the Sandia LDRD project "Adverse Event Prediction Using Graph-Augmented Temporal Analysis." The goal of the project was to develop a method for analyzing multiple time-series data streams to identify precursors providing advance warning of the potential occurrence of events of interest. The proposed approach combined temporal analysis of each data stream with reasoning about relationships between data streams using a geospatial-temporal semantic graph. This class of problems is relevant to several important topics of national interest. In the course of this work we developed new temporal analysis techniques, including temporal analysis using Markov Chain Monte Carlo techniques, temporal shift algorithms to refine forecasts, and a version of Ripley's K-function extended to support temporal precursor identification. This report summarizes the project's major accomplishments, and gathers the abstracts and references for the publication submissions and reports that were prepared as part of this work. We then describe work in progress that is not yet ready for publication.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Adverse Event Prediction Using Graph-Augmented Temporal Analysis (Final Report)

This report summarizes the work performed under the Sandia LDRD project "Adverse Event Prediction Using Graph-Augmented Temporal Analysis." The goal of the project was to develop a method for analyzing multiple time-series data streams to identify precursors providing advance warning of the potential occurrence of events of interest. The proposed approach combined temporal analysis of each data stream with reasoning about relationships between data streams using a geospatial-temporal semantic graph. This class of problems is relevant to several important topics of national interest. In the course of this work we developed new temporal analysis techniques, including temporal analysis using Markov Chain Monte Carlo techniques, temporal shift algorithms to refine forecasts, and a version of Ripley's K-function extended to support temporal precursor identification. This report summarizes the project's major accomplishments, and gathers the abstracts and references for the publication sub-missions and reports that were prepared as part of this work. We then describe work in progress that is not yet ready for publication.

42 ENGINEERING↗
DOE OSTI2020

Estimating Watershed Subsurface Permeability From Stream Discharge Data using Deep Neural Networks, Frontiers in Earth Science: Dataset

This data package contains watershed modeling inputs and outputs as well as deep neural networks training and testing results used in "Estimating Watershed Subsurface Permeability From Stream Discharge Data using Deep Neural Networks" (Cromwell et al., 2021). We train various deep neural network models with different architectures to predict subsurface permeability from stream discharge hydrograph at the watershed outlet. The training data are obtained from ensemble simulations of hydrographs corresponding to an permeability ensemble using a fully-distributed, integrated surface-subsurface hydrologic model. The trained model is then applied to estimate the permeability of the real watershed using its observed hydrograph at the outlet. Our study demonstrates that the permeabilities of the soil and geologic facies that make significant contributions to the outlet discharge can be more accurately estimated from the discharge data. Their estimations are also more robust with observation errors. Compared to the traditional ensemble smoother method, DNNs show stronger performance in capturing the nonlinear relationship between permeability and stream hydrograph to accurately estimate permeability. Our study sheds new light on the value of the emerging deep learning methods in assisting integrated watershed modeling by improving parameter estimation, which will eventually reduce the uncertainty in predictive watershed models. For detailed information regarding watershed model description and DNNs setup, please refer to Cromwell et al., 2021. The deep_learning.zip file contains the inputs and outputs for the different deep neural networks. The watershed_model.zip file contains the inputs and outputs for watershed simulation using ATS. The figures.zip file contains raw figures and their corresponding scripts. For a more detailed description, please see the README.md file within each compressed file.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Rain Gauges Handbook

To improve the quantitative description of precipitation processes in climate models, the Atmospheric Radiation Measurement (ARM) Climate Research Facility deployed rain gauges located near disdrometers (DISD and VDIS data streams). This handbook deals specifically with the rain gauges that make the observations for the RAIN data stream. Other precipitation observations are made by the surface meteorology instrument suite (i.e., MET data stream).

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

EJFAT Scientific Perspective

Presented new computing model to the test by deploying the EJFAT system alongside a data-stream processing framework running the production-level CLAS12 event reconstruction application. In this experiment, a continuous stream of CLAS12 Level-1 identified events was processed in real-time using the EJFAT load balancer, distributing the workload across 90 computing nodes located across the U.S. This marks the first-ever large-scale, real-time distributed data stream processing experiment, demonstrating that scientific data-streaming pipelines can efficiently scale across four dimensions, thanks to EJFAT’s advanced hardware and software capabilities.

Gyurjyan, Vardan [Thomas Jefferson National Accele↗