Search NASA⌕ Search

SEARCH · Search NASA

Results for “stream processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

CLAS12 remote data-stream processing using ERSAP framework

Implementing a physics data processing application is relatively straightforward with the use of current containerization technologies and container image runtime services, which are prevalent in most high-performance computing (HPC) environments. However, the process is complicated by the challenges associated with data provisioning and migration, impacting the ease of workflow migration and deployment. Transitioning from traditional file-based batch processing to data-stream processing workflows is suggested as a method to streamline these workflows. This transition not only simplifies file provisioning and migration but also significantly reduces the necessity for extensive disk space. Data-stream processing is particularly effective for real-time processing during data acquisition, thereby enhancing data quality assurance. This paper introduces the integration of the JLAB CLAS12 event reconstruction application within the ERSAP data-stream processing framework that facilitates the execution of streaming event reconstruction at a remote data center and enables the return streaming of reconstructed events to JLAB while circumventing the need for temporary data storage throughout the process.

Gyurjyan, Vardan↗
DOE OSTI2021

Monitoring Radiochemical Processing Streams for the 238 Pu Supply Program with Process Pulse II

Oak Ridge National Laboratory (ORNL) is developing advanced spectroscopic and real-time monitoring capabilities to improve the timeliness of analytical measurements and process decisions for the 238Pu Supply Program. Reducing the time, resources, and costs associated with each production campaign is critical because overlapping campaigns will be required to meet the production goals of the National Aeronautics and Space Administration. Real-time, in situ analytical measurements in the heavily shielded hot cells at the Radiochemical Engineering Development Center (REDC) will allow for rapid process information feedback and operational benefits that help the 238Pu supply program scale-up production efforts. Noteworthy steps were taken during Campaign 5 to establish the ability to monitor processing streams in real time with spectrophotometry and a commercially available online monitoring software called The Unscrambler X Process Pulse II (PP) multivariate statistical process monitoring system by Camo Analytics (version 5.60). PP automates univariate-type calculations within the software itself and executes multivariate models built using The Unscrambler X (version 10.4 or newer). The Unscrambler is a commercially available data analysis software made by the same company. PP is composed of easy-to-use-tools for all personnel, including data scientists and technicians. The software can be used to plot analyte concentration profiles, spectral data, and other process variables in real time. All process data are represented in a single view with interactive charts useful for viewing how a process evolves over time.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2024

Streaming Readout and Data-Stream Processing With ERSAP

With the exponential growth in the volume and complexity of data generated at high-energy physics and nuclear physics research facilities, there is an imperative demand for innovative strategies to process this data in real or near-real-time. Given the surge in the requirement for high-performance computing, it becomes pivotal to reassess the adaptability of current data processing architectures in integrating new technologies and managing streaming data. This paper introduces the ERSAP framework, a modern solution that synergizes flow-based programming with the reactive actor model, paving the way for distributed, reactive, and high performance in data stream processing applications. Additionally, we unveil a novel algorithm focused on time-based clustering and event identification in data streams. The efficacy of this approach is further exemplified through the data-stream processing outcomes obtained from the recent beam tests of the EIC prototype calorimeter at DESY.

Vardan, Gyurjyan↗
DOE OSTI2026

Membrane-based solvent extraction for the recovery of rare earths from phosphate mining process streams

This study reports on the capture of rare earth elements (REEs) from phosphate industry process streams, including phosphoric acid (PA) sludge and phosphogypsum (PG), using a membrane solvent extraction (MSX) process. While MSX has been proven effective for a relatively concentrated feed, its effectiveness for dilute REEs solutions remains unexplored. Investigated PA-sludge and PG particles contain total REEs concentrations of ∼1100 and ∼320 ppm, respectively. Acid leaching, implemented to dissolve the REEs, significantly dilutes the REEs concentration to ∼210 ppm for PA-sludge leachate and ∼60 ppm for PG leachate. These low concentrations, compounded by the higher levels of non-REE ions and radioactive species, uranium (U) and thorium (Th), poses challenges to the MSX process. Here, we demonstrated that N,N,N′,N′-tetraoctyl-diglycolamide (TODGA) selectively binds REEs from a >3 M nitric-acid leachate while effectively rejecting U and Th. Concentrations of light REEs in strip solution were doubled compared to the feed, while heavy REEs were preferentially extracted. Furthermore, >99% purity gypsum, free of U and Th, was precipitated during the acid leaching process, aiding separation by removing significant amounts of non-REEs species (e.g., calcium) prior to the MSX process. Molecular simulations support the experimental data, suggesting preferential separation of heavy over light REEs. Based on these results, a cost-effective integrated process including pretreatment, acid leaching, MSX, and wastewater treatment is proposed for the co-recovery of REEs, phosphoric acid, gypsum, and U. This study shows MSX as a technically and economically feasible process for the recovery of REEs from low-concentration process streams, offering advantages over conventional solvent extraction.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Watermarks in stream processing systems: semantics and comparative analysis of Apache Flink and Google cloud dataflow

Streaming data processing is an exercise in taming disorder: from oftentimes huge torrents of information, we hope to extract powerful and timely analyses. But when dealing with streaming data, the unbounded and temporally disordered nature of real-world streams introduces a critical challenge: how does one reason about the completeness of a stream that never ends? In this paper, we present a comprehensive definition and analysis of watermarks, a key tool for reasoning about temporal completeness in infinite streams.First, we describe what watermarks are and why they are important, highlighting how they address a suite of stream processing needs that are poorly served by eventually-consistent approaches:• Computing a single correct answer, as in notifications.• Reasoning about a lack of data, as in dip detection.• Performing non-incremental processing over temporal subsets of an infinite stream, as in statistical anomaly detection with cubic spline models.• Safely and punctually garbage collecting obsolete inputs and intermediate state.• Surfacing a reliable signal of overall pipeline health.Second, we describe, evaluate, and compare the semantically equivalent, but starkly different, watermark implementations in two modern stream processing engines: Apache Flink and Google Cloud Dataflow.

Akidau, Tyler↗
DOE OSTI2025

Rapid Sensing to Facilitate Purification of Rare Earth Element-Containing Process Streams Produced Through Membrane-Assisted Solvent Extraction

The development of an economically competitive domestic supply of rare earth elements and yttrium (REY) is necessary for our nation’s economic growth and national security. The achievement of a secure domestic supply of REY requires not only the development of effective processes for recovery of REY from naturally occurring materials and/or recycled products, but also the development of downstream processes for the ultimate production of high-REY content solids. An impediment to the development of such processes is the scarcity of analytical methods that provide rapid determination of the process stream compositions. In this work, a membrane-based extraction process was used to selectively recover REYs from a dilute solution in the presence of much higher concentrations of Ca and Al. In tandem, the use of a portable spectrometer equipped with an immobilized zinc adeninate benzene tricarboxylate metal-organic framework sensing material makes possible the rapid detection of the presence of ppm concentrations of Tb and Eu in both weakly acidic and strongly acidic process streams within minutes. A solvent extraction processing time of 15-60 min maximized REY selectivity over gangue ions while achieving up to 80% REY and minimal gangue ion recovery. Taken together, these experiments highlight not only an innovative method for REY purification but also the importance of inexpensive, portable characterization methods for near real-time analysis of REY content.

Membrane-assisted solvent extraction↗
DOE OSTI2024

Arroyo Stream Processing Toolset (arroyopy) v0.1.0

Processing event or streaming data presents several technological challenges. A variety of technologies are often used by scientific user facilities. ZMQ is used to stream data and messages in a peer-to-peer fashion. Message brokers like Kafka, Redis Pubsub, EPICS PVA and RabbitMQ are often employed to route and pass messages from instruments to processing workflows. Arroyopy provides an API and structure to flexibly integrate with these tools and incorporate arbitrarily complex processing workflows, letting the hooks to the workflow code be independent of the connection code and hence reusable at a variety of instruments.

Chavez Esparza, Tanny Andrea [Lawrence Berkeley Na↗
DOE OSTI2021

Online Monitoring of Radiochemical Processing Streams for the Plutonium-238 Supply Program

Online monitoring with spectrophotometry is being developed to improve the timeliness of analytical measurements for the Plutonium-238 Supply Program at Oak Ridge National Laboratory. A commercially available online monitoring software was used to calculate and view process data in real time to help identify process deviations and optimize system performance. Monitoring detailed process data will improve processing efficiency and help technicians make decisions during hot-cell operations.

Sadergaski, Luke↗
DOE OSTI2022

Leveraging Multiple Raman Excitation Wavelength Systems for Process Monitoring of Nuclear Waste Streams

Processing nuclear waste from sites such as Hanford is a significant environmental cleanup need while being a significant logistical challenge. Integration of process monitoring tools, that can provide in situ and real-time feedback about the process, can significantly alleviate needs to collect grab samples for process control and product characterization. Raman spectroscopy paired with chemometric analysis is one process monitoring tool that can provide chemical composition information on a large number of chemical targets in nuclear waste streams. However, methods to improve limits of detection as well as drop uncertainty in quantification are needed. Optimizing instrument specifications can achieve this, here this is demonstrated by comparing limits of detection for key analytes when using Raman systems with 671 nm, 532 nm, and 405 nm excitation wavelengths. Generally, limits of detection decease (allowing the measurement of lower salt concentrations) with decreasing wavelength. Similarly, data collection times and averaging were optimized. Finally, multiple chemometric modeling approaches were leveraged, including multiblock methods that combined data from all three Raman systems to simultaneously quantify targets with improved sensitivity.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Sensor Fusion: Comprehensive Real-Time, On-Line Monitoring for Process Control via Visible, Near-Infrared, and Raman Spectroscopy

On-line monitoring based on optical spectroscopy provides unprecedented insight into the chemical composition of process streams or batches. Amplifying this approach through utilizing multiple forms of optical spectroscopy in sensor fusion can greatly expand the number and type of chemical species that can be identified and quantified. This is demonstrated herein, on the analysis of used nuclear fuel recycling streams: highly complex processes with multiple target and interfering analytes. The optical techniques of visible absorbance, near infrared absorbance, and Raman spectroscopy were combined to quantify plutonium (III, IV, VI), uranium (IV, VI), neptunium (IV, V, VI), and nitric acid. Chemometric modeling was used to quantify analytes in process streams in real-time and results were successfully used to enable immediate process control and generation of a product stream at a set composition ratio. This represents a significant step forward in the ability to monitor and control complex chemical processes occurring in harsh chemical environments.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Natural gas liquefaction employing independent refrigerant path

A method of liquefying natural gas. The method comprises cooling a gaseous natural gas process stream with a refrigerant flowing in a path isolated from the natural gas process stream. The refrigerant may differ in composition from a composition of the natural gas process stream, and the refrigerant composition may be selected to enhance efficiency of the refrigerant path with regard to a specific composition of the natural gas process stream. The refrigeration path may be operated at pressures, temperatures and flow rates differing from those of the natural gas process stream. Other methods of liquefying natural gas are described. A natural gas liquefaction plant is also described.

03 NATURAL GAS↗
DOE OSTI2021

Raman Spectroscopy Based On-Line, Real-Time Monitoring to Reduce Composition Uncertainties: Enhanced sensitivity through optimization of Raman Parameters

Optical spectroscopy-based on-line monitoring of Hanford processing streams can enable real-time characterization of chemical composition of process streams and batches, ultimately enabling and enhancing process control. It can provide immediate feedback on process conditions and has the potential to reduce the needed number of grab sample collections, thereby reducing times and costs associated with laboratory processing. Here we discuss the utilization of Raman spectroscopy to quantify multiple target analytes that are common within Hanford tanks and waste processing streams. Analytes include: nitrate, nitrite, carbonate, chromate, sulfate, phosphate, hydroxide, oxalate, ammonia, and aluminate. Most notably in this work, Raman applications to low-concentration streams are explored and optimized. Raman instrument specifications are compared; specifically, the impact of utilizing three different Raman excitation wavelengths, 405, 532, and 671 nm, is discussed. Also, Raman data collection parameters such as collection time and spectral averaging are measured and discussed. Finally, optical libraries of chemical targets were collected using optimized collection parameters and chemometric models were built to automate quantification of chemical targets. These models were validated through application to simulants and real Hanford process samples. Chemometric models performed well on both training and validation sets, suggesting these approaches can be successfully applied to on-line and real-time monitoring of low concentration Hanford processing streams. The use of the combine Raman wave lengths with the enhanced chemometric models for the low concentration streams significantly improved the chemical detection levels and significantly reduced uncertainties of those measurements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Analysis of NuCycle® Process Waste Streams and Identification of Candidate Waste Forms (CRADA 719) Abstract

The objective of this project is to support collaboration between PNNL and Curio to address technical gaps in the immobilization of waste streams generated by the NuCycle® process, thereby improving the overall viability of the technology. This work will be carried out in two phases, described below. Phase 1 – Waste Management Study The first phase consists of a comprehensive waste management study focused on the waste streams produced during the NuCycle® process, including those containing long lived fission products (LLFPs, e.g., I 129) and transuranic elements (TRUs). This phase involves compiling waste compositions and characteristics based on recently completed NuCycle® testing and modeling efforts. Established waste forms and processing methods will be identified for streams where they are applicable.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Methods for separating components

Method for concentrating components through a distillation column are disclosed. A process stream is provided containing hydrocarbons, carbon dioxide, and water. The process stream is passed into the distillation column at a bottom portion of the distillation column. The process stream is fractionally distilled in the distillation column, forming an overhead vapor stream, a middle fluid stream, and a bottoms liquid stream. The middle fluid stream is removed from a middle location of the distillation column. The bottoms liquid stream is removed from the distillation column. The overhead vapor stream is removed from the distillation column. The middle fluid stream consists of a first portion of the carbon dioxide and a first portion of the hydrocarbons. The bottoms liquid stream consists of a second portion of the hydrocarbons as a first phase and substantially all the water as a second phase.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Hyporheic-zone Processes and Stream Oxygen Dynamics: Insights from a Multiscale Reactive Transport Model: Modeling Archive

This archive contains the data and Python scripts required to reproduce the analyses and figures in the study: Gomez-Velez, J. D., Rathore, S. S., Cohen, M. J., & Painter, S. L. (2025). Hyporheic-zone Processes and Stream Oxygen Dynamics: Insights from a Multiscale Reactive Transport Model. Submitted to Water Resources Research. The analysis utilizes the subgrid model Advection Dispersion Equation with Lagrangian Subgrids (ADELS) implemented in the Advanced Terrestrial Simulator (ATS; https://amanzi.github.io/ats/stable/). In this case, the ATS and Amanzi versions are (1) ATS version 1.5.1_f5ba18f8 and (2) Amanzi version 1.6-dev_53444cca4. The repository includes a Jupyter Notebook and the necessary data (Pandas DataFrames stored as pickle files) to generate the figures for the manuscript. Additionally, it contains Python scripts to create ATS input files, run the ATS simulations, and post-process the results. Finally, it provides routines for parameter estimation using the Single-Station Metabolism (SSM) model with the Differential Evolution Adaptive Metropolis (DREAM) Markov Chain Monte Carlo (MCMC) algorithm with ZS enhancements (DREAM-ZS).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Preliminary Testing of a Continuous Cryopump for Primary Fusion Device Pumping and Direct Internal Recycling

Here, the concept of directly recirculating fusion machine exhaust gas, bypassing the tritium plant, to make fuel pellets was proposed in the 1990s and later termed direct internal recycling (DIR). In the DIR concept, the residual fusion fuel in the machine exhaust stream is separated from impurities locally and diverted directly to the fueling systems, bypassing isotopic separation and other processing equipment, and therefore significantly reducing the required size of the fuel processing plant, reducing plant inventory, and thus increasing the economic viability of fusion as an energy source. One concept for DIR consists of a series of cryogenic pumps to separate the impurities from the machine exhaust gas using different triple point temperatures and saturation curves of exhaust constituents. In this concept, the plasma exhaust is initially passed through an impurity trap operating at ~25–30 K to desublimate impurities such as hydrocarbons, argon, oxygen, and nitrogen. The resulting process stream will consist of DT fuel and helium. The process stream is then pumped by a continuous cryopump known as a “snail pump.” This pump is a steady-state continuous cryopump that desublimates all remaining exhaust gas constituents while allowing helium, a byproduct of the fusion reaction, to pass through. The helium is pumped to the tritium plant for processing while the desublimated material is continuously scraped off, heated up, and transported to the fueling system. This article will present the cryogenic DIR concept and outline the design and operation of the snail pump, along with results from preliminary testing. Tests to assess pumping and separation efficiency found that at D2 flows below 50.7 Pa ⋅ m3/s with 1% helium, the pump is capable of pumping and separating the gas with a resulting DIR fraction of >99%, with no helium entrained in the primary fuel exhaust stream. The main limitation is due to the thermal performance of the cryogenic circuits of the pump, which will be addressed in future testing.

Gebhart III, Trey E. [Oak Ridge National Laborator↗