Search NASA⌕ Search

SEARCH · Search NASA

Results for “stream processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A Dose Assessment Model for Radioactive Waste Exposed by Gully Erosion at West Valley - 20513

The Western New York Nuclear Service Center (WNYNSC), located approximately 48 km south of Buffalo, New York, is the site of a former nuclear fuel reprocessing and radioactive waste disposal facility. Spent nuclear fuel was processed there from 1966 to 1972, leaving behind radioactive and chemical wastes in two disposal areas and a waste tank farm. Site operations also resulted in releases of radioactivity to site soils, groundwater, and to surface waters draining the site. The New York State Energy Research and Development Authority (NYSERDA) and the U.S. Department of Energy (DOE) are collaborating in a process of decision making for decommissioning those facilities remaining at the WNYNSC following the completion of Phase 1 decommissioning. Neptune and Company, Inc. (Neptune) was contracted to develop a probabilistic performance assessment (PPA) computer model to assist the agencies in this process. The PPA Model includes a contaminant transport component focusing on the movement of contaminants within and among environmental media including groundwater and surface water transport, contaminant translocation by plants and animals, diffusion, and erosion. The model also includes evaluation of potential exposure and health effects for a Resident Farmer exposure scenario, where the Resident Farmer represents a critical group, described as that group of individuals reasonably expected to receive the greatest exposure to residual radioactivity for any applicable set of circumstances. The West Valley Site is located in the glaciated Allegheny Plateau region of western New York State. The waste reprocessing and disposal areas were constructed on a relatively fat area of plateau dissected by drainages of Buttermilk Creek, including Erdman Brook, Franks Creek, and Quarry Creek. An important aspect of the contaminant transport component of the PPA model is consideration of erosive processes such as slumping of the stream slopes and the advance of gullies from these streams. These erosion processes remove material from the plateau, growing the size of the creek valleys and making them wider and deeper. Of particular interest for the impact of erosion is radioactive waste contained in the Nuclear Regulatory Commission (NRC)-Licensed Disposal Area (NDA), the New York State-Licensed Disposal Area (SDA), and residual radiological inventory in the underground storage tanks at the Waste Tank Farm (WTF). The PPA Model is organized around geographically-defined facilities which were constructed upon the plateau, including the NDA, SDA, and WTF. Ongoing stream erosion processes will potentially transfer radioactive waste and residual inventory from these facilities to the ground surface on adjacent hillslope areas where erosion has breached the facility. Hence, it is important to evaluate the consequences of potential exposures to a Resident Farmer on the hillslopes below a breached facility. Two interrelated aspects of the dose assessment model related to hillslope exposure are discussed: 1) representation of the physical processes related to transport of radionuclides from facilities onto the hillslopes, and from the hillslopes into adjoining creeks where contaminated material migrates downstream with surface water and sediment, and; 2) adaptation of the activities associated with the Resident Farmer scenario to assess potential exposures to contamination in the hillslope areas. This discussion will cover the conceptual basis of the hillslope exposure and transport models, and also implementation in the PPA computer model. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

In Situ Transmission Electron Microscopy: Signal processing challenges and examples

Transmission electron microscopy (TEM) is a powerful tool for imaging material structure and characterizing material chemistry. Recent advances in data collection technology for TEM have enabled high-volume and high-resolution data collection at a microsecond frame rate. Here, taking advantage of these advances in data collection rates requires the development and application of data processing tools, including image analysis, feature extraction, and streaming data processing techniques. In this article, we highlight a few areas in materials science that have benefited from combining signal processing and statistical analysis with data collection capabilities in TEM and present a future outlook on opportunities of integrating signal processing with automated TEM data analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Roadmap to Iodine and Mercury Abatement Materials Selection in Nuclear Waste Processing Off-Gas Streams

This work provides a guide for candidate mercury (Hg) and iodine-129 (129I) abatement material identification, screening, evaluation, technical gap identification, and bench-scale testing prioritization while also conceptualizing a materials deployment roadmap for implementation of new materials in the Hanford Waste Treatment and Immobilization Plant (WTP) Low-Activity Waste (LAW) Facility at Hanford in Richland, Washington, and elsewhere in the U.S. Department of Energy (DOE) complex for similar applications. The study was prompted by the need to replace the Kombisorb BAT-37 due to uncertainties in Hg and 129I capture performance and future availability for use in the WTP LAW Facility secondary off-gas system, specifically the Carbon Adsorber units. However, replacement of this material could also mitigate two other issues with Kombisorb BAT-37 (and its successor BAT II 37): (1) fire safety risk due to exothermic heat generated by adsorption reactions between the carbon material and Hg and (2) the risk of low retention of 129I in glass and subsequent downstream impacts on secondary liquid waste treatment at the Effluent Treatment Facility.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

CO2 Streams Captured From Industrial Processes and Corrosion of Steel Pipelines

Based on the knowledge accumulated in the last decade related to Carbon Capture and Storage (CCS) processes and the identified gaps in knowledge, our study identifies and quantifies the key issues that impact the corrosion of materials, specifically relating to the integrity of structures for the CO2 transport pipelines and provides a better understanding of the materials’ compatibility with CO2 steams with impurities. This will help to understand the corrosion risks associated with CO2 transport and injection and leverage existing infrastructure or rebuild new pipeline networks to transport CO2 with impurities for EOR or geological storage.

CO2 corrosion↗
NASA NTRS2009

Hamming and Accumulator Codes Concatenated with MPSK or QAM

In a proposed coding-and-modulation scheme, a high-rate binary data stream would be processed as follows: 1. The input bit stream would be demultiplexed into multiple bit streams. 2. The multiple bit streams would be processed simultaneously into a high-rate outer Hamming code that would comprise multiple short constituent Hamming codes a distinct constituent Hamming code for each stream. 3. The streams would be interleaved. The interleaver would have a block structure that would facilitate parallelization for high-speed decoding. 4. The interleaved streams would be further processed simultaneously into an inner two-state, rate-1 accumulator code that would comprise multiple constituent accumulator codes - a distinct accumulator code for each stream. 5. The resulting bit streams would be mapped into symbols to be transmitted by use of a higher-order modulation - for example, M-ary phase-shift keying (MPSK) or quadrature amplitude modulation (QAM). The novelty of the scheme lies in the concatenation of the multiple-constituent Hamming and accumulator codes and the corresponding parallel architectures of the encoder and decoder circuitry (see figure) needed to process the multiple bit streams simultaneously. As in the cases of other parallel-processing schemes, one advantage of this scheme is that the overall data rate could be much greater than the data rate of each encoder and decoder stream and, hence, the encoder and decoder could handle data at an overall rate beyond the capability of the individual encoder and decoder circuits.

Divsalar, Dariush↗
DOE OSTI2025

Remote sensing images, DEM, and point clouds associated with “Accuracy evaluation of cost-effective 3D reconstruction approaches for hydrobiogeochemical processes in non-perennial stream riverbeds”

This data package is associated with the publication “Accuracy evaluation of cost-effective 3D reconstruction approaches for hydrobiogeochemical processes in non-perennial stream riverbeds” published in Frontiers in Environmental Science, Environmental Informatics and Remote Sensing (Bao et al., 2026; doi: 10.3389/fenvs.2026.1725258). This data package includes the drone photos for a section of Umtanum Creek in Washington, Unted States. The photos were used to reconstruct the 3-dimensional (3D) digital elevation model (DEM) of the riverbed for the investigated stream section. The reconstruction results from four approaches are provided: (1) unoccupied aerial vehicle (UAV, colloquially known as drone) imagery-based Structure-from-Motion (SfM), (2) a machine learning-based 3D reconstruction model, Visual Geometry Grounded Deep Structure from Motion (VGGSfM), (3) Visual Geometry Grounded Transformer for long sequence of images (VGGT-Long), and (4) handheld smartphone LiDAR scanning. The ground truth measurements by tripod-mounted optical level kit and ground control points GPS locations for evaluating the accuracy of the four reconstruction approaches are also provided in this data package. A preliminary version of this data package was published in October 2025 at the time of manuscript submission. It was updated in March 2026, at the time of manuscript acceptance, to include additional metadata (this readme, data dictionary, and file level metadata). The data did not change. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) 8 folders; (2) the detailed flight configuration html files; (3) field metadata; (4) a readme; (5) a data dictionary; and (6) file-level metadata. The folders “2024_10_18_d01” and “2024_10_18_d02” contain the original drone photos for the two drone flights (d01 and d02) on October 18, 2024. The reconstruction results from each of the approaches are in the folders called “ODM_SfM”, “VGGSfM”, “VGGTLong”, and “LiDAR”. The ground truth measurements are in the folder called “optical_level_kit”. Lastly, results comparing the different approaches are in the folder called “comparisons”. All files are .csv, .html, .jpg, .obj, .txt, and .npy. For information on using the .obj and .npy files, see the readme files within the same folder as the files.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2019

Thirty-Six Combined Years of MODIS Geolocation Trending

Two Moderate Resolution Imaging Spectroradiometer (MODIS) sensors have been in operations for more than 19 and 17 years (thus 36 combined years) as part of NASA's Earth Observing System (EOS) on the Terra platform that was launched in December 1999 and on the Aqua platform that was launched in May 2002, respectively. Accurate geolocation is a critical element needed for accurate retrieval of global biogeophysical parameters. In this paper, we describe the latest trends in the continuously improved MODIS geolocation accuracy in Collection-5 (C5), C6 and C6.1 re-processing and forward-processing data streams. We improved geolocation accuracy in the re-processed data and corrected for geolocation biases found in forward-processed data, including those caused by operations such as the stop-go-stop status of the Advanced Microwave Scanning Radiometer for EOS (AMSR-E) instrument on the Aqua platform. We discuss scan-toscan underlaps near nadir over the equator regions that was discovered in checking the non-underlapping requirement in the Visible Infrared Imaging Radiometer Suite (VIIRS) based on trending parameters from the actual Suomi National Polar-orbiting Partnership (S-NPP) satellite orbit. The underlaps are closely tied to instrument effective focal length that is measured from on-orbit data using a technique we recently developed. We also discuss potential improvements for the upcoming C7 re-processing.

ground control point↗
DOE OSTI2019

ANSTO Synroc Wasteforms for the Immobilization of a Range of Mo-99 Production Wastes - 19326

Tc-99m, the most commonly used radioisotope in nuclear medicine, is produced from the decay of Mo-99 which is predominantly produced in reactors by irradiation of enriched uranium targets. This results in three waste streams, one of which is a uranium filter cake containing fission products. This filter cake could itself be treated as a waste stream, or processed to retrieve the enriched uranium for further use, thus producing additional waste streams during reprocessing. All wastes are considered intermediate-level wastes and require an appropriate wasteform design for termination. ANSTO is collaborating with NECSA (Nuclear Energy Corporation of South Africa) to develop wasteform and processing solutions for the treatment of intermediate level wastes (ILW) from the production of Mo-99. The work is supported by the U.S. Department of Energy's (DOE) National Nuclear Security Administration (NNSA) with technical support from Argonne National Laboratory (ANL). Here we highlight some of the outcomes of this collaboration which has seen the investigation and performance testing of a range of waste treatment technologies. Among the treatment options for the various waste streams, including the enriched uranium filter cake itself, were glass, glass-ceramic, ceramic and geopolymer wasteforms. All wasteforms were characterized (e.g., by X-ray diffraction (XRD) and scanning electron microscopy (SEM)) and their performance characteristics, for example compressive strength and aqueous durability, determined by standard methods and compared. Consideration has been given to the most suitable, practical and cost effective solutions for the immobilization of these Mo-99 wastes. This paper will discuss the down-selection of wasteforms and process technologies for Mo-99 wastes and compare these with regards to wasteform performance and process technology readiness. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2005

Monitoring Agents for Assisting NASA Engineers with Shuttle Ground Processing

The Spaceport Processing Systems Branch at NASA Kennedy Space Center has designed, developed, and deployed a rule-based agent to monitor the Space Shuttle's ground processing telemetry stream. The NASA Engineering Shuttle Telemetry Agent increases situational awareness for system and hardware engineers during ground processing of the Shuttle's subsystems. The agent provides autonomous monitoring of the telemetry stream and automatically alerts system engineers when user defined conditions are satisfied. Efficiency and safety are improved through increased automation. Sandia National Labs' Java Expert System Shell is employed as the agent's rule engine. The shell's predicate logic lends itself well to capturing the heuristics and specifying the engineering rules within this domain. The declarative paradigm of the rule-based agent yields a highly modular and scalable design spanning multiple subsystems of the Shuttle. Several hundred monitoring rules have been written thus far with corresponding notifications sent to Shuttle engineers. This chapter discusses the rule-based telemetry agent used for Space Shuttle ground processing. We present the problem domain along with design and development considerations such as information modeling, knowledge capture, and the deployment of the product. We also present ongoing work with other condition monitoring agents.

Semmel, Glenn S.↗
DOE OSTI2020

Application of a Heat Integrated Post-combustion CO 2 Capture System with Hitachi Advanced Solvent into Existing Coal-Fired Power Plant (Final Technical Report)

The goal of this final project report is to comprehensively summarize the work conducted on project DE-FE0007395. In accordance with the Project Management Plan (PMP), Revision F dated 5/10/2019, and Statement of Project Objectives (SOPO) within, the University of Kentucky (UK) Center for Applied Energy Research (CAER) (Recipient) has successfully demonstrated a unique, versatile CO 2 capture system (CCS) using a heat integrated process combined with two-stage stripping for process intensification, heat recovery and demineralized (DM) water generation. This project involved the design, fabrication, installation, testing of and data analysis from the UK CAER 0.7 MWe small pilot scale CO2 capture process installed at Kentucky Utilities (KU) E.W. Brown Generating Station in Harrodsburg, KY. During each of the four project Budget Periods (BPs), UK CAER met all project deliverables, all project milestones, with National Energy Technology Laboratory (NETL) approved adjustments made to the campaign long-term hours during BP4. The CCS was constructed in modular skids. Two solvent campaigns were initially conducted; the first with a 30 wt% monoethanolamine (MEA) as a baseline, and the second with the Hitachi H3-1 advanced solvent. Additional tests were performed with two advanced solvents including CAER and Proprietary Solvent C. Short-period testing with a higher concentration of 40 wt% MEA was conducted to evaluate the potential saving with high alkalinity. From the various solvent campaigns, unique aspects of the UK CAER CCS technology, as well as its flexibility and versatility were experimentally validated and demonstrated. With respect to solvent evaluation efforts in identifying candidates with significant operational and capital cost savings potential, performance of solvents were evaluated to determine the energy requirements for regeneration; environmental impacts from secondary emissions and degradation products; degradation rates, solvent make-up rates and stability. The assessments were done from parametric tests that determined optimum operating conditions for the individual solvents to maximize process efficiency and minimize the parasitic load of the power plant, and from long term campaigns (1000 hours for 30 wt% MEA and 1000 hours for H3-1) which collectively informed the techno-economic analyses (TEA) of the process. The long term campaigns included corrosion studies which used three types of metal coupons in different sections of the process: (absorber, primary stripper, lean carbon-loaded and rich carbon loaded flow streams in process) to mimic heat and flow dynamics process equipment were exposed to. The estimated corrosion rates were used to elucidate corrosion mechanisms and to further guide process material selection for potential capital cost savings.

gas separation↗
DOE OSTI2020

Evaluating the Long-Term Ecological Responses to Riparian Ecosystem Restoration at the Fort Benning, Georgia Military Installation

Riparian and instream restorations are commonly implemented to improve ecosystem condition. However, very few restoration activities are monitored for multiple years to assess long-term effectiveness. At the Department of Defense’s Fort Benning Military Installation (FBMI) in west-central Georgia, two experimental restoration projects (ephemeral drainage revitalization and instream habitat augmentation [i.e., coarse woody debris (CWD) additions]) were implemented ~14 years ago with the goal of reducing environmental impacts of military training activities. A SERDP project found that instream restoration improved some aspects of ecosystem function and community structure but not others. The objective of the current study was to re-evaluate the effect of these restoration practices on stream ecosystem processes ~14 years after implementation.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2005

NESTA: NASA Engineering Shuttle Telemetry Agent

The Spaceport Processing Systems Branch at NASA Kennedy Space Center has developed and deployed an agent based tool to monitor the Space Shuttle's ground processing telemetry stream. The application, the NASA Engineering Shuttle Telemetry Agent, increases situational awareness for system and hardware engineers during ground processing of the Shuttle's subsystems. The agent provides autonomous monitoring of the telemetry stream and automatically alerts system engineers when predefined criteria have been met. Efficiency and safety are improved through increased automation. Sandia National Labs' Java Expert System Shell is employed as the rule engine. The shell's predicate logic lends itself well to capturing the heuristics and specifying the engineering rules of this spaceport domain. The declarative paradigm of the rule-based agent yields a highly modular and scalable design spanning multiple subsystems of the Shuttle. Several hundred monitoring rules have been written thus far with corresponding notifications sent to Shuttle engineers. This paper discusses the rule-based telemetry agent used for Space Shuttle ground processing and explains the problem domain, development of the agent software, benefits of AT technology, and deployment and sustaining engineering of the product.

Semmel, Glenn S.↗
DOE OSTI2025

Allometric Scaling of Hyporheic Respiration Across Basins in the Pacific Northwest United States

Abstract Hyporheic zones regulate biogeochemical processes in streams and rivers, but high spatiotemporal heterogeneity makes it difficult to predict how these processes scale from individual reaches to river basins. Recent work applying allometric scaling (i.e., power‐law relationships between size and function) to river networks provides a new paradigm for understanding cumulative hyporheic biogeochemical processes. We used previously published model predictions of reach‐scale hyporheic aerobic respiration to explore patterns in allometric scaling across two climatically divergent basins with differing characteristics in the Pacific Northwest, United States. In the model, hydrologic exchange fluxes (HEFs) regulate hyporheic respiration, so we examined how HEFs might influence allometric scaling of respiration. We found consistent scaling behaviors where HEFs were either very low or very high, but differences between basins when HEFs were moderate. Our findings provide initial model‐generated hypotheses for factors influencing allometric scaling of hyporheic respiration. These hypotheses can be used to optimize new data generation efforts aimed at developing predictive understanding of allometries that can, in turn, be used to scale biogeochemical dynamics across watersheds.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Unconventional Quantum Advantages for Computation (U-QuAC)

While quantum computing offers the promise of exponential advantages, limited quantum speedups are known, especially for practical applications. To open new avenues for quantum advantages, we propose Unconventional Quantum Advantages for Computation (U-QuACs), with respect to unconventional resources such as space (number of bits or quantum bits of memory required to solve a problem), accuracy of solution, communication, or energy consumption. We focus on space-efficient quantum algorithms, where we seek to design algorithms that solve a problem using much less space than the total size of the input. A natural setting in which space is critical is the streaming model of computation, where the input data arrives sequentially in pieces that must each be processed individually. Streaming is motivated by a variety of problems including analysis of internet traffic or social networks. We design the first exponential quantum space advantage for a natural streaming problem, which also constitutes the first quantum advantage for approximating a discrete optimization problem, albeit with respect to space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Surface Water Quality Data from Beaver-Impacted Streams; Trail Creek and East River, Colorado 2025

This data package contains surface water chemistry measurements collected in 2025 to evaluate how beaver damming and low-tech process-based stream restoration influence water quality and metal mobility in mountainous headwater systems of the Upper Colorado River Basin. Sampling was conducted at Trail Creek (Taylor Park watershed, Colorado), a tributary undergoing restoration through installation of low-tech process-based structures (i.e., beaver dam analogs), and at off-channel beaver ponds within the East River floodplain (East River watershed, Colorado). Samples were collected along longitudinal transects spanning upstream control reaches, beaver-influenced ponded reaches, and downstream segments. Additional samples were collected from near-surface pore waters within a beaver dam seepage face. The dataset includes concentrations of major and trace elements measured by inductively coupled plasma–mass spectrometry (ICP-MS) and inductively coupled plasma–optical emission spectrometry (ICP-OES), major anions measured by ion chromatography (IC), and dissolved organic carbon (DOC; reported as non-purgeable organic carbon, NPOC). Samples were size-fractionated at 0.45 micrometers (µm), 0.22 µm, and 0.02 µm to distinguish particulate (>0.45 µm), colloidal (0.22–0.02 µm), and dissolved (<0.02 µm) fractions. The data package consists of comma-separated value (.csv) files containing tabulated chemical concentration data, sample metadata (site identifiers, geographic coordinates, sampling dates, fraction type), and quality control flags. All files are provided in open, non-proprietary formats that can be accessed using standard data analysis software such as Microsoft Excel, R, Python, MATLAB, or other programs capable of reading .csv files. Units, detection limits, and analytical methods are documented in accompanying metadata files. The dataset is designed to support analyses of (1) how beaver impoundment and restoration structures alter elemental partitioning and transport, (2) the role of iron and organic carbon in mediating trace metal mobility, and (3) reach-scale changes in water quality across restoration gradients. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231. Part of this work was performed at SLAC Accelerator Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-76SF00515.

Anions↗
DOE OSTI2023

Composite media for non-oxidative ethane dehydrogenation, and related ethane activation systems and method of processing an ethane-containing stream

A composite media for non-oxidative C 2 H 6 dehydrogenation comprises an aluminosilicate zeolite matrix, and an EDH catalyst on one or more of an external surface of the aluminosilicate zeolite matrix and internal surfaces within pores of the aluminosilicate zeolite matrix. The EDH catalyst comprises one or more of Fe, Zn, Pt, Ga, alloys thereof, and oxides thereof. A C 2 H 6 activation system, and a method of processing a C 2 H 6 -containing stream are also described.

Wang, Lucun↗
DOE OSTI2026

Technoeconomic analysis and life cycle assessment of purification processes for captured CO 2 streams

Captured carbon dioxide (CO 2 ) streams contain impurities that must be removed to meet specifications for safe transport, storage, and utilization. Among these impurities, oxygen poses challenges due to its high reactivity and potential to cause corrosion, motivating stringent purity limits below 10 ppmv. Building on recent experimental demonstrations of catalytic oxygen removal using hydrogen (H 2 ), carbon monoxide (CO), methanol (CH 3 OH), and methane (CH 4 ) as reducing agents, this study presents a technoeconomic (TEA) and life cycle assessment (LCA) of these four catalytic purification pathways. Process flowsheets were developed and simulated in Aspen Plus for CO 2 streams representative of both low-temperature and high-temperature capture processes, with integrated heat recovery and energy optimization. Results showed that total purification costs were dominated by feedstock procurement and electricity consumption. Among the studied reducing agents, the CH 4 -assisted route achieved the lowest purification cost and highest CO 2 recovery. Sensitivity analyses showed that the H 2 route became competitive at H 2 prices below $\$$0.56/kg to $\$$0.84/kg, depending on the CO 2 feed temperature conditions. In conclusion, environmental impacts were primarily driven by indirect CO 2 emissions from raw material production and utility consumption.

CO2 pipeline specifications↗
DOE OSTI2021

Advanced Health Information Technology Analytic Framework and Application to Hazard Detection

Health Information Technology (HIT) aims to improve healthcare outcomes by organizing and analyzing various health-related data. With data accumulating at a staggering rate, the importance of real-time analytics has been increasing dramatically, shifting the focus of informatics from batch processing to streaming analytics. HIT is also facing unprecedented challenges in adapting to this new requirement and leveraging advanced IT technologies. This paper introduces a HIT data and compute platform that supports multi-granularity real-time analytics from heterogeneous data sources. The paper first identifies functional requirements and proposes a framework that satisfies the requirements using state-of-the-art big data technologies including Apache Kafka, Spark Structured Streaming Engine, and Delta Lake. To demonstrate its capability to support data analytics in multiple time granularities analytics, a statistical process control-based hazard detection algorithm has been implemented on top of the framework to detect unexpected hazards from order cancellation data of the Department of US Veterans Affairs (VA) in near real-time.

Kumar, Mohit↗