Search NASASearch

SEARCH · Search NASA

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Catalytic hydrogenation of HMF to BHMF over copper catalysts

2,5-Bis(hydroxymethyl)furan (BHMF) is a bio-derived building block for polyester production, obtained via the hydrogenation of 5-hydroxymethylfurfural (HMF). First-principles thermodynamic equilibrium calculations indicate that this reaction is not thermodynamically limited under relevant conditions (e.g., 100 °C and high H 2 partial pressure). In this work, crude HMF was employed as the feedstock for BHMF synthesis. Initially, acidic impurities and humins were removed from unrefined HMF through filtration using a packed bed of γ-alumina. A comprehensive study of the filtration process is presented, including filtration kinetics, breakthrough curve analysis, and mathematical modeling. The purified HMF was subsequently hydrogenated over a 10 wt% CuZrO 2 catalyst, using ethanol as the reaction solvent. Batch reactions were first performed for collection of kinetic data to guide the transition to continuous flow operation. Kinetic data was collected in a fixed bed reactor at varying contact time, time on stream, temperature, and HMF concentration. This data was used to develop a kinetic model for HMF hydrogenation. Maximum BHMF production rates were achieved at 130 °C, accompanied by minor formation of byproducts from BHMF ring-opening reactions. The BHMF selectivity was 100 % at 100 °C although with lower reaction rates. Furthermore, catalyst stability tests revealed a loss of up to 50 % in catalytic activity within the first 24 h, likely due to the adsorption of HMF-derived oligomers that are not easily removed by filtration.

Crude HMF filtration

Unconventional Quantum Advantages for Computation (U-QuAC)

While quantum computing offers the promise of exponential advantages, limited quantum speedups are known, especially for practical applications. To open new avenues for quantum advantages, we propose Unconventional Quantum Advantages for Computation (U-QuACs), with respect to unconventional resources such as space (number of bits or quantum bits of memory required to solve a problem), accuracy of solution, communication, or energy consumption. We focus on space-efficient quantum algorithms, where we seek to design algorithms that solve a problem using much less space than the total size of the input. A natural setting in which space is critical is the streaming model of computation, where the input data arrives sequentially in pieces that must each be processed individually. Streaming is motivated by a variety of problems including analysis of internet traffic or social networks. We design the first exponential quantum space advantage for a natural streaming problem, which also constitutes the first quantum advantage for approximating a discrete optimization problem, albeit with respect to space.

97 MATHEMATICS AND COMPUTING

Low flow characteristics for regulated and unregulated streams in North Carolina and prediction using climate signals

In this work, low flow statistics of regulated and unregulated streams in the state of North Carolina were updated with streamflow data through 2019. About 22% of the streams considered show a significant downward trend, but considerable low-frequency variability confounding trends. The relationship between regional index time series of groundwater-depth low flow and the Atlantic Multidecadal Oscillation (AMO) was examined to determine whether low-frequency climate modes can account for the long-term pattern in low flows. Consequently, a significant correlation was found between AMO and groundwater-depth low flow, such that positive AMO is associated with lower groundwater-depth low flow and vice versa, particularly for the Piedmont region. Predictive equations for annual low flows at the ecoregion level shows that springtime average streamflow and AMO were selected as the primary predictors of low flow for coastal and Piedmont regions, whereas springtime average streamflow and the November–December–January average Oceanic Niño Index were used as the primary predictors for the mountain region. The relative root mean square error (RMSE) of the disaggregated predictions to US Geologic Survey gauge locations was <23% at 79% of the stations, between 24% and 43% at 10% of the stations, and greater than 44% at 1% of the stations. The remaining 10% of stations showed large RMSEs. This latter percentage is characterized by smaller drainage basins and intermittent flows, suggesting the prediction models are not applicable to drainage basins smaller than roughly 20 square km and intermittent streams.

54 ENVIRONMENTAL SCIENCES

DeepLynx Ecosystem 2025

Poor data integration and governance continue to plague complex engineering projects, resulting in missed cost, schedule, and performance targets. Departments operate in isolated systems with manual data exchange, creating fragmented information that compounds errors and leads to significant delays and cost overruns. The DeepLynx ecosystem addresses these challenges through an open-source, modular data management platform that transforms fragmented project data into an integrated digital thread. Built on a federated microservice architecture, the ecosystem comprises seven specialized tools centered around DeepLynx Nexus, a unified data catalog with hierarchical organization and graph-based navigation capabilities. The ecosystem includes: DeepLynx Stream for real-time timeseries data ingestion from industrial sources; DeepLynx Ingest for governed data uploads with formal review workflows; DeepLynx Lattice for ontology-based entity and relationship extraction; DeepLynx Run for workflow orchestration and secure AI/ML compute; DeepLynx Visualize for 3D digital twin visualization; and DeepLynx Insight for AI-assisted document analysis with traceable, grounded responses. Deployable in cloud, on-premise, or hybrid environments using containerized Docker applications and Helm charts, the DeepLynx ecosystem provides flexible infrastructure that adapts to organizational requirements. By consolidating project data into a unified data lake with role-based access controls and OAuth2 authentication, DeepLynx enables digital thread and digital twin capabilities that improve decision-making, reduce risk, and support complex engineering workflows throughout the project lifecycle.

42 - ENGINEERING

Intelligent Experiments through Real-Time AI: Fast Data Processing and Autonomous Detector Control for High-Energy Nuclear Experiments

The aim of this project is to develop software and hardware for fast real-time data processing and autonomous detector control and calibration for the sPHENIX and the future EIC experiments. Below summarizes Georgia Tech team efforts in the past year: 1. We developed a real-time clustering algorithm and FPGA-based pipeline architecture for processing fired pixel data from ALPIDE sensors in sPHENIX experiments. Our Columnar Clustering Co-Design introduces a hardware-aware, stream-friendly approach that segments pixel data by column pairs using a Column Pair Clustering (CPC) strategy, followed by Cluster Stitching to merge adjacent subclusters. Implemented in Vitis HLS, the pipeline comprises five stages—read-in, subclustering, stitching, analysis, and write-out—connected by tagged HLS streams with custom end-of-event signaling for robust synchronization. We designed a pipelined dataflow model optimized for throughput, low latency, and minimal buffering, enabling scalable clustering across events of arbitrary size. Our system maintains spatial precision via center-of-mass and shape key extraction and efficiently handles edge cases such as fragmented or nested clusters. Compared against DBSCAN in both software and hardware, our approach demonstrates competitive performance under FPGA constraints. 2. We also conducted a comprehensive algorithm-to-hardware co-design of connected component analysis tailored for sPHENIX experiments, focusing on real-time, low-latency processing using FPGAs and High-Level Synthesis (HLS). Starting from a Python-based particle tracking pipeline, the team translated the core logic—graph traversal via DFS and Union-Find—into an HLS-compatible C++ model, replacing dynamic memory and recursion with static arrays and pipelined control flow. The final design includes a fully streamed and dataflow-compatible Union-Find kernel optimized across five iterations, incorporating loop pipelining, array partitioning, AXI/FIFO interface tuning, and function flattening. Experimental results show up to 14.8× speedup over the CPU baseline, reducing per-graph latency to 1.58 μs and demonstrating strong resource efficiency with only ~7k LUTs and zero BRAM usage. The design maintains functional correctness against the Python reference using a Python-based C-simulation framework and Mean Squared Error metrics. This work validates the potential of HLS-driven FPGA designs for edge-level HEP data acquisition, laying a scalable foundation for future integration with real-time detector pipelines and multi-graph processing systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS

Steel Creek, Pen Branch, and D-Area Watershed Stream Gauging Stations

A network of stream gauging systems were installed in the Steel Creek, Pen Branch, D-Area Discharge Canal, and the D006 Stream in support of the groundwater modeling efforts for the P-Area Groundwater Operable Unit (OU); Chemical, Metals, and Pesticides (CMP) Pits OU; and the D-Area Watershed, respectively. Each location is monitored by a MACE Floseries3 FloPro data logger and a MACE doppler ultrasonic area/velocity sensor. Each stream gauging system is powered by an internal 12-volt battery supplied by a solar panel with a trickle charger. Information collected by each data logger is logged internally and telecommunicated via a cellular network to an online server for real time analysis and monitoring. The MACE doppler ultrasonic area/velocity sensor can measure stream depth and velocities to give output values of flow rates, total flow, net flow, and volumes. The water depth is measured by a ceramic pressure transducer located on the top of the sensor. The velocity is measured by a continuous wave doppler sensor to give an average velocity across the whole stream profile. This report discusses the equipment and methods used to install continuous stream gauging stations and provides a summary of data collected through FY2025.

54 ENVIRONMENTAL SCIENCES

Evaluation of New Additions to OLI Software in Predicting Mercuric and Mercurous Species in Liquid Waste Operations

Speciation of mercury during the pretreatment steps of tank waste processing is critical to successful mercury removal prior to vitrification during Liquid Waste Operations (LWO) at SRS. OLI software has been used to predict mercury speciation and activity throughout LWO. The OLI software operates based on a thermodynamic framework called the Mixed Solvent Electrolyte (MSE) framework. The MSE framework allows prediction in theoretically infinitely dilute to concentrated mixtures (e.g., purely solute solutions). Before modification to the MSE framework databanks, certain critical mercury species were missing in the MSE databank, and some thermodynamic data needed to be updated for the OLI software to accurately predict mercury chemical species in SRS waste tanks. To better reflect streams across LWO, new mercury species were integrated into the MSE database. To evaluate the changes to the OLI MSE framework per the Technical Task Request (TTR) and the Task Technical and Quality Assurance Plan (TTQAP), waste stream compositions from Tanks 38, 43, and Tank 50 decontaminated salt solution (DSS) were used as model inputs. Models were developed and executed using both the old and new databases. Compositional analyses from caustic Tank 50 DSS and caustic Tanks 38 and 43 were used as the input streams. These streams represent the most comprehensive chemical data sets where both mercury and tank constituents were measured together. Results for Tank 50 DSS predict HgO as the predominant species in both databases. Both methyl and dimethyl Hg species are present when the new database is ‘on’ and are not predicted with the new database turned ‘off’. The new database predicts a greater amount of HgO and a greater fraction of it in the solid phase. Pourbaix diagrams (potential vs. pH) generated for each Tank 50 DSS were identical regardless of which database was used. Elemental Hg and HgO were predicted in the water stable region under basic conditions. Tanks 38 and 43 follow similar trends as the Tank 50 DSS models. Unlike Tanks 38 and 50 DSS, the Tank 43 Pourbaix plot shows a region of stability for an aqueous HgOHCO3 - species between approximately pH 7-11. In all streams, when MeHg+ is included in the inputs, the new database predicts aqueous MeHgOH as the dominant species. If elemental or dimethyl mercury is in the waste stream, the new database model predicts they are unchanged and remain in those states and quantities. Additionally, the total mercury values are reported for both the measured input data and the OLI output data for all considered tanks. The summary indicates that the percentage error between the measured and calculated values is less than 1% in all cases The reconciliations and generation of the Pourbaix diagrams for Tank 50 DSS took approximately ten times longer with the new database ‘on’. In addition, over the course of that time, models with the new database ‘on’ were more likely to crash or display an error. Some modest performance improvements were noted when modeling with an i7 processor versus an i5. An example error is found in Appendix A. Furthermore, Appendix B provides V&V for two chemical systems analyzed with the OLI software, results were satisfactory. It is recommended to utilize the new databases (i.e., HCO.ddb and SR-Hg.ddb) in future Savannah River Mission Completion applications of OLI to represent pseudo steady-state. Furthermore, the integration and utilization of the new databases (i.e., HCO.ddb and SR-Hg.ddb) in modeling applications (e.g., Aspen) is also recommended.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W

Rapid organic carbon spiraling in a headwater stream linked with streamflow, biogeochemistry, and canopy phenology

Headwater streams are abundant worldwide and important to global biogeochemical cycles, serving as critical processors and transporters of C. C spiraling is a useful way to understand the retention and mineralization of organic C (OC) in streams. However, analyses of seasonal and interannual variability in OC spiraling are currently limited. In this study, we aimed to understand the temporal patterns and driving mechanisms of OC spiraling, which will inform our understanding of future OC changes under climate change. We used 7 y of daily data in a small headwater stream (Walker Branch, Tennessee, USA) to assess seasonal and interannual variability in OC spiraling length (S OC ) and mineralization velocity (v fOC ), as well as their potential related variables. On average, S OC in Walker Branch was ~10× shorter than in previously studied small streams, indicating strong connections between the water column and the benthic environment where OC mineralization mostly takes place. OC spiraling was faster during the more biologically active periods of spring and autumn compared with more elongated OC spiraling in summer and winter, when OC retention was lower and downstream transport was higher. Gross primary production (GPP) was most strongly related to S OC and v fOC . Photosynthetically active radiation (PAR) and NO 3 − were also positively and negatively related to v fOC , respectively. Trends toward earlier and longer canopy cover and reduced GPP and PAR may result in longer S OC and slower v fOC , reducing localized instream processing of OC and potentially shunting more OC downstream. However, long-term observations indicate reduced NO 3 − at Walker Branch, suggesting opposing effects to those of GPP and PAR, leading to faster v fOC and greater OC retention. Time-series analyses of OC spiraling in streams can enhance our understanding of current and future responses of OC processing and downstream transport to climate change, as well as implications for downstream OC dynamics.

biological activity

Constraining the Galactic bar using the M92 stellar stream

Stellar streams are excellent probes of the gravitational potential in which they evolve. In the Milky Way (MW), globular cluster (GC) streams are routinely used to infer properties about time-dependent perturbations of the underlying potential. This implies that streams with Galactocentric radii small enough to be perturbed by the MW bar should offer constraints on it, such as its pattern speed, which currently has a wide range of values reported in the literature and is important when studying stellar kinematics. The GC M92 has a small pericentre and should be affected by the bar. It has a diffuse stellar stream, but confirming stream members has previously been hindered by a lack of spectroscopic data. In this paper, we use Dark Energy Spectroscopic Instrument (DESI) observations together with photometric and astrometric data to obtain spectroscopic members of the M92 stream for the first time. We identify a clear spatial distribution and gradients in distance moduli, proper motions, and radial velocities that confirm the stream's existence. We compare the observed stream to mock streams generated in different barred potentials and estimate the MW bar's pattern speed $Ω= 29.1^{+0.7}_{-0.4}$ km s$^{-1}$ kpc$^{-1}$ and $\dot Ω= 0.7^{+3.5}_{-2.3}$ km s$^{-1}$ kpc$^{-1}$ Gyr$^{-1}$. This is the first time a stellar stream is used to probabilistically infer these bar properties, and it opens up an exciting realm of inner Galactic potential characterisation using stellar streams.

Byström, Amanda [Edinburgh U., Inst. Astron.] (ORC

Synoptic NO3 in Slate River Watershed, Colorado (2022)

The synoptic nitrate (NO3) dataset in the Slate River Watershed, Colorado consists of NO3 data collected at 19 locations three times during the summer of 2022. Stream samples were collected in early summer (early July), mid summer (late August), and late summer (late September). The samples include mainstem, tributary, and point source input water samples. These data were collected to evaluate spatiotemporal variability in stream NO3 during the summer, and evaluate anthropogenic controls on stream NO3 dynamics. This data package contains: (1) a csv of all NO3 samples and (2) a csv of locations for each sampling site. The dataset additionally includes a file-level metadata (flmd.csv) file that lists each file contained in the dataset with associated metadata; and a data dictionary (dd.csv) file that contains column/row headers used throughout the files along with a definition, units, and data type. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

EARTH SCIENCE > BIOSPHERE > ECOSYSTEMS > FRESHWATE

Historic climate, cosmogenic 10Be, denudation-rate, and geospatial datasets from the Pikes Peak region, Colorado, USA

This data package contains geographic information system (GIS) layers and tabular datasets associated with the study of elevation-dependent denudation rates on Pikes Peak in the Front Range of the Rocky Mountains, Colorado, USA. The package includes GIS layers used to produce the study-area map, including sample locations, sample watershed boundaries, the Pikes Peak batholith, Pleistocene glacier extent, weather station locations, and elevation and hillshade rasters, together with comma-separated value (CSV) tables and matching CSV data dictionaries. These mapped layers provide the geographic framework for interpreting denudation patterns across the Pikes Peak region and for relating sample locations to watershed geometry, bedrock setting, glacial history, and nearby climate stations. The first group of tables reports climate and geospatial context for the study area. These files include station-based temperature and precipitation data used to characterize elevational gradients in mean annual climate and monthly climate seasonality, sample locations, denudation-rate and topographic metrics, fixed frost-cracking model parameters, frost-cracking intensity and precipitation-frequency metrics, and stream-power inversion results. Together, these data provide the basis for evaluating how denudation varies with elevation, climate, and landscape form across sampled catchments on Pikes Peak. The second group of tables reports cosmogenic nuclide and erosion-model results used in the denudation analysis. Included files contain accelerator mass spectrometry (AMS) measurements for in situ-produced cosmogenic beryllium-10 (10Be), including sample identifiers, measured 10Be:9Be ratios, analytical uncertainties, carrier mass, quartz mass, blank corrections, blank-group statistics, and calculated 10Be concentrations and uncertainties. Additional tables summarize stream-power-law inversion results for sampled catchments, including optimized model parameters, predicted erosion rates, residual metrics, channel-pixel counts, and convergence status, as well as regression equations and summary statistics used to evaluate relationships among elevation, climate, frost cracking, precipitation forcing, and denudation rate. The package contains GIS files, comma-separated value files (.csv), Microsoft Excel files (.xlsx), CSV data dictionaries, a file-level metadata table, and a readme text file.

10Be cosmogenic nuclides

Hydrologic Model Data for the East Fork Poplar Creek Watershed Simulated with the Advanced Terrestrial Simulator (ATS): Streamflow and Network Expansion–Contraction Dynamics

This dataset supports hydrologic modeling and stream network expansion–contraction analysis for the East Fork Poplar Creek (EFPC) Watershed in Tennessee. It includes a Jupyter notebook for model setup, model configuration files, simulation outputs, and derived products used to evaluate model performance and investigate stream dynamics under varying hydrologic conditions. The dataset was generated using the Watershed Workflow Python package and the Advanced Terrestrial Simulator (ATS), enabling integrated surface–subsurface hydrologic simulations using a stream-aligned mesh. Outputs include high-resolution time series of streamflow, active network length, water table depth, and related hydrologic variables. Also included are spatially explicit stream persistency indices and classifications of reaches as perennial or non-perennial. These data facilitate reproducibility and support further research on stream intermittency and variability in network extent.The model data archive is organized in following directories:1) model_setup_inputsContains the Watershed Workflow Jupyter notebooks (accessed through any open source code editor), selected input datasets, and resulting ATS input files, including XML files (access through any open source code editor), computational mesh (.exo files can be viewed using Paraview), and meteorological forcing files (.h5 files can be accessed through h5py python package and HDFView open source software). 2) model_outputsIncludes ATS simulation outputs relevant to this study. Time series of spatially integrated or averaged variables (e.g., streamflow, water table depth) are provided as CSV files. Select spatial fields (e.g., ponded depth and water table depth) are saved as pickled Python objects to reduce file size, and can be accessed through pickle package in Python. Key geometry objects from Watershed Workflow—such as the surface mesh and river tree—are also included to support analysis of streamflow persistency and expansion–contraction dynamics. These files can also be accessed through Watershed Workflow Python package.3) model_evaluationProvides observed streamflow time series and field survey-based flow regime classifications used to evaluate model performance. Jupyter notebooks for processing ATS outputs and comparing model predictions with observations to build confidence in the model prior to scientific analysis are also included.4) Q_L_relationshipsContains workflows for generating time series of discharge, active network length, and related hydrologic variables used in the stream network expansion–contraction analysis. Includes routines for delineating baseflow-dominated periods. For each catchment, notebooks and processed data (as pickled DataFrames accessed through Pandas Python package) are provided. 5) figure_scriptsProvides the Jupyter notebooks used to generate the figures presented in the paper.

54 ENVIRONMENTAL SCIENCES

HydroBio: Hydropower Capacity and Freshwater Biodiversity in Conterminous United States Sub-basins

This dataset summarizes existing and potential hydropower capacity and freshwater biodiversity at the sub-basin level throughout the conterminous United States (CONUS). It contains descriptive information regarding each sub-basin (e.g., 8-digit hydrologic unit code identifier, name, states, and size) along with sub-basin-level summaries of: 1) existing hydropower capacity (MW), 2) potential nominal non-powered dam (NPD) capacity (MW), 3) potential capacity of new stream reach development (NSD) (MW), and 4) freshwater biodiversity, including the total richness of fish, crayfish, and mussels and metrics that account for how rare and threatened those species tend to be. Hydropower data were obtained from Oak Ridge National Laboratory data resources (Existing Hydropower Assets, Non-Powered Dam Technical Potential, and New Stream Reach Development). Freshwater biodiversity data were obtained from NatureServe. Sub-basin characteristic information was obtained from the United States Geological Survey. Additionally, long data that provide lists of unique elements within each sub-basin for each constituent data resource (e.g., NatureServe, Existing Hydropower Assets) are provided to enhance dataset utility for users. The dataset provides, for the first time, a national-level assessment of existing and potential hydropower capacity in the context of freshwater biodiversity and is a valuable resource for stakeholders tasked with providing affordable, reliable energy to the American public while maintaining or enhancing invaluable freshwater resources. The dataset contains six data files in comma separated (*.csv) format that are within a zipped file.

Bozeman, Bryan [Oak Ridge National Laboratory (ORN

Williston Basin CORE-CM Initiative Final Report

The University of North Dakota Energy & Environmental Research Center (EERC) is leading the Williston Basin Carbon Ore, Rare Earth, and Critical Minerals (CORE-CM) Initiative to drive the expansion and transformation of coal and coal-based resource usage within the Williston Basin to produce rare-earth elements (REEs), CMs, and nonfuel carbon-based products (CBPs). This project is the first phase in a long-term program and set the stage for future work by assessing resource, market, technology, and infrastructure knowledge; identifying knowledge gaps; developing a series of plans to be carried out in future work; and initiating stakeholder engagement. Composed of several tasks, the project sought to identify, characterize, and assess several necessary aspects vital to make this future work a reality. The project’s fundamental task was to characterize the Williston Basin CORE-CM resources. Over 2500 samples from multiple sources were utilized to begin the assessment. Several locations were identified in western North Dakota where sample analysis identified the total REE (TREE) concentration as being over 500 parts per million (ppm), which is at a concentration level that would be suitable to consider for mining and extraction. Current operating coal mines have sufficient concentrations of TREEs for consideration. However, the current data across the basin are still not adequate to fully characterize REE and CM content nor give reliable estimates of the total resource potential. Waste stream reuse was also considered, and several streams were identified which ranged from potential energy sources to chemicals to material wastes. This includes streams that result from oil and gas production. These streams are not fully characterized, and further data are needed before they can be accurately assessed. Infrastructure within the Williston Basin is suitable for expansion of a new industry to mine, extract, and concentrate REEs and CMs. The development of this industry will not only preserve many existing jobs in the coal-mining industry but produce many new jobs. The supply chain for REEs and CMs is currently controlled outside of the United States in nations such as China, but the potential to develop the supply chain within the basin is considered possible. Processing of the mined materials for REEs and CMs needs further research. The technology and knowhow exist outside of the United States, and within the country much of the knowledge has been lost and must be regained. To develop the supply chain and regain lost processing technology, the creation of technology innovation centers (TICs) is crucial. The Williston Basin contains several similar centers and entrepreneurial assistance for other industries that can be applied in the development of REE and CM innovation centers. Education to develop the new skill sets required is also needed. Outreach is important for the development of the REE and CM industry within the basin. Understanding throughout federal and state governments, state agencies, industry, and resource end users is vital for the industry to form and grow. Through this project these groups have been contacted through bulletins, presentations, webinars, and annual symposiums. The report is a summary of the work conducted and throughout refers to a series of appendixes which contain more thorough and specific information about each section.

01 COAL, LIGNITE, AND PEAT

Refining HPCToolkit for application performance analysis at exascale

As part of the US Department of Energy’s Exascale Computing Project (ECP), Rice University has been refining its HPCToolkit performance tools to better support measurement and analysis of applications executing on exascale supercomputers. To efficiently collect performance measurements of GPU-accelerated applications, HPCToolkit employs novel non-blocking data structures to communicate performance measurements between tool threads and application threads. To attribute performance information in detail to source lines, loop nests, and inlined call chains, HPCToolkit performs parallel analysis of large CPU and GPU binaries involved in the execution of an exascale application to rapidly recover mappings between machine instructions and source code. To analyze terabytes of performance measurements gathered during executions at exascale, HPCToolkit employs distributed-memory parallelism, multithreading, sparse data structures, and out-of-core streaming analysis algorithms. To support interactive exploration of profiles up to terabytes in size, HPCToolkit’s hpcviewer graphical user interface uses out-of-core methods to visualize performance data. The result of these efforts is that HPCToolkit now supports collection, analysis, and presentation of profiles and traces of GPU-accelerated applications at exascale. These improvements have enabled HPCToolkit to efficiently measure, analyze and explore terabytes of performance data for executions using as many as 64K MPI ranks and 64K GPU tiles on ORNL’s Frontier supercomputer. HPCToolkit’s support for measurement and analysis of GPU-accelerated applications has been employed to study a collection of open-science applications developed as part of ECP. This paper reports on these experiences, which provided insight into opportunities for tuning applications, strengths and weaknesses of HPCToolkit itself, as well as unexpected behaviors in executions at exascale.

Adhianto, Laksono

A 28 nm multiply-accumulate ASIC architecture for on-chip data compression in MHz frame rate X-ray and electron pixel detectors

Modern X-ray detector systems urgently require compact, efficient, and fast data compression schemes to handle the transmission of big data from pixel arrays, enabling frame rates in the MHz regime. Here, in this work, a data compression ASIC that implements a streaming fixed-length lossy compression scheme is introduced and analyzed, proving the feasibility and benefits of on-chip compression. The compression scheme utilizes a vector matrix product logic, which performs a number of floating-point multiplications, additions, and accumulations. The logic is verified, synthesized, and shown to fit in the area resource available for the X-ray detector under study, which comprises 192 × 168 pixels each of 12-bit width, and having a total area of 20 mm× 20 mm, about 2 mm× 20 mm of which are available for the digital logic. Several system architectures, precisions, and compression ratios ranging from 100 to 250 were analyzed to pave the way for on-chip fixed-length compression (e.g., principal component analysis, singular value decomposition) and data reduction (e.g., azimuthal integration) for X-ray and electron detectors.

Data compression

rustpix

rustpix is a high-performance, open-source Rust library with first-class Python bindings (via PyO3) for processing pixel-detector data in neutron imaging. It targets time-stamping detectors such as Timepix3 (TPX3) at ORNL's Spallation Neutron Source (VENUS beamline), where each detected neutron deposits charge across a cluster of pixels within a very high-rate event stream (96M+ hits/sec). rustpix parses TPX3 event data in parallel using memory-mapped I/O, offers four interchangeable clustering algorithms (ABS adjacency-based search, DBSCAN, graph/union-find connected components, and a parallel grid method), and extracts weighted, super-resolved centroids to produce neutron-event lists. A streaming architecture lets it process files larger than available memory. rustpix is distributed as a pip-installable Python package (with NumPy integration), Rust crates, a command-line tool, and an interactive GUI; it writes HDF5, Apache Arrow, and CSV; and it is designed to extend to TPX4 and other detector types. Released as open-source under the MIT License.

Zhang, Chen [Oak Ridge National Laboratory (ORNL),

nng_stream

This project uses the NanoMsg next-generation library to receive data, temporarily store in an in-memory buffer, and then send it out. The buffer handles a high-bandwidth stream from multiple producers (sources) to multiple consumers (sinks).

Rogers, DavidM [Oak Ridge National Laboratory (ORN