Search NASA⌕ Search

SEARCH · Search NASA

Results for “stream networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Resilient Distributed Field Estimation

We study resilient distributed field estimation under measurement attacks. A network of agents or devices measures a large, spatially distributed physical field parameter. An adversary arbitrarily manipulates the measurements of some of the agents. Each agent's goal is to process its measurements and information received from its neighbors to estimate only a few specific components of the field. Here, we present SAFE, the saturating adaptive field estimator, a consensus + innovations distributed field estimator that is resilient to measurement attacks. Under sufficient conditions on the compromised measurement streams, the physical coupling between the field and the agents' measurements, and the connectivity of the cyber communication network, SAFE guarantees that each agent's estimate converges almost surely to the true value of the components of the parameter in which the agent is interested. Finally, we illustrate the performance of SAFE through numerical examples.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2006

Interconnect Performance Evaluation of SGI Altix 3700 BX2, Cray X1, Cray Opteron Cluster, and Dell PowerEdge

We study the performance of inter-process communication on four high-speed multiprocessor systems using a set of communication benchmarks. The goal is to identify certain limiting factors and bottlenecks with the interconnect of these systems as well as to compare these interconnects. We measured network bandwidth using different number of communicating processors and communication patterns, such as point-to-point communication, collective communication, and dense communication patterns. The four platforms are: a 512-processor SGI Altix 3700 BX2 shared-memory machine with 3.2 GB/s links; a 64-processor (single-streaming) Cray XI shared-memory machine with 32 1.6 GB/s links; a 128-processor Cray Opteron cluster using a Myrinet network; and a 1280-node Dell PowerEdge cluster with an InfiniBand network. Our, results show the impact of the network bandwidth and topology on the overall performance of each interconnect.

Fatoohi, Rod↗
DOE OSTI2020

Untangling the Galaxy. II. Structure within 3 kpc

We present the results of the hierarchical clustering analysis of the Gaia DR2 data to search for clusters, comoving groups, and other stellar structures. The current paper builds on the sample from the previous work, extending it in distance from 1 to 3 kpc and increasing the number of identified structures up to 8292. To aid in the analysis of the population properties, we developed a neural network called Auriga to robustly estimate the age, extinction, and distance of a stellar group based on the input photometry and parallaxes of the individual members. We apply Auriga to derive the properties of not only the structures found in this paper, but also previously identified open clusters. Through this work, we examine the temporal structure of the spiral arms. Specifically, we find that the Sagittarius Arm has moved by >500 pc in the last 100 Myr and the Perseus Arm has been experiencing a relative lull in star formation activity over the last 25 Myr. We confirm the findings of the previous paper on the transient nature of the spiral arms, with the timescale of transition of a few 100 Myr. Finally, we find a peculiar ~1 Gyr old stream of stars that appears to be heliocentric. Its origin is unclear.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Imputation of urban environmental sensor data using gated attention bidirectional long short-term memory (GA-BiLSTM): methods, performance, and implications

Urban environmental monitoring networks frequently encounter significant data gaps due to sensor malfunctions, environmental disturbances, and communication failures. Reliable approaches to address these gaps are essential for ensuring the continuity and quality of environmental data streams. In this study, we developed a gated attention bidirectional long short-term memory (GA-BiLSTM) model to impute missing data in a dense urban monitoring network. Using observations from the CROCUS network in Chicago, we evaluated GA-BiLSTM against widely used approaches (XGBoost and K-nearest neighbors) under scenarios of both short-term intermittent gaps and prolonged outages. GA-BiLSTM consistently outperformed comparative methods, particularly during extended outages of up to ten days, demonstrating its ability to capture spatiotemporal dependencies across sensor nodes. Beyond performance metrics, feature importance and spatial network analyses highlighted the unexpected but critical predictive role of peripheral rural nodes, underlining their strategic value for maintaining robust urban monitoring systems. These results emphasize that advanced imputation methods can substantially improve the reliability of environmental monitoring networks and support more resilient data infrastructures for urban sustainability.

Data imputation↗
NASA NTRS1988

Cosmic string induced peculiar velocities

This paper considers the scenario of a flat universe with a network of heavy cosmic strings as the primordial fluctuation spectrum. The joint probability of finding streaming velocities of at least 600 km/s on large scales and local peculiar velocities of less than 800 km/s is calculated. It is shown how the effects of loops breaking up and being born with a spectrum of sizes can be estimated. It is found that to obtain large-scale streaming velocities of at least 600 km/s, it is necessary that either a large value for beta G mu exist or the effect of loop fissioning and production details be considerable.

Van Dalen, Anthony↗
NASA NTRS2006

Advanced Visualization of Experimental Data in Real Time Using LiveView3D

LiveView3D is a software application that imports and displays a variety of wind tunnel derived data in an interactive virtual environment in real time. LiveView3D combines the use of streaming video fed into a three-dimensional virtual representation of the test configuration with networked communications to the test facility Data Acquisition System (DAS). This unified approach to real time data visualization provides a unique opportunity to comprehend very large sets of diverse forms of data in a real time situation, as well as in post-test analysis. This paper describes how LiveView3D has been implemented to visualize diverse forms of aerodynamic data gathered during wind tunnel experiments, most notably at the NASA Langley Research Center Unitary Plan Wind Tunnel (UPWT). Planned future developments of the LiveView3D system are also addressed.

Schwartz, Richard J.↗
NASA NTRS2012

Digital Motion Imagery, Interoperability Challenges for Space Operations

With advances in available bandwidth from spacecraft and between terrestrial control centers, digital motion imagery and video is becoming more practical as a data gathering tool for science and engineering, as well as for sharing missions with the public. The digital motion imagery and video industry has done a good job of creating standards for compression, distribution, and physical interfaces. Compressed data streams can easily be transmitted or distributed over radio frequency, internet protocol, and other data networks. All of these standards, however, can make sharing video between spacecraft and terrestrial control centers a frustrating and complicated task when different standards and protocols are used by different agencies. This paper will explore the challenges presented by the abundance of motion imagery and video standards, interfaces and protocols with suggestions for common formats that could simplify interoperability between spacecraft and ground support systems. Real-world examples from the International Space Station will be examined. The paper will also discuss recent trends in the development of new video compression algorithms, as well likely expanded use of Delay (or Disruption) Tolerant Networking nodes.

Grubbs, Rodney↗
NASA NTRS2015

Use of Generalized Fluid System Simulation Program (GFSSP) for Teaching and Performing Senior Design Projects at the Educational Institutions

This paper describes the experience of the authors in using the Generalized Fluid System Simulation Program (GFSSP) in teaching Design of Thermal Systems class at University of Alabama in Huntsville. GFSSP is a finite volume based thermo-fluid system network analysis code, developed at NASA/Marshall Space Flight Center, and is extensively used in NASA, Department of Defense, and aerospace industries for propulsion system design, analysis, and performance evaluation. The educational version of GFSSP is freely available to all US higher education institutions. The main purpose of the paper is to illustrate the utilization of this user-friendly code for the thermal systems design and fluid engineering courses and to encourage the instructors to utilize the code for the class assignments as well as senior design projects. The need for a generalized computer program for thermofluid analysis in a flow network has been felt for a long time in aerospace industries. Designers of thermofluid systems often need to know pressures, temperatures, flow rates, concentrations, and heat transfer rates at different parts of a flow circuit for steady state or transient conditions. Such applications occur in propulsion systems for tank pressurization, internal flow analysis of rocket engine turbopumps, chilldown of cryogenic tanks and transfer lines, and many other applications of gas-liquid systems involving fluid transients and conjugate heat and mass transfer. Computer resource requirements to perform time-dependent, three-dimensional Navier-Stokes computational fluid dynamic (CFD) analysis of such systems are prohibitive and therefore are not practical. Available commercial codes are generally suitable for steady state, single-phase incompressible flow. Because of the proprietary nature of such codes, it is not possible to extend their capability to satisfy the above-mentioned needs. Therefore, the Generalized Fluid System Simulation Program (GFSSP1) has been developed at NASA Marshall Space Flight Center (MSFC) as a general fluid flow system solver capable of handling phase changes, compressibility, mixture thermodynamics and transient operations. It also includes the capability to model external body forces such as gravity and centrifugal effects in a complex flow network. The objectives of GFSSP development are: a) to develop a robust and efficient numerical algorithm to solve a system of equations describing a flow network containing phase changes, mixing, and rotation; and b) to implement the algorithm in a structured, easy-to-use computer program. The analysis of thermofluid dynamics in a complex network requires resolution of the system into fluid nodes and branches, and solid nodes and conductors as shown in Figure 1. Figure 1 shows a schematic and GFSSP flow circuit of a counter-flow heat exchanger. Hot nitrogen gas is flowing through a pipe, colder nitrogen is flowing counter to the hot stream in the annulus pipe and heat transfer occurs through metal tubes. The problem considered is to calculate flowrates and temperature distributions in both streams. GFSSP has a unique data structure, as shown in Figure 2, that allows constructing all possible arrangements of a flow network with no limit on the number of elements. The elements of a flow network are boundary nodes where pressure and temperature are specified, internal nodes where pressure and temperature are calculated, and branches where flowrates are calculated. For conjugate heat transfer problems, there are three additional elements: solid node, ambient node, and conductor. The solid and fluid nodes are connected with solid-fluid conductors. GFSSP solves the conservation equations of mass and energy, and equation of state in internal nodes to calculate pressure, temperature and resident mass. The momentum conservation equation is solved in branches to calculate flowrate. It also solves for energy conservation equations to calculate temperatures of solid nodes. The equations are coupled and nonlinear; therefore, they are solved by an iterative numerical scheme. GFSSP employs a unique numerical scheme known as simultaneous adjustment with successive substitution (SASS), which is a combination of Newton-Raphson and successive substitution methods. The mass and momentum conservation equations and the equation of state are solved by the Newton-Raphson method while the conservation of energy and species are solved by the successive substitution method. GFSSP is linked with two thermodynamic property programs, GASP2 and WASP3 and GASPAK4, that provide thermodynamic and thermophysical properties of selected fluids. Both programs cover a range of pressure and temperature that allows fluid properties to be evaluated for liquid, liquid-vapor (saturation), and vapor region. GASP and WASP provide properties of 12 fluids. GASPAK includes a library of 36 fluids. GFSSP has three major parts. The first part is the graphical user interface (GUI), visual thermofluid analyzer of systems and components (VTASC). VTASC allows users to create a flow circuit by a 'point and click' paradigm. It creates the GFSSP input file after the completion of the model building process. GFSSP's GUI provides the users a platform to build and run their models. It also allows post-processing of results. The network flow circuit is first built using three basic elements: boundary node, internal node, and branch.

Majumdar, A. K.↗
NASA NTRS1982

NASA Deep Space Network performance analysis

As part of the Control Center Operations, the Performance Analysis Group has responsibility for validating all committed Deep Space Network support to the various user projects. The group provides the Network with the analysis support necessary to assure that actual Network performance meets or exceeds committed levels throughout the mission. Validation is carried out as the signals are received in real time by monitoring the data streams and comparing the data quality against predicted values and established performance standards and tolerance limits. An important function is to generate the predictions used to point the antennas, acquire the radio frequency, and validate the monitored Network performance. Another function is anomaly investigation. An integral part of validation is the support task of generating and maintaining performance standards and tolerance limits for each flight project. The Performance Analysis Group is broadly divided into two subgroups: real-time personnel and nonreal-time personnel.

Bartok, C. D.↗
DOE OSTI2021

On the Representation of Hyporheic Exchange in Models for Reactive Transport in Stream and River Corridors

Efforts to include more detailed representations of biogeochemical processes in basin-scale water quality simulation tools face the challenge of how to tractably represent mass exchange between the flowing channels of streams and rivers and biogeochemical hotspots in the hyporheic zones. Multiscale models that use relatively coarse representations of the channel network with subgrid models for mass exchange and reactions in the hyporheic zone have started to emerge to address that challenge. Two such multiscale models are considered here, one based on a stochastic Lagrangian travel time representation of advective pumping and one on multirate diffusive exchange. The two models are formally equivalent to well-established integrodifferential representations for transport of non-reacting tracers in steady stream flow, which have been very successful in reproducing stream tracer tests. Despite that equivalence, the two models are based on very different model structures and produce significantly different results in reactive transport. In a simple denitrification example, denitrification is two to three times greater for the advection-based model because the multirate diffusive model has direct connections between the stream channel and transient storage zones and an assumption of mixing in the transient storage zones that prevent oxygen levels from dropping to the point where denitrification can progress uninhibited. By contrast, the advection-based model produces distinct redox zonation, allowing for denitrification to proceed uninhibited on part of the hyporheic flowpaths. These results demonstrate that conservative tracer tests alone are inadequate for constraining representation of mass transfer in models for reactive transport in streams and rivers.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Optimization of the deep neural network parameters for generating homogenized fuel assembly data for nodal codes

Homogenized fuel assembly (FA) data is a typical input data for nodal codes. Generating that data, however, could be time-consuming. One of promising ways to mitigate the computational burden of generating macroscopic cross-sections is to use trained artificial neural network (ANN) models for predicting nuclear data. However, there is a challenge to make the model support variable FA geometry. In this work, two most common types of FA were combined in one ANN model. Since there could be multiple ways of converting 2-dimensional FA data into 1-dimensional input vector for ANN, three different approaches of data flattening were evaluated. The input parameters included each fuel pin enrichment, fuel temperature, moderator temperature and boron concentration. The output parameters were 2-group macroscopic cross-sections (XS) and pin power distribution (HFF). A fully connected deep neural network (DNN) model was trained and tested using pre-generated data obtained with lattice physics code STREAM. The results of this study showed no statistically significant difference in the accuracy of XS and HFF generation for all 3 tested input vector orders. This means that fully connected DNN for XS generation demonstrated input sequence invariance. Results of comparing predicted XS data with reference solutions were found sufficiently close considering the reduction of computation time offered by ANN. Mean relative difference (MRD) for all output XS parameters was found below 0.7%, while HFF MRD was found higher compared to XS values, in some cases slightly exceeding 1%, mostly near guide tube locations. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1993

Intelligent neuroprocessors for in-situ launch vehicle propulsion systems health management

Efficacy of existing on-board propulsion systems health management systems (HMS) are severely impacted by computational limitations (e.g., low sampling rates); paradigmatic limitations (e.g., low-fidelity logic/parameter redlining only, false alarms due to noisy/corrupted sensor signatures, preprogrammed diagnostics only); and telemetry bandwidth limitations on space/ground interactions. Ultra-compact/light, adaptive neural networks with massively parallel, asynchronous, fast reconfigurable and fault-tolerant information processing properties have already demonstrated significant potential for inflight diagnostic analyses and resource allocation with reduced ground dependence. In particular, they can automatically exploit correlation effects across multiple sensor streams (plume analyzer, flow meters, vibration detectors, etc.) so as to detect anomaly signatures that cannot be determined from the exploitation of single sensor. Furthermore, neural networks have already demonstrated the potential for impacting real-time fault recovery in vehicle subsystems by adaptively regulating combustion mixture/power subsystems and optimizing resource utilization under degraded conditions. A class of high-performance neuroprocessors, developed at JPL, that have demonstrated potential for next-generation HMS for a family of space transportation vehicles envisioned for the next few decades, including HLLV, NLS, and space shuttle is presented. Of fundamental interest are intelligent neuroprocessors for real-time plume analysis, optimizing combustion mixture-ratio, and feedback to hydraulic, pneumatic control systems. This class includes concurrently asynchronous reprogrammable, nonvolatile, analog neural processors with high speed, high bandwidth electronic/optical I/O interfaced, with special emphasis on NASA's unique requirements in terms of performance, reliability, ultra-high density ultra-compactness, ultra-light weight devices, radiation hardened devices, power stringency, and long life terms.

Gulati, S.↗
NASA NTRS2018

Using the cFS Command and Data Dictionary (CCDD) to Automate Software Development on Habulous

Final paper is attached. The NASA developed Core Flight System (cFS) is a reusable software architecture that has been used on multiple spaceflight missions. By using this framework, missions are able to reuse code from other missions, as well as leverage deployment onto similar computer architectures (i.e. not "reinvent the wheel" on each new mission). The success in the cFS concept can be seen in the large number of projects using cFS at FSW-2018. The Habulous project is an Earth-based testbed, used for hardware and software that may one day be used on a future space habitat unit, with many participating groups from various NASA centers and aerospace organizations around the country. The distributed nature of the various teams mean that defining (and following) an interface definition is critical on the project. Additionally, since various groups use various types of computer hardware (32/64-bit, big/little endian, Linux/VxWorks/Windows) many additional complications exist in interfacing all the various components into a final integrated system. cFS is used on the majority the flight software (FSW) in running in Habulous. But some subsystems have elected to not use cFS, and use a software bridge (called SBN_lib) to interact with the other cFS nodes in Habulous. In order to most efficiently develop the FSW, a central database is used to define and store each message sent by cFS. A Command and Data Dictionary (CDD) is something nearly universal on spacecraft, but as a team we worked to develop the CDD before the SW development was complete, and not treat it like "as built" documentation. To manage the CDD, the cFS Command and Data Dictionary (CCDD) tool was chosen (available from NASA as open source software). The CCDD tool has successfully been used to automate/autocode a large amount of software used on Habulous, as we are hoping to use it to define even more items in the future (time-triggered Ethernet (TTE) network maps, CPU scheduling). Additionally, Habulous has been exploring the use of cFS on wildly heterogeneous CPUs, and how to coordinate all those various machines using/extending the software bus – network (SBN) application in cFS, as well as TTE to coordinate message passing between various synchronized machines. The major topics to be covered in the presentation are: (1) Updating to the CCSDS_v2 extended headers (and using CPU# as subsystem ID). (2) Managing all the message identification numbers for each cFS message sent/received on any of the various CPUs. (3) Using the CCDD information to automatically generate the C-header files that define the structure for all software bus (SB) commands/telemetry messages. (4) Using the CCDD to automatically generate XML Telemetry and Command Exchange (XTCE) files, which streams display production/integration/testing in a web based display architecture (5) Extending/customizing SBN to pass messages among computers on multiple networks. (6) Using "Protobetter" inside SBN to manage different endian-ness/architectures. (7) Using SBN_lib to allow non-cFS node to communicate with cFS nodes. (8) Developing TTE network and schedule tables for all the various CPUs to use.

Hirsh, Robert L.↗
DOE OSTI2021

PMU-data-driven Event Classification in Power Transmission Grids

This paper presents an event classification in transmission grids. The convolutional neural network (CNN)-based classifier is proposed to capture the temporal similarity of time-synchronized data stream from phasor measurement units (PMUs). The proposed CNN is trained using Bayesian optimization to search for the best hyperparameters. The effectiveness of the proposed event classification is validated through the real-world dataset from the U.S. transmission grids. This dataset includes line outage, transformer outage, frequency, and oscillation events. The validation process also includes different PMU outputs, such as voltage magnitude, phase angle, current magnitude, frequency, and rate of change of frequency (ROCOF). The results show that ROCOF gives the best classification performance compared to other PMU outputs. In addition, it is shown that the classifier trained with a larger dataset has higher accuracy. Moreover, the superiority of the proposed method is validated through comparison with other state-of-the-art classification methods.

Niazazari, Iman↗
DOE OSTI2022

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2018

Enriching the Twitter Stream Increasing Data Mining Yield and Quality Using Machine Learning

Social media data streams are important sources of real-time and historical global information for science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we are exploring the Twitter data stream for its potential in augmenting the validation program of NASA Earth science missions, specifically the Global Precipitation Measurement (GPM) mission. We have implemented a tweet processing infrastructure that outputs classified precipitation tweets. Inputs are "passive" tweets, along with a smaller number of tweets from "active" participants, i.e., those knowingly contributing to our effort. The "active" tweets, presumably of higher quality, enrich the Twitter stream. "Active" sources include data scraped from other social media (e.g., public Facebook posts) and data from existing crowdsourcing programs (e.g., mPING reports). In addition, there is likely relevant precipitation information in images and documents that are the end points of links often included in tweets. Information derived from these "active" sources could then be tweeted into the Twitter stream, thus enriching its quality. The objective of our current work is to mine these tweet­ linked images and documents, using neural networks, to increase the information content and quality related to precipitation. For images, we classified them as either precipitation-related or not. For training and validation, we used images obtained via the Google custom search API. We created two models: (1) by training a simple Convolutional Neural Network and (2) by using transfer learning principles to adapt a pre-trained object recognition model. For documents, both those linked to tweets and the tweet contents, we trained Hierarchical Attention Networks to determine precipitation occurrence, type, and intensity. For training and validation, we used a keyword-filtered tweet data set labelled with ground truth data from Dark Sky (an API to retrieve weather-related labels) and the National Severe Storms Laboratory's Multi­ Radar/Multi-Sensor (MRMS) system. Our results demonstrated the efficacy of our machine learning approaches for enriching the Twitter stream, to derive information potentially useful for validation of earth science satellite data.

Albayrak, Arif↗
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
NASA NTRS2003

A computational-grid based system for continental drainage network extraction using SRTM digital elevation models

We describe a new effort for the computation of elevation derivatives using the Shuttle Radar Topography Mission (SRTM) results. Jet Propulsion Laboratory's (JPL) SRTM has produced a near global database of highly accurate elevation data. The scope of this database enables computing precise stream drainage maps and other derivatives on Continental scales. We describe a computing architecture for this computationally very complex task based on NASA's Information Power Grid (IPG), a distributed high performance computing network based on the GLOBUS infrastructure. The SRTM data characteristics and unique problems they present are discussed. A new algorithm for organizing the conventional extraction algorithms [1] into a cooperating parallel grid is presented as an essential component to adapt to the IPG computing structure. Preliminary results are presented for a Southern California test area, established for comparing SRTM and its results against those produced using the USGS National Elevation Data (NED) model.

stream extraction↗