Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,459 records · Page 81

Prescriptive provenance for streaming analysis of workflows at scale

Abstract- We extend our approach capturing and relating the provenance and performance metrics of computational workflows as a diagnostic tool for runtime optimization and placement. One important challenge is the volume of extracted data, both for performance metrics and provenance, even when specifying filters and focusing on quantities of interest in a simulation. We reduce this data by performing anomaly detection on streaming data and store provenance for the detected anomalies, an approach we call prescriptive provenance. This paper discusses the Chimbuko architecture enabling the approach. We present the use of a protein structure propagation workflow based on NWChemEx. We are testing algorithms for anomaly detection and present preliminary results here obtained with Local Outlier Factor. While scaling remains a challenge, these results show that our robust Chimbuko architecture for streaming analysis with prescriptive provenance is a promising approach.

97 MATHEMATICS AND COMPUTING↗

The turbulence structure in the plane two-stream mixing layer.

Experimental data on the turbulent structure of free shear layers is examined in incompressible air streams. Both single-point and two-point data are used to examine the large coherent structure of the mixing layer. The use of spatial scales to define this large structure is explored. Application of these scales to modeling of free shear layers to show the similarity of such flows is demonstrated. Comparison of the statistical structure with observations from shadowgraph displays of plane shear layer flows shows good qualitative and quantitative agreement.

Jones, B. G.↗

Geostatistical Methods For Determination of Roughness, Topography, And Changes of Antarctic Ice Streams From SAR And Radar Altimeter Data

The central objective of this project has been the development of geostatistical methods fro mapping elevation and ice surface characteristics from satellite radar altimeter (RA) and Syntheitc Aperture Radar (SAR) data. The main results are an Atlas of elevation maps of Antarctica, from GEOSAT RA data and an Atlas from ERS-1 RA data, including a total of about 200 maps with 3 km grid resolution. Maps and digital terrain models are applied to monitor and study changes in Antarctic ice streams and glaciers, including Lambert Glacier/Amery Ice Shelf, Mertz and Ninnis Glaciers, Jutulstraumen Glacier, Fimbul Ice Shelf, Slessor Glacier, Williamson Glacier and others.

Herzfeld, Ute C.↗

A stream classification system for the conterminous United States

Stream classifications are important for understanding stream ecosystem diversity while also serving as tools for aquatic conservation and management. With current rates of land and riverscape modification within the United States (US), a comprehensive inventory and evaluation of naturally occurring stream habitats is needed, as this provides a physical template upon which stream biodiversity is organized and maintained. To adequately represent the heterogeneity of stream ecosystems, such a classification needs to be spatially extensive where multiple stream habitat components are represented at the highest resolution possible. Herein, we present a multi-layered empirically-driven stream classification system for the conterminous US, constructed from over 2.6 million stream reaches within the NHDPlus V2 stream network. The classification is based on emergent natural variation in six habitat layers meaningful at the stream-reach resolution: size, gradient, hydrology, temperature, network bifurcation, and valley confinement. To support flexibility of use, we provide multiple alternative approaches to developing classes and report uncertainty in classes assigned to stream reaches. The stream classification and underlying data provide valuable resources for stream conservation and research.

54 ENVIRONMENTAL SCIENCES↗

Early Fault Detection in Particle Accelerator Power Electronics Using Ensemble Learning

Early fault detection and fault prognosis are crucial to ensure efficient and safe operations of complex engineering systems such as the Spallation Neutron Source (SNS) and its power electronics (high voltage converter modulators). Following an advanced experimental facility setup that mimics SNS operating conditions, the authors successfully conducted 21 early fault detection experiments, where fault precursors are introduced in the system to a degree enough to cause degradation in the waveform signals, but not enough to reach a real fault. Nine different machine learning techniques based on ensemble trees, convolutional neural networks, support vector machines, and hierarchical voting ensembles are proposed to detect the fault precursors. Although all 9 models have shown a perfect and identical performance during the training and testing phase, the performance of most models has decreased in the next test phase once they got exposed to realworld data from the 21 experiments. The hierarchical voting ensemble, which features multiple layers of diverse models, maintains a distinguished performance in early detection of the fault precursors with 95% success rate (20/21 tests), followed by adaboost and extremely randomized trees with 52% and 48% success rates, respectively. The support vector machine models were the worst with only 24% success rate (5/21 tests). The study concluded that a successful implementation of machine learning in the SNS or particle accelerator power systems would require a major upgrade in the controller and the data acquisition system to facilitate streaming and handling big data for the machine learning models. In addition, this study shows that the best performing models were diverse and based on the ensemble concept to reduce the bias and hyperparameter sensitivity of individual models.

43 PARTICLE ACCELERATORS↗

Downlink data multiplexer

A data multiplexer that accommodates both industry standard CCSDS data packets and bits streams and standard IEEE 1394 data is described. The multiplexer provides a statistical allotment of bandwidth to the channels in turn, preferably four, but expandable in increments of four up to sixteen. A microcontroller determines bandwidth requested by the plurality of channels, as well as the bandwidth available, and meters out the available bandwidth on a statistical basis employing flow control to the input channels.

Holland, S. Douglas↗

WHONDRS River Corridor Sediment and Water Geochemistry and In Situ Sensor Data from 7 Perennial and 7 Intermittent Streams across San Antonio, Texas (v3)

This dataset supports a broader study examining the effects of intermittency on sediment respiration. The dataset provides sediment and surface water geochemistry and in situ sensor data from 7 perennial and 7 intermittent streams in San Antonio, Texas. Each stream/site was visited both in summer during base flow (July-September 2023) and winter during peak flow (January-February 2024). Related data were collected and will be published separately in collaboration with A. Veach. The data package was originally published in April 2025. It was updated in June 2025 (v2; modified and new files) and September 2025 (v3; modified files). See the change history section in the readme for more details. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. This dataset is comprised of two folders of field photos and videos, one folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data and one main data folder containing (1) file-level metadata; (2) data dictionary; (3) field metadata; (4) readme; (5) international generic sample number (IGSN) mapping file; (6) field protocol; (7) a subfolder with sample data; and (8) a subfolder with sensor data. The sample data subfolder contains (1) surface water and sediment dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (2) surface water and sediment total nitrogen data and averages; (3) sediment grain size data; (4) sediment iron (II) data and averages; (5) wet sediment mass, dry sediment mass, water mass, and wet sediment volume in incubation and sediment ICR vials; (7) sediment incubation respiration rate data and averages; (8) normalized respiration rate data and averages; (9) methods codes; (10) sediment percent carbon and nitrogen; (11) sediment X-ray diffraction (XRD) data; (12) gravimetric moisture and averages; (13) a subfolder with sediment incubation respiration data, scripts, and plots; (14) surface water and sediment FTICR methods; and (15) a subfolder of 9.4 Tesla (9.4T) FTICR-MS data. This folder contains five subfolders, one containing the sediment .xml data files, one containing the water .xml files, one containing the sediment CoreMS output files, one containing the water CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). The sensor data subfolder contains (1) a subfolder with miniDOT dissolved oxygen and temperature data and plots; (2) miniDOT dissolved oxygen and temperature summary data; and (3) miniDOT installation methods. All files are .csv, .pdf, .R, .xml, .d, .html, .Rmd, .py, .cal, .json, .jpg, .jpeg, .png, .mov, or .mp4. CORRECTION: The data processing methods for FTICR described in “v3_WHONDRS_AV1_Methods_Codes.csv” mistakenly indicate that users should process the data in Formultitude. The corrected description should read: “Both unprocessed and processed data are provided to allow users flexibility in data processing. Instructions and scripts for processing the data using CoreMS are included.” CORRECTION: Carbon and nitrogen content are reported as percentages. The current column headers "01395_C_percent_per_mg" and "01397_N_percent_per_mg" are incorrect. These should read "01395_C_percent" and "01397_N_percent" and will be corrected in the next version of this data package.

54 ENVIRONMENTAL SCIENCES↗

Ultrafast structural response to charge redistribution within a photosinthetic reaction centre

This data set contains stream files, difference maps and SVD maps obtained from data processing of time-resolved serial femtosecond crystallography measurements performed at LCLS XFEL, used to construct the images and perform the analysis published in the paper "Ultrafast structural response to charge redistribution within a photosinthetic reaction centre". Two (2) time points plus a dark state from 2015 ('a' label) as well as five (5) time points plus a dark state from 2016 ('b' label) are provided.

CXI↗

FPGA-based compression of streaming x-ray photon correlation spectroscopy data

A data acquisition system to perform real-time background subtraction and lower-level-discrimination-based compression of streaming x-ray photon correlation spectroscopy (XPCS) data from a fast charge-coupled device (CCD) area detector has been built and put into service at the Advanced Photon source (APS) at Argonne National Laboratory. A commercial frame grabber with on-board field-programmable gate array (FPGA) was used in the design, and continuously processes 60 frames per second each consisting of 1,024 x 1,024 pixels with up to 64512 photon hits per frame.

43 PARTICLE ACCELERATORS↗

Once in a blue stream: Detection of recent star formation in the NGC 7241 stellar stream with MEGARA

Aims.In this work we study the striking case of a narrow blue stream with a possible globular cluster-like progenitor around the NGC 7241 galaxy and its foreground dwarf companion. We want to figure out if the stream was generated by tidal interaction with NGC 7241 or if it first interacted with the foreground dwarf companion and later both fell together toward NGC 7241. Methods.We used four sets of observations, including a follow-up spectroscopic study of this stream based on data taken with the MEGARA instrument at the 10.4 m Gran Telescopio Canarias using the integral field spectroscopy mode, the Mount Lemmon 0.80 m telescope, the Telescopio NazionaleGalileo, the DESI Imaging Legacy surveys, and GALEX archival data. We also used high-resolution zoomed-in cosmological simulations. Results.Our data suggest that the compact object we detected in the stream is a foreground Milky Way halo star. Near this compact object we detect emission lines overlapping a less compact, bluer, and fainter blob of the stream that is clearly visible in both ultraviolet and optical deep images. From its heliocentric systemic radial velocity derived from the [O III]λ5007 Å lines (V syst = 1548.58 ± 1.80 km s −1 ) and new UV and optical broadband photometry, we conclude that this overdensity could be the actual core of the stream, with an absolute magnitude ofM g ∼ −10 and ag − r = 0.08 ± 0.11, consistent with a remnant of a low-mass dwarf satellite undergoing a current episode of star formation. From the width of the stream and assuming a circular orbit, we calculate that the progenitor mass can be typical of a dwarf galaxy, but it could also be substantially lower if the stream is on a very radial orbit or if it was created by tidal interaction with the companion dwarf instead of with NGC 7241. These estimates also suggest that this is one of the lowest mass streams detected to date beyond the Local Group. Finally, we find that blue stellar streams containing star formation regions are commonly predicted by high-resolution cosmological simulations of galaxies lighter than the Milky Way. This scenario is consistent with the processes explaining the bursty star formation history of some dwarf satellites, which are followed by a gas depletion and a fast quenching once they enter within the virial radius of their host galaxies for the first time. Thus, it is likely that the stream’s progenitor is undergoing a star formation burst comparable to those that have shaped the star formation history of several Local Group dwarfs in the last few gigayears.

Astronomy & Astrophysics↗

Global Landslide Forecasting System for Hazard Assessment and Situational Awareness

Landslides triggered by extreme rainfall can be devastating, resulting in loss of life, property, and infrastructure. Landslide forecasting systems provide an opportunity to build awareness of potential hazards and ultimately take preemptive measures. There is currently a dearth of forecasting systems that provide regional or global coverage, but these systems can offer important situational awareness in data-sparse, ungauged, or large-scale catchments. A near global, primarily satellite-based system calledthe Landslide Hazard Assessment for Situational Awareness (LHASA) provides near real-time estimates of potential landslide hazard and exposure around the world. In this work, a precipitation forecast module is introduced into LHASA to complement the existing LHASA framework and provide an estimate of landslide hazard up to three days in advanceat 1km resolution. The model-based Goddard Earth Observing System-Forward Processing (GEOS-FP) precipitation forecast product is used as the forcing input for the model in place of the satellite-based Integrated Multi-satellitE Retrievals for Global Precipitation Mission product. Soil moisture and snow depth from the GEOS-FP assimilated product are also incorporated. The study period January 2020–January 2021 is usedto test the model performance against the LHASA near real-time estimates at multiple spatiotemporal scales. Validation of the model is carried out using a collection of rainfall-triggered landslide inventories from around the world as case studies to demonstrate the potential utility and limitations of this system.The rescaling of the GEOS-FP precipitation product is a critical step in incorporating the forecasted precipitation data within LHASA-Forecast (LHASA-F). Combining different streams of forecasted data within the LHASA-F framework shows promise, particularly for larger events at the 1-and 2-day lead time for events. Results indicate that for the case studies evaluated, the LHASA-F is generally able to resolve major landslide events triggered by extreme rainfall, such as from tropical cyclones. The analysis shows that landslide forecast outputs may be represented differently depending on the user’s needs. This framework serves as a first milestone in providing a global predictive view of landslide hazard.

Landslide forecasting system↗

Vis-SAGA: Visual Analytics for Situational Awareness of Grid Anomalies: Preprint

We describe supporting near real-time situational awareness of the electric distribution system by visualizing novel data from voltage sensors deployed on existing broadband cable television network equipment. Our scalable web-based visual analytics platform supports interactive geospatial exploration, time-series analysis, and summarization of grid behavior during potentially anomalous events. The broadband cable television sensor network provides observability of the electrical distribution system at a higher local spatial resolution than is typically available to most utilities, revealing the operational state of the network and aiding in the detection of abnormal behaviors or deviations from expected patterns, particularly across electric utility service areas. We outline the design and development of interactive geospatial and time-series visualization components and the scalable data services that supply metadata, historical, and real-time streams of sensor data across the network. We evaluate our platform during periods of extreme weather, demonstrating its ability to assist in detecting patterns of operation that affect power availability, quality, resiliency, and service restoration.

cable television↗

Pitot pressure measurements in flow fields behind circular-arc nozzles with exhaust jets at subsonic free-stream Mach numbers

The flow field behind a circular arc nozzle with exhaust jet was studied at subsonic free stream Mach numbers. A conical probe was used to measure the pitot pressure in the jet and free stream regions. Pressure data were recorded for two nozzle configurations at nozzle pressure ratios of 2.0, 2.9, and 5.0. At each set of test conditions, the probe was traversed from the jet center line into the free stream region at seven data acquisition stations. The survey began at the nozzle exit and extended downstream at intervals. The pitot pressure data may be applied to the evaluation of computational flow field models, as illustrated by a comparison of the flow field data with results of inviscid jet plume theory.

Mason, M. L.↗

The radial evolution of the bulk properties of the solar wind

Simultaneous solar-wind proton data obtained at several heliocentric distances during radial alignments are compared. The radial variations associated with two high-speed streams in the solar wind are studied as examples of the radial evolution of the solar-wind speed distribution and of high-speed streams in the solar wind as observed in the ecliptic plane in 1973. Pioneer 11 data on high-speed streams in the solar wind observed at about 1.5 and 3.7 AU are compared with the corresponding high-speed-stream data obtained at earth. These analyses indicate that as these high-speed streams propagated to these extended heliocentric distances, there was an erosion of the highest speeds and a general narrowing of the speed distribution. These observations are consistent with the exchange of momentum in the solar wind between high-speed streams and low-speed streams as they propagate outward from the sun.

Intriligator, D. S.↗

PlanetSense: A Real-time Streaming and Spatio-temporal Analytics Platform for Gathering Geo-spatial Intelligence from Open Source Data

Geospatial intelligence has traditionally relied on the use of archived and unvarying data for planning and exploration purposes. In consequence, the tools and methods that are architected to provide insight and generate projections only rely on such datasets. Albeit, if this approach has proven effective in several cases, such as land use identification and route mapping, it has severely restricted the ability of researchers to inculcate current information in their work. This approach is inadequate in scenarios requiring real-time information to act and to adjust in ever changing dynamic environments, such as evacuation and rescue missions. In this work, we propose PlanetSense, a platform for geospatial intelligence that is built to harness the existing power of archived data and add to that, the dynamics of real-time streams, seamlessly integrated with sophisticated data mining algorithms and analytics tools for generating operational intelligence on the fly. The platform has four main components – i) GeoData Cloud – a data architecture for storing and managing disparate datasets; ii) Mechanism to harvest real-time streaming data; iii) Data analytics framework; iv) Presentation and visualization through web interface and RESTful services. Using two case studies, we underpin the necessity of our platform in modeling ambient population and building occupancy at scale.

big data architecture↗

The nature and orbit of the Ophiuchus stream

The Ophiuchus stream is a recently discovered stellar tidal stream in the Milky Way. We present high-quality spectroscopic data for 14 stream member stars obtained using the Keck and MMT telescopes. We confirm the stream as a fast moving (v{sub los} ∼ 290 km s{sup −1}), kinematically cold group (σ{sub v{sub l{sub o{sub s}}}}≲1 km s{sup −1}) of α-enhanced and metal-poor stars ([α/Fe] ∼ 0.4 dex, [Fe/H] ∼ −2.0 dex). Using a probabilistic technique, we model the stream simultaneously in line-of-sight velocity, color–magnitude, coordinate, and proper motion space, and so determine its distribution in 6D phase-space. We find that the stream extends in distance from 7.5 to 9 kpc from the Sun; it is 50 times longer than wide, merely appearing highly foreshortened in projection. The analysis of the stellar population contained in the stream suggests that it is ∼12 Gyr old, and that its initial stellar mass was ∼2 × 10{sup 4} M{sub ⊙} (or at least ≳7 × 10{sup 3} M{sub ⊙}). Assuming a fiducial Milky Way potential, we fit an orbit to the stream that matches the observed phase-space distribution, except for some tension in the proper motions: the stream has an orbital period of ∼350 Myr, and is on a fairly eccentric orbit (e ∼ 0.66) with a pericenter of ∼3.5 kpc and an apocenter of ∼17 kpc. The phase-space structure and stellar population of the stream show that its progenitor must have been a globular cluster that was disrupted only ∼240 Myr ago. We do not detect any significant overdensity of stars along the stream that would indicate the presence of a progenitor, and conclude that the stream is all that is left of the progenitor.

79 ASTRONOMY AND ASTROPHYSICS↗

F3D Image Processing and Analysis for Many - and Multi-core Platforms

F3D is written in OpenCL, so it achieve[sic] platform-portable parallelism on modern mutli-core CPUs and many-core GPUs. The interface and mechanims to access F3D core are written in Java as a plugin for Fiji/ImageJ to deliver several key image-processing algorithms necessary to remove artifacts from micro-tomography data. The algorithms consist of data parallel aware filters that can efficiently utilizes[sic] resources and can work on out of core datasets and scale efficiently across multiple accelerators. Optimizing for data parallel filters, streaming out of core datasets, and efficient resource and memory and data managements over complex execution sequence of filters greatly expedites any scientific workflow with image processing requirements. F3D performs several different types of 3D image processing operations, such as non-linear filtering using bilateral filtering and/or median filtering and/or morphological operators (MM). F3D gray-level MM operators are one-pass constant time methods that can perform morphological transformations with a line-structuring element oriented in discrete directions. Additionally, MM operators can be applied to gray-scale images, and consist of two parts: (a) a reference shape or structuring element, which is translated over the image, and (b) a mechanism, or operation, that defines the comparisons to be performed between the image and the structuring element. This tool provides a critical component within many complex pipelines such as those for performing automated segmentation of image stacks. F3D is also called a "descendent" of Quant-CT, another software we developed in the past. These two modules are to be integrated in a next version. Further details were reported in: D.M. Ushizima, T. Perciano, H. Krishnan, B. Loring, H. Bale, D. Parkinson, and J. Sethian. Structure recognition from high-resolution images of ceramic composites. IEEE International Conference on Big Data, October 2014.

Source record↗