Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,639 records · Page 91

Machine Learning for NASA Advanced Information Systems

NASA's Advanced Information Systems Technology (AIST) Program is one of several Technology programs managed by the Earth Science Technology Office (ESTO) in the Earth Science Division (ESD). The AIST Program focuses on advanced information systems and novel computer science technologies that will be needed by NASA Earth Science in the next 5 to 10 years. The three main thrusts of the AIST Program deal with Novel Observing Strategies (NOS), Analytic Collaborative Frameworks (ACF) and Earth System Digital Twins (ESDT). For all these thrusts, Machine Learning (ML) is increasingly being used in multiple aspects of Earth science systems, e.g., for onboard autonomy and decision making, for the analysis of massive and diverse datasets as well as more recently for developing surrogate models that will represent one of the main components of future Digital Twins of the Earth. Particularly, ESDT technologies developed by the AIST Program will allow to develop integrated Earth Science frameworks that will mirror the Earth with state-of-the-art models (Earth system models and others), timely and relevant observations, and analytic tools. These information systems will be used for supporting near- and long-term science and policy decisions. ESDT frameworks will build on previously developed AIST capabilities and technologies to integrate interconnected models with continuous streams of observations, data analytics, data assimilation, simulations, advanced visualizations and the ability to conduct "what-if" scenarios. This talk will describe the three thrusts of the AIST Program with a special focus on Machine Learning and how it is being used at all steps of the Earth Science data lifecycle.

Mathematical and Computer Sciences (General)↗

Risks of Increased Earth Independence

As human space exploration begins to extend beyond the immediate vicinity of Earth, the ability of expertise on the ground to support operations will progressively decrease. Ground control expertise has been the primary countermeasure preventing loss of life and mission over the past sixty years and it will be gradually stripped away as expeditions reach further into space. The large, flexible and adaptive teams of experts on the ground, provide not only engineering analysis and problem-solving but also greatly increased work capacity. Monitoring of the large and fast moving stream of telemetry data is carried out around the clock by 20-30 flight controllers per shift. Analysis of that data and troubleshooting is carried out by an additional 50-100 engineers per shift. Artificial intelligence capabilities will need to be part of the solution but will not the entire solution. As progress continues to be made on intelligent systems, especially those for use in complex, dynamic environments where humans will remain a part of the activity, there is an ever increasing need to focus on how those systems will interact with the humans. Although capabilities such as natural language processing and facial recognition have become common place, even basic aspects of problem-solving, causal reasoning and generative decision-making remain well-beyond of our current capabilities. Advanced data visualization, procedure execution support, and new training approaches will be needed to close the gap between what is currently provided by experts on the ground and what the crew will need to do in increasingly autonomous ways with greater distance from Earth.

human-systems integration architecture↗

On the Advantages of Using Harmonized Landsat Sentinel-2 Data for Monitoring Environmental Change

NASA coordinates the Satellite Needs Working Group, dedicated to identifying, communicating, and addressing Earth observation needs of federal agencies. In 2016, the Harmonized Landsat Sentinel-2 (HLS) dataset was formulated and implemented to fulfill multiple needs. The combination of acquisitions from the Landsat and Sentinel-2 platforms results in a global dataset of surface reflectance with a temporal resolution of two days, while retaining the geometry and 30-meter spatial resolution of Landsat data. This harmonization allows for seamless integration with the 40-year archive of Landsat data. The HLS dataset is now available on the Google Earth Engine, enabling HLS utilization in various algorithms and frameworks essential for monitoring environmental change worldwide. During this presentation, we will demonstrate and discuss the advantages of using HLS data in comparison to using separate streams of Landsat and Sentinel-2 data in existing time series-based frameworks for change monitoring. Specifically, we will explore the application of HLS for continuous monitoring of deforestation using time series-based algorithms traditionally run with Landsat data. Additionally, we will showcase the benefits of HLS data for near real-time monitoring of forest disturbance in tropical regions. These examples underscore the value and utility of the HLS dataset for environmental monitoring and analysis.

Pontus Olofsson↗

Progress towards accelerating HOMME on hybrid multi- core systems

The suitability of a spectral element based dynamical core (HOMME) within the Community Atmospheric Model (CAM) for GPU-based architectures is examined and initial performance results are reported. This work was done within a project to enable CAM to run at high resolution on next generation, multi-petaflop systems. The dynamical core is the present focus because it dominates the performance profile of our target problem. HOMME enjoys good scalability due to its underlying cubed-sphere mesh with full two-dimensional decomposition and the localization of all computational work within each element. The thread blocking and code changes that allow HOMME to effectively use GPUs are described along with a rewritten vertical remapping scheme which improves performance on both CPUs and GPUs. Validation of results in the full HOMME model is also described. Remaining issues affecting performance include optimizing the boundary exchanges for the case of multiple spectral elements being computed on the GPU and using multiple CUDA streams to overlap data transfers with computations.

99 GENERAL AND MISCELLANEOUS↗

Systems, methods, and devices for failure detection of one or more energy storage devices

An energy storage device management system can include a management portion for charging/discharging an energy storage device and an ultrasound interrogation portion for passing ultrasound energy through the energy storage device during charge/discharge cycles. A memory stores a stream of capture data instances derived from ultrasound energy exiting the energy storage device and baseline ultrasound data instances corresponding with the energy storage device during normal charging/discharging thereof. A processor can compare each capture data instance with the baseline ultrasound data and detect abnormal operating states of the energy storage device. A warning system can issue a notification when abnormal operating states are detected.

Kowalski, Jeffrey A.↗

ADCP Upsample

This R script takes reads the csv file of stream-wise velocity data and generates a new time series of with comparable but distinct noise using the Langevin method and library. It simulates a time series measured at 1/10 the sampling rate by calculating a moving average of 10, though other sampling rates can be calculated by adjusting the moving average window. A stationary signal is calculated by taking the difference of the original signal and the moving average, then normalizing by the standard deviation of the moving average. The stationary signal is used to calculate drift and diffusion coefficients. Those coefficients are used to generate a new stationary signal. The new stationary signal is un-normalized and added to the moving average (or new equivalent signal) to create a new up-sampled velocity time-series.

Green, Nathan↗

Data and scripts associated with a manuscript on a meta-analysis synthesizing stream biogeochemical response to wildfires across space and time (v2)

This data package is associated with the publication “Catchment characteristics modulate the influence of wildfires on nitrate and dissolved organic carbon in lotic systems across space and time: A meta-analysis” submitted to Global Biogeochemical Cycles (Cavaiani et al. 2025). This study uses meta-analytical techniques to evaluate the effect of wildfire on in-stream responses in burned and unburned watersheds. The study aims to provide additional insight into the range of responses and net influences that wildfires have on hydro-biogeochemistry across broad spatial scales, burn extents, and the persistence of water-quality change. This study compiles data and metadata from 18 total publications that includes 1) surface water geochemistry data (dissolved organic carbon; nitrate), 2) climate classifications, 3) year of the wildfire, 4) the time lag between when the fire occurred and when the sampling occurred, and 5) study design of the publication. In total, this meta-analysis draws data that spans 8 climate guilds, 3 biomes, 62 watersheds, and 20 unique wildfires. See Sites_meta_data.csv for citations of the papers used in this meta-analysis. All R scripts and the associated data can also be found on GitHub at This data package was originally published in March 2024. It was updated in April 2025 (v2; new and modified files). See the change history section in the readme for more details. This data package contains five primary folders that include the following: (1) inputs; (2) output for analysis; (3) initial plots; (4) R scripts; and (5) GIS data. The data package also contains a data dictionary (dd) that provides column header definitions and a file-level metadata (flmd) file that describes every file. The “inputs” folder contains a list of all publications identified during the formal web search and an indication of whether each publication was included in the final analysis. Additionally, it includes site-level metadata, catchment characteristics, and GIS data for all publications included in the final analysis. The “Output_for_analysis” folder contains all data frames and figures generated from each R script used for additional data analysis. The “initial_plots” folder includes all exploratory figures that will be included in a supplemental and figures that will be submitted with the manuscript for publication. The “R_scripts” folder contains the scripts that perform all the data manipulations, statistical analyses, and plots. The “gis_data” folder includes shape files for each fire included in this meta-analysis. This data package contains the following file types: csv, pdf, jpeg, cpg, dbf, prj, shp, shp.ea.iso.xml, shp.iso.xml, shx.

54 ENVIRONMENTAL SCIENCES↗

Iterative map-making with two-level preconditioning for polarized cosmic microwave background data sets: A worked example for ground-based experiments

Context. An estimation of the sky signal from streams of time ordered data (TOD) acquired by the cosmic microwave background (CMB) experiments is one of the most important steps in the context of CMB data analysis referred to as the map-making problem. The continuously growing CMB data sets render the CMB map-making problem progressively more challenging in terms of computational cost and memory in particular in the context of ground-based experiments with their operational limitations as well as the presence of contaminants. Aims. We study a recently proposed, novel class of the Preconditioned Conjugate Gradient (PCG) solvers which invoke two-level preconditioners in the context of the ground-based CMB experiments. We compare them against the PCG solvers commonly used in the map-making context considering their precision and time-to-solution. Methods. We compare these new methods on realistic, simulated data sets reflecting the characteristics of current and forthcoming CMB ground-based experiments. We develop a divide-and-conquer implementation of the approach where each processor performs a sequential map-making for a subset of the TOD. Results. We find that considering the map level residuals, the new class of solvers permits us to achieve a tolerance that is better than the standard approach by up to three orders of magnitude, where the residual level often saturates before convergence is reached. This often corresponds to an important improvement in the precision of the recovered power spectra in particular on the largest angular scales. The new method also typically requires fewer iterations to reach a required precision and therefore shorter run times are required for a single map-making solution. However, the construction of an appropriate two-level preconditioner can be as costly as a single standard map-making run. Nevertheless, if the same problem needs to be solved multiple times, for example, as in Monte Carlo simulations, this cost is incurred only once, and the method should be competitive, not only as far as its precision is concerned but also its performance.

79 ASTRONOMY AND ASTROPHYSICS↗

Future interactive data systems

The information and control requirements of the investigator-users of future free-flying, platform-based, or Space Station-based scientific experiments are assessed, and the implications of these requirements for the design of the information systems are discussed. The principal requirements include near-real-time interaction with the payload, distributed or remote payload command and control, rapid access to quick-look data, flexibility of payload command sequences, on-demand uplink-downlink availability, system transparency, and modular design to facilitate replacement of system components. Current developments in systems architecture such as increased use of onboard microprocessors to select and preprocess data and transmit autonomous streams directly to the user, standardization of interfaces, and the NASA Tracking and Data Relay Satellite System are characterized, and block diagrams of present and future systems are provided.

Bracken, P. A.↗

Near-memory data reorganization engine

A memory subsystem package is provided that has processing logic for data reorganization within the memory subsystem package. The processing logic is adapted to reorganize data stored within the memory subsystem package. In some embodiments, the memory subsystem package includes memory units, a memory interconnect, and a data reorganization engine ("DRE"). The data reorganization engine includes a stream interconnect and DRE units including a control processor and a load-store unit. The control processor is adapted to execute instructions to control a data reorganization. The load-store unit is adapted to process data move commands received from the control processor via the stream interconnect for loading data from a load memory address of a memory unit and storing data to a store memory address of a memory unit.

97 MATHEMATICS AND COMPUTING↗

National Energy Water Treatment & Speciation (NEWTS): A Water & Critical Mineral Database and Dashboard

The scarcity of water resources, the need for beneficial water reuse, and the challenges of wastewater treatment are becoming increasingly pressing in economic, social, and environmental domains. Addressing these concerns requires effective treatment strategies to manage wastewater streams and tackle environmental and economic issues. Furthermore, the recovery of critical minerals from the waste streams associated with energy production holds the promise of offsetting treatment costs and securing local sources of valuable minerals. However, relevant data on these waste streams are dispersed and challenging to locate. The process of ingesting such data into modeling software often involves multiple steps, requiring data restructuring to meet software-input requirements. The non-standardized reporting of water data makes data aggregation and reformatting a time-consuming process. Additionally, essential attributes necessary for modeling water treatment and mineral scale formation are frequently missing. Moreover, data gaps vary depending on the region of interest. Consequently, there is a pressing need for high-quality energy-water composition data that can be easily imported into water chemistry modeling software. To address this need, the National Energy Technology Laboratory has created the National Energy Water Treatment and Speciation (NEWTS) Database and Dashboard—a free online tool catering to community leaders and water researchers. NEWTS facilitates a comprehensive understanding of the composition of energy-related wastewater streams in the United States. The datasets provide detailed concentrations and speciation of major and minor aqueous compounds in energy-related wastewater streams, including power plant leachate, acid mine drainage, brackish water, and oil and gas produced water across the United States. Many of the aqueous species are critical minerals (Li, REEs) in high demand to modernize the world’s energy infrastructure. Many of the datasets also contain volumetric flow-rates needed to model the treatment and reuse scenarios in advanced aqueous chemistry software programs. The NEWTS Database and Dashboard offer public access to hitherto challenging-to-access datasets, presented in a standardized format that is tailored for easy input into aqueous chemistry modeling software. By performing the work needed to transform dispersed, disparate data sources into unified, model-ready datasets, NEWTS serves as an essential resource in advancing water treatment research and sustainable water resource management.

produced water management↗

Acquisition and Pointing for Mars Optical Access Link

Optical characteristics can potentially benefit "access" links at Mars when transmitting data from surface to orbiting assets because of the higher gain and modulation bandwidth, compared to radio frequency (RF). Furthermore, higher bits/kg/W can be realized with low mass and power optical systems, enabling the streaming of high definition imagery. In this paper we present a conceptual design for a low complexity, autonomous optical communications link for returning data at 50-200 Mb/s from the Martian surface and for lower forward data rates of 50 kb/s to the surface. The pointing control is simplified by widening the transmitted laser beams (0.5 - 2.0 mrad) for the short distance (400-1200 Km) links. Link acquisition is based on the orbiter transceiver (OT) "blind"-pointing a laser beam to illuminate the lander transceiver (LT) on the surface. The LT acquires the link with a spectrally-filtered wide-field-of-view camera and subsequently tracks the orbiter transceiver with a two-axis, stepper-motor-actuator, to send back a laser modulated with high-rate data to the orbiting asset. The system design also has a provision for the OT transitioning from blind-pointing to closed loop tracking once it acquires the signal from the lander transceiver. Results from successful ground-based demonstrations performed at JPL, in which the pointing rate required to track an orbiter was emulated by mounting both transceivers on rotating stages, and in which we transmitted live video and pseudo-random data streams, are presented.

Optical Communication↗

Developing Fluorescence-Based Sensors to Support Rare Earth Element Separation

Rare earth elements (REEs) are essential to most renewable energy technologies. Unfortunately, as we transition to sustainable energy production, the demand for REEs is rapidly growing well beyond current rates of production. As a result, novel means of efficient, scalable, and easily adaptable methods for processing primary and recycle feedstocks are needed. Development and integration of sensors for highly selective in-line monitoring can support more efficient design and testing of such novel separation processes, as well as more cost-effective deployment of those separation flowsheets. Work here will explore the application of fluorescence spectroscopy, a highly sensitive and selective technique, to quantify multiple lanthanides in complex mixtures including known interferents or quenching agents. Results include identification of the optimal excitation wavelength and the limit of detection of various rare earth elements as well as the performance of data-science-based quantification approaches in streams where “unknowns” are present. Overall, the data science tools in conjunction with optical sensor data were able to quantify analytes in the presence of other lanthanides which can be anticipated in the actual industrial stream. Here we include characterization of lanthanides in a microfluidic device similar to those used in new process development. This study demonstrates the capability of utilizing fluorescence spectroscopy to quantify analytes in a complicated solution matrix, suggesting this is a successful approach for in-line monitoring to optimize the separation efficiency in an industrial stream.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Berkeley Nuclear Data Cloud (BDC) v2.3

The software was developed to serve and curate arbitrarily large datasets comprising data acquired from various mobile platforms. The software is contained in a number of server and client libraries. The former manage the ingestion, indexing, querying, and serving of the data. The latter libraries are distributed for Linux, Mac OSX, and Windows and enable users to interact with data downloaded from the service either in the form of an HDF5 file or streamed in a BSON data chunk. Using the Berkeley Data Cloud, researchers from varying fields can collaborate, compare results and curate both their raw data and the derived products of their analysis.

Cholia, Shreyas↗

Analysis of DWPF Sludge Batch 6 (Macrobatch 7): Pour Stream Glass Samples

The Defense Waste Processing Facility (DWPF) began processing Sludge Batch 6 (SB6), also referred to as Macrobatch 7 (MB7), in June 2010. SB6 is a blend of the heel of Tank 40 from Sludge Batch 5 (SB5), H-Canyon Np transfers and SB6 that was transferred to Tank 40 from Tank 51.1 SB6 was processed using Frit 418. Sludge is received into the DWPF Chemical Processing Cell (CPC) and is processed through the Sludge Receipt and Adjustment Tank (SRAT) and Slurry Mix Evaporator Tank (SME). The treated sludge slurry is then transferred to the Melter Feed Tank (MFT) and fed to the melter. During processing of each sludge batch, the DWPF is required to take at least one glass sample to meet the objectives of the Glass Product Control Program (GPCP) and to complete the necessary Production Records so that the final glass product may be disposed of at a Federal Repository. The DWPF requested various analyses of radioactive glass samples obtained from the melter pour stream during processing of SB6 as well as reduction/oxidation (REDOX) analysis of MFT samples to determine the impact of Argon bubbling. Sample analysis followed the Task Technical and Quality Assurance Plan (TTQAP) and an Analytical Study Plan (ASP). Four Pour Stream (PS) glass samples and two MFT slurry samples were delivered to the Savannah River National Laboratory (SRNL) from the DWPF. Table 1-1 lists the sample information for each pour stream glass sample. SB6 PS3 (S03472) was selected as the official pour stream sample for SB6 and full analysis was requested. This report details the visual observations of the as-received SB6 PS No.3 glass sample as well as results for the chemical composition, Product Consistency Test (PCT), radionuclide content, noble metals, and glass density. REDOX results will be provided for all four pour stream samples and vitrified samples of MFT-558 and MFT-568A. Where appropriate, data from other pour stream samples will be provided.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Towards understanding HPC users and systems: A NERSC case study

High performance computing (HPC) scheduling landscape currently faces new challenges due to thechanges in the workload. Previously, HPC centers were dominated by tightly coupled MPI jobs. HPCworkloads increasingly include high-throughput, data-intensive, and stream-processing applications. Asa consequence, workloads are becoming more diverse at both application and job levels, posing newchallenges to classical HPC schedulers. There is a need to understand the current HPC workloads andtheir evolution to facilitate informed future scheduling research and enable efficient scheduling in futureHPC systems.In this paper, we present a methodology to characterize workloads and assess their heterogeneity,at a particular time period and its evolution over time. We apply this methodology to the workloads ofthree systems (Hopper, Edison, and Carver) at the National Energy Research Scientific Computing Center(NERSC). We present the resulting characterization of jobs, queues, heterogeneity, and performance thatincludes detailed information of a year of workload (2014) and evolution through the systems’ lifetime(2010–2014).

Rodrigo, Gonzalo P.↗

From the Fire: A Deeper Look at the Phoenix Stream

We use 6 yr of data from the Dark Energy Survey to perform a detailed photometric characterization of the Phoenix stellar stream, a 15° long, thin, dynamically cold, low-metallicity stellar system in the Southern Hemisphere. We use natural splines, a nonparametric modeling technique, to simultaneously fit the stream track, width, and linear density. This updated stream model allows us to improve measurements of the heliocentric distance (17.4 ± 0.1 (stat.) ±0.8 (sys.) kpc) and distance gradient (−0.009 ± 0.006 kpc deg$^{−1}$) of Phoenix, which corresponds to a small change of 0.13 ± 0.09 kpc in heliocentric distance along the length of the stream. We measure linear intensity variations on degree scales, as well as deviations in the stream track on ∼2° scales, suggesting that the stream may have been disturbed during its formation and/or evolution. We recover three peaks and one gap in linear intensity along with fluctuations in the stream track. Compared to other thin streams, the Phoenix stream shows more fluctuations and, consequently, the study of Phoenix offers a unique perspective on gravitational perturbations of stellar streams. We discuss possible sources of perturbations to Phoenix, including baryonic structures in the Galaxy and dark matter subhalos.

79 ASTRONOMY AND ASTROPHYSICS↗

An H chondrite stream - Identification and confirmation

Fall data indicate that a significant, elongate cluster of co-orbital H chondrite falls in May between 1855 and 1895 (H Cluster 1) records encounters with two or three closely spaced and probably related meteoroid stream components, each of which was met near its perihelion. Although meteorites included in the cluster vary widely in petrographic type, shock facies, and Ne-21 exposure age, they have a distinct labile trace element signature that confirms a common thermal history and, thus, a common source region within an H chondrite parent body. Hence, meteorites selected by one criterion (fall parameters) as distinguishable from all other H chondrites, are distinguished from them by another completely different criterion (contents of labile trace elements).

Dodd, Robert T.↗