Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Continual learning in the presence of repetition

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.

Class-incremental learning↗

AGR-2 Data Qualification Report for ATR Cycle 154B

This report provides the data qualification status of Advanced Gas Reactor-2 (AGR-2) fuel irradiation experimental data from Advanced Test Reactor (ATR) Cycle 154B as recorded in the Nuclear Data Management and Analysis System (NDMAS). This is the last cycle of AGR-2 irradiation, as the test train was pulled from the ATR core during the outage portion of ATR Cycle 155A. The AGR-2 data streams addressed in this report include thermocouple (TC) temperatures, sweep gas data (flow rates including new Fission Product Monitoring (FPM) downstream flows from Fission Product Monitoring System (FPMS) detectors, pressure, and moisture content), and FPMS data (release rates and release-to-birth rate ratios [R/Bs]) for each of the six capsules in the AGR-2 experiment. The final data qualification status for these data streams is determined by a Data Review Committee (DRC) comprised of AGR technical leads, Sitewide Quality Assurance (QA), and NDMAS analysts. The Data Review Committee reviewed the data acquisition process, considered whether the data met the requirements for data collection as specified in QA-approved Very High Temperature Reactor (VHTR) data collection plans, examined the results of NDMAS data testing and statistical analyses, and confirmed the qualification status of the data as given in this report.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

AGR-3/4 Data Qualification Report for ATR Cycles 151A, 151B, 152A, 152B, 154A, and 154B

This data report provides the qualification status of Advanced Gas Reactor-3/4 (AGR-3/4) fuel irradiation experimental data from Advanced Test Reactor (ATR) Cycles 151A, 151B, 152A, 152B, 154A, and 154B, as recorded in the Nuclear Data Management and Analysis System (NDMAS). Of these cycles, ATR Cycle 152A is a low power cycle that occurred when the ATR core was briefly at low power. The irradiation data are not used for physics and thermal calculation, but the qualification status of these cycle data is still covered in this report. On the other hand, during ATR Cycles 153A (unplanned Outage cycle) and 153B (Power Axial Locator Mechanism [PALM] cycle), the AGR-3/4 was pulled out from the ATR core and stored in the canal to avoid being overheated. Therefore, qualification of the AGR-3/4 irradiation data from these 2 cycles was excluded in this report. By the end of ATR Cycle 154B, AGR-3/4 was irradiated for a total of 264.1 effective full power days. The AGR-3/4 data streams addressed in this report include thermocouple (TC) temperatures, sweep gas data (flow rates, pressure, and moisture content), and Fission Product Monitoring System (FPMS) data (release rates and release-to-birth rate ratios [R/Bs]) for each of the twelve capsules in the AGR-3/4 experiment. The final data qualification status for these data streams is determined by a Data Review Committee (DRC) composed of AGR technical leads, Sitewide Quality Assurance (QA), and NDMAS analysts. The DRC convened on February 12, 2014, reviewed the data acquisition process, and considered whether the data met the requirements for data collection as specified in QA-approved Very High Temperature Reactor (VHTR) Technology Development Office (TDO) data collection plans. The DRC also examined the results of NDMAS data testing and statistical analyses, and confirmed the qualification status of the data as given in this report.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

AGR-3/4 Final Data Qualification Report for ATR Cycles 151A through 155B-1

This report provides the qualification status of experimental data for the entire Advanced Gas Reactor 3/4 (AGR 3/4) fuel irradiation. AGR-3/4 is the third in a series of planned irradiation experiments conducted in the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL) for the AGR Fuel Development and Qualification Program, which supports development of the advanced reactor technology under the INL ART Technology Development Office (TDO). The main objective of AGR-3/4 irradiation is to provide a known source of fission products for subsequent transport through compact matrix and structural graphite materials due to the presence of designed-to-fail fuel particles. Full power irradiation of the AGR 3/4 test began on December 14, 2011 (ATR Cycle 151A), and was completed on April 12, 2014 (end of ATR Cycle 155B) after 369.1 effective full power days of irradiation. The AGR-3/4 test was in the reactor core for eight of the ten ATR cycles between 151A and 155B. During the unplanned outage cycle, 153A, the experiment was removed from the ATR northeast flux trap (NEFT) location and stored in the ATR canal. This was to prevent overheating of fuel compacts due to higher than normal ATR power during the subsequent Powered Axial Locator Mechanism cycle, 153B. The AGR 3/4 test was inserted back into the ATR NEFT location during the outage of ATR Cycle 154A on April 26, 2013. Therefore, the AGR-3/4 irradiation data received during these 2 cycles (153A and 153B) are irrelevant and their qualification status isnot included in this report. Additionally, during ATR Cycle 152A the ATR core ran at low power for a short enough duration that the irradiation data are not used for physics and thermal calculations. However, the qualification status of irradiation data for this cycle is still covered in this report. As a result, this report includes data from 8 ATR Cycles: 151A, 151B, 152A, 152B, 154A, 154B, 155A, and 155B, as recorded in the Nuclear Data Management and Analysis System (NDMAS). The AGR 3/4 data streams addressed in this report include thermocouple (TC) temperatures, sweep gas data (flow rates, pressure, and moisture content), and Fission Product Monitoring System (FPMS) data (release rates, release to birth rate ratios [R/Bs], and particle failure counts) for each of the twelve capsules in the AGR 3/4 experiment. During Outage Cycle 155A, fourteen flow meters were installed downstream from fourteen FPMS monitors to measure flows from the monitors; qualification status of these data are also included in the report. The final data qualification status for these data streams is determined by a Data Review Committee (DRC) composed of AGR technical leads, Sitewide Quality Assurance (QA), and NDMAS analysts. For ATR Cycles 151A through 154B, the DRC convened on February 12, 2014, reviewed the data acquisition process, and considered whether the data met the requirements for data collection as specified in QA approved INL ART TDO data collection plans. The DRC also examined the results of NDMAS data testing and statistical analyses, and confirmed the qualification status of the data as given in this report. The qualification status of AGR-3/4 irradiation data during the first six cycles were previously reported in INL/EXT-14-31186 document. This report presents data qualification status for the entire AGR-3/4 irradiation.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Learning Global Proliferation Expertise Evolution Using AI-Driven Analytics and Public Information

Detecting and anticipating global proliferation expertise and capability evolution from unstructured, noisy, and incomplete public data streams is a highly desired, but extremely challenging task. Here, in this article, we present our pioneering data-driven approach to support the non-proliferation mission to detect and explain the evolution of proliferation expertise and capability development globally from terabytes of publicly available information (PAI), focusing on our knowledge extraction pipeline and descriptive analytics. We first discuss how we fuse nine open-source data streams, including multilingual data, to convert 4 TB of unstructured data to structured knowledge and encode dynamically evolving proliferation expertise representations—content and context graphs. For this, we rely on natural language processing (NLP) and deep learning (DL) models to perform information extraction, topic modeling, and distributed text representation (aka embedding) learning. We then present interactive, usable, and explainable descriptive analytics to refine domain knowledge and present it in a human-understandable form. Finally, we introduce future work avenues that will leverage our dynamic knowledge representations and descriptive analytics to enable predictive and prescriptive inferences to achieve real-time domain understanding and contextual reasoning about global proliferation expertise and capability evolution.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

SPRUCE S1 Bog Porewater, Groundwater, and Stream Chemistry Data: 2011-2013

This data set reports the chemistry of S1 bog porewater, outlet stream water, and groundwater from upland wells from 2011 through 2013. These data provide information for studies of the biogeochemical status of the bog and surrounding groundwater aquifer. Porewater samples were collected from one nest of 6 piezometers installed in a hollow in the south end of the S1 bog to characterize solute chemistry depth profiles. Peat porewater was collected from piezometers that opened at 0, 0.3, 0.5, 1, 2, and 3 m depths below hollow surfaces. Samples were collected weekly (2011) or monthly (2012 & 2013) when the samplers were not frozen (typically June – November). Stream water was sampled weekly (2011-2013) from the outlet of the S1 bog (located on the southern edge of the bog). Four groundwater wells located in the uplands of the S1 watershed were sampled monthly in 2013. Bog water level data are also reported for the S1 bog. Water samples were analyzed for pH, specific conductivity, nutrient concentrations (ammonium, nitrate, soluble reactive phosphorus, total nitrogen, total phosphorus), cation concentrations (calcium, potassium, magnesium, sodium, aluminum, iron, manganese, silicon, strontium), anion concentrations (chloride, sulfate), and total organic carbon concentrations. The experimental work is to be conducted in a Picea mariana [black spruce] – Sphagnum spp. bog forest in northern Minnesota, 40 km north of Grand Rapids, in the USDA Forest Service Marcell Experimental Forest (MEF).

54 ENVIRONMENTAL SCIENCES↗

Accelerating Data Acquisition, Reduction, and Analysis at the Spallation Neutron Source

ORNL operates the world's brightest neutron source, the Spallation Neutron Source (SNS). Funded by the US DOE Office of Basic Energy Science, this national user facility hosts hundreds of scientists from around the world, providing a platform to enable break-through research in materials science, sustainable energy, and basic science. While the SNS provides scientists with advanced experimental instruments, the deluge of data generated from these instruments represents both a big data challenge and a big data opportunity. For example, instruments at the SNS can now generate multiple millions of neutron events per second providing unprecedented experiment fidelity but leaving the user with a dataset that cannot be processed and analyzed in a timely fashion using legacy techniques. To address this big data challenge, ORNL has developed a near real-time streaming data reduction and analysis infrastructure. The Accelerating Data Acquisition, Reduction, and Analysis (ADARA) system provides a live streaming data infrastructure based on a high-performance publish subscribe system, in situ data reduction, visualization, and analysis tools, and integration with a high-performance computing and data storage infrastructure. ADARA allows users of the SNS instruments to analyze their experiment as it is run and make changes to the experiment in real-time and visualize the results of these changes. In this paper we describe ADARA, provide a high-level architectural overview of the system, and present a set of use-cases and real-world demonstrations of the technology.

Campbell, Stuart I↗

Why wait? Let's start computing while data is still on the wire

In this era of Big Data, computing useful information from data is becoming increasingly complicated, particularly due to the ever increasing volumes of data that need to travel over the network to data centers to be stored and processed, all highly expensive operations in the long haul. In this paper we suggest that we can do computing and analysis of data "on the wire," i.e., while data is still in transit. The nature of these computations include analysis, visualization, pattern recognition, and prediction, or forecasting, on the streaming data. We follow a service function chaining architecture to implement this, assuming that the data packets arrive within a single network administrative domain. As a demonstration of this new computing paradigm, we present three examples. Firstly, we demonstrate pattern recognition and data visualization on streaming forex data, which can be used for lucrative trading in the forex market. In our second example, we analyze and learn user buying patterns from clickstream data streaming from multiple websites. Finally, we monitor solar sensors for a zero reading while the packets are still on their way to the data center, to schedule any maintenance and requisite repairs with no time delay.

97 MATHEMATICS AND COMPUTING↗

Adaptive Waveform Correlation Detectors for Arrays: Algorithms for Autonomous Calibration

Waveform correlation detectors compare a signal template with successive windows of a continuous data stream and report a detection when the correlation coefficient, or some comparable detection statistic, exceeds a specified threshold. Since correlation detectors exploit the fine structure of the full waveform, they are exquisitely sensitive when compared to power (STA/LTA) detectors. The drawback of correlation detectors is that they require complete knowledge of the signal to be detected, which limits such methods to instances of seismicity in which a very similar signal has already been observed by every station used. Such instances include earthquake swarms, aftershock sequences, repeating industrial seismicity, and many other forms of controlled explosions. The reduction in the detection threshold is even greater when the techniques are applied to arrays since stacking can be performed on the individual channel correlation traces to achieve significant array gain. In previous years we have characterized the decrease in detection threshold afforded by correlation detection across an array or network when observations of a previous event provide an adequate template for signals from subsequent events located near the calibration event. Last year we examined two related issues: (1) the size of the source region calibration footprint afforded by a master event, and (2) the use of temporally incoherent detectors designed to detect the gross envelope structure of the signal to extend the footprint. In Case 1, results from the PETROBAR-1 marine refraction profile indicated that array correlation gain was usable at inter-source separations out to one or two wavelengths. In Case 2, we found that incoherent detectors developed from a magnitude 6 event near Svalbard were successful at detecting aftershocks where correlation detectors derived from individual aftershocks were not. Incoherent detectors might provide 'seed' events for correlation detectors that then could extend detection to lower magnitudes. This year we addressed a problem long known to limit the acceptance of correlation detectors in practice: the labor intensive development of templates. For example, existing design methods cannot keep pace with rapidly unfolding aftershock sequences. We successfully built and tested an object-oriented framework (as described in our 2005 proposal) for autonomous calibration of waveform correlation detectors for an array. The framework contains a dynamic list of detectors of several types operating on a continuous array data stream. The list has permanent detectors: beam forming power (STA/LTA) detectors which serve the purpose of detecting signals not yet characterized with a waveform template. The framework also contains an arbitrary number of subspace detectors which are launched automatically using the waveforms from validated power detections as templates. The implementation is very efficient such that the computational cost of adding subspace detectors was low. The framework contains a supervisor that oversees the validation of power detections, and periodically halts the processing to revise the portfolio of detectors. The process of revision consists of collecting the waveforms from all detections, performing cross-correlations pairwise among all waveforms, clustering the detections using correlations as a distance measure, then creating a new subspace detector from each cluster. The collection of new subspace detectors replaces the existing portfolio and processing of the data stream resumes. This elaborate scheme was implemented to prevent proliferation of closely-related subspace detectors. The method performed very well on several simple sequences: 2005 'drumbeat' events observed locally at Mt. St. Helens, and the 2003 Orinda, CA aftershock sequence. Our principal test entailed detection of the aftershocks of the San Simeon earthquake using the NVAR array; in this case, the system automatically detected and categorized approximately 2/3 of the events above magnitude 2.8.

58 GEOSCIENCES↗

LHC physics dataset for unsupervised New Physics detection at 40 MHz

In the particle detectors at the Large Hadron Collider, hundreds of millions of proton-proton collisions are produced every second. If one could store the whole data stream produced in these collisions, tens of terabytes of data would be written to disk every second. The general-purpose experiments ATLAS and CMS reduce this overwhelming data volume to a sustainable level, by deciding in real-time whether each collision event should be kept for further analysis or be discarded. We introduce a dataset of proton collision events that emulates a typical data stream collected by such a real-time processing system, pre-filtered by requiring the presence of at least one electron or muon. This dataset could be used to develop novel event selection strategies and assess their sensitivity to new phenomena. In particular, we intend to stimulate a community-based effort towards the design of novel algorithms for performing unsupervised new physics detection, customized to fit the bandwidth, latency and computational resource constraints of the real-time event selection system of a typical particle detector.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Selecting Essential Information for Biosurveillance - A Multi-Criteria Decision Analysis

This paper proposes the use of Multi-Attribute Utility Theory to address the issue of identifying and selecting essential information for inclusion into a biosurveillance system or process. Additionally, we developed a decision support framework that can facilitate identifying data streams for use in biosurveillance systems or processes and demonstrated utility by applying the framework to the problem of evaluating data streams for use in an global infectious disease surveillance system.

59 BASIC BIOLOGICAL SCIENCES↗

Atmospheric Radiation Measurement Program Climate Research Facility Operations Quarterly Report - October 1 - December 31, 2008

Description. Individual raw data streams from instrumentation at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) fixed and mobile sites are collected and sent to the Data Management Facility (DMF) at Pacific Northwest National Laboratory (PNNL) for processing in near real-time. Raw and processed data are then sent daily to the ACRF Archive, where they are made available to users. For each instrument, we calculate the ratio of the actual number of data records received daily at the Archive to the expected number of data records. The results are tabulated by (1) individual data stream, site, and month for the current year and (2) site and fiscal year (FY) dating back to 1998.

54 ENVIRONMENTAL SCIENCES↗

Atmospheric Radiation Measurement Program Climate Research Facility Operations Quarterly Report - July 1 - September 30, 2008

Description. Individual raw data streams from instrumentation at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) fixed and mobile sites are collected and sent to the Data Management Facility (DMF) at Pacific Northwest National Laboratory (PNNL) for processing in near real-time. Raw and processed data are then sent daily to the ACRF Archive, where they are made available to users. For each instrument, we calculate the ratio of the actual number of data records received daily at the Archive to the expected number of data records. The results are tabulated by (1) individual data stream, site, and month for the current year and (2) site and fiscal year (FY) dating back to 1998.

54 ENVIRONMENTAL SCIENCES↗

Stream splitting moving target defense

Systems and methods for utilizing stream splitting Moving Target Defense (MTD) to provide enhanced computer system communication system security by splitting a data stream in to a plurality of paths is described. In some implementations, Stream splitting MTD, involves splitting a single data stream (e.g., TCP stream) into a plurality of discrete units, then sending and receiving those discrete units from and to different (ideally geographically disparate) receiving servers, with the stream being reassembled on the receiving end. The plurality of discrete units of data include resequencing data. The size of each discrete unit may vary depending on the specific implementation, even down to small unit sizes (e.g., a single packet).

Evans, Nathaniel↗

Atmospheric Radiation Measurement Program Climate Research Facility Operations Quarterly Report January 1 - March 31, 2009

Individual raw data streams from instrumentation at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) fixed and mobile sites are collected and sent to the Data Management Facility (DMF) at Pacific Northwest National Laboratory (PNNL) for processing in near real-time. Raw and processed data are then sent daily to the ACRF Archive, where they are made available to users. For each instrument, we calculate the ratio of the actual number of data records received daily at the Archive to the expected number of data records. The results are tabulated by (1) individual data stream, site, and month for the current year and (2) site and fiscal year (FY) dating back to 1998.

54 ENVIRONMENTAL SCIENCES↗

WHONDRS Surface Water Geochemistry and Organic Matter Characterization Data from Streams Distributed across Latin America

This dataset supports a broader study examining global transferability of stream biogeochemistry and was generated in collaboration with the MicroSudAqua (µSudAqua) network (https://microsudaqua.netlify.app/en/). The dataset provides surface water geochemistry (dissolved organic carbon, total dissolved nitrogen, cations) and organic matter characterization (FTICR-MS) from streams in Argentina, Brazil, Chile, and Colombia. Samples were collected across stream orders (1st to 6th order) within five basins. Related data were collected and will be published separately in collaboration with the µSudAqua network. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to this readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) a folder of field photos; (2) a folder of surface water sample data, (3) a folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data; (4) file-level metadata; (5) data dictionary; (6) field metadata; (7) readme; (8) international generic sample number (IGSN) mapping file; and (9) field protocol. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (2) total dissolved nitrogen data and averages; (3) anions and averages; (4) methods codes; (5) FTICR-MS methods; and (15) a subfolder of 9.4 Tesla (9.4T) FTICR-MS data. This folder contains the processed data and three subfolders, one containing the .xml files, one containing the water CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .d, .html, .Rmd, .py, .cal, .json, .jpg, .jpeg, .png, .mov, or .mp4.

Anions↗

Machine Learning Digital Twin for Lithium Ion Battery State of Health Predictions

A digital twin system has been established to model the long term degradation of the state of health of lithium ion batteries. Two data streams result from the computational model of the system and from the physical experiment measurements. A machine learning pipeline has been developed at the nexus of these data streams. Leveraging the unique data sources in multiple transfer learning approaches has lead to the development of multiple cell specific machine learning digital twins. Our discussion on this first of its kind technology will cover challenges in data ingestion, scalability, architecture and orchestration, and research findings.

25 - ENERGY STORAGE↗

G-mode magnetic force microscopy: Separating magnetic and electrostatic interactions using big data analytics

In this work, we develop a full information capture approach for Magnetic Force Microscopy (MFM), referred to as generalized mode (G-Mode) MFM. G-Mode MFM acquires and stores the full data stream from the photodetector, captured at sampling rates approaching the intrinsic photodiode limit. The data can be subsequently compressed, denoised, and analyzed, without information loss. Here, G-Mode MFM is implemented and compared to the traditional heterodyne-based MFM on model systems, including domain structures in ferromagnetic Yttrium Iron Garnet and the electronically and magnetically inhomogeneous high entropy alloy, CoFeMnNiSn. We investigate the use of information theory to mine the G-Mode MFM data and demonstrate its usefulness for extracting information which may be hidden in traditional MFM modes, including signatures of nonlinearities and mode-coupling phenomena. Finally, we demonstrate detection and separation of magnetic and electrostatic tip-sample interactions from a single G-Mode image, by analyzing the entire frequency response of the cantilever. G-Mode MFM is immediately implementable on any atomic force microscopy platform and as such is expected to be a useful technique for probing spatiotemporal cantilever dynamics and mapping material properties, as well as their mutual interactions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗