Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data noise”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Seismic Spatial Gradients and Machine Learning-Based Classifiers for Explosion Monitoring (LDRD 218327)

This final report summarizes the work completed under the Laboratory Directed Research and Development (LDRD) project “Seismic Spatial Gradients as a Machine Learning-Based Classifier for Explosion Monitoring.” The overarching goal of the project was to explore the efficacy of using machine learning-based classification algorithms where the input data are the spatial gradient of the seismic wavefield collected at a single point on the Earth’s surface. The methods that I describe here are in direct contrast to conventional methods of seismic discrimination which typically rely on a spatially extended network of instruments and physics-based wavefield attributes such as, for example, the ratio between $\textit{P}$ and $\textit{S}$ waves. Rather, we use the spatial gradient of the seismic wavefield observed at a single point on the Earth’s surface and data processing approaches inspired by the machine learning community. We tested two algorithms, a neural network and a modified version of principal component analysis termed Spectrally Filtered Principal Component Analysis (SFPCA). To test these algorithms, we first conducted a series of numerical tests using synthetic data and then conducted a small-scale controlled field experiment. The tests using synthetic data showed that both algorithms had high success rates on gradiometric data, even when simulated noise was added to the signal. Furthermore, we found that using seismic spatial gradients increased the performance of our discrimination algorithms when compared to using just the traditional translational motion seismic data. The tests with field data also showed a high degree of discriminative success.

58 GEOSCIENCES↗
DOE OSTI2025

A Bayesian Framework for Spectral Reprojection

Abstract Fourier partial sum approximations yield exponential accuracy for smooth and periodic functions, but produce the infamous Gibbs phenomenon for non-periodic ones. Spectral reprojection resolves the Gibbs phenomenon by projecting the Fourier partial sum onto a Gibbs complementary basis, often prescribed as the Gegenbauer polynomials. Noise in the Fourier data and the Runge phenomenon both degrade the quality of the Gegenbauer reconstruction solution, however. Motivated by its theoretical convergence properties, this paper proposes a new Bayesian framework for spectral reprojection, which allows a greater understanding of the impact of noise on the reprojection method from a statistical point of view. We are also able to improve the robustness with respect to the Gegenbauer polynomials parameters. Finally, the framework provides a mechanism to quantify the uncertainty of the solution estimate.

Li, Tongtong (ORCID:0000000276644764)↗
DOE OSTI2022

Monitoring and flaw detection during wire-based directed energy deposition using in-situ acoustic sensing and wavelet graph signal analysis

The goal of this work is to detect flaw formation in the wire-based directed energy deposition (W-DED) process using in-situ sensor data. The W-DED studied in this work is analogous to metal inert gas electric arc welding. The adoption of W-DED in industry is limited because the process is susceptible to stochastic and environmental disturbances that cause instabilities in the electric arc, eventually leading to flaw formation, such as porosity and suboptimal geometric integrity. Moreover, due to the large size of W-DED parts, it is difficult to detect flaws post-process using non-destructive techniques, such as X-ray computed tomography. Accordingly, the objective of this work is to detect flaw formation in W-DED parts using data acquired from an acoustic (sound) sensor installed near the electric arc. To realize this objective, we develop and apply a novel wavelet integrated graph theory approach. The approach extracts a single feature called graph Laplacian Fiedler number from the noise-contaminated acoustic sensor data, which is subsequently tracked in a statistical control chart. Using this approach, the onset of various types of flaws are detected with a false alarm rate less-than 2%. This work demonstrates the potential of using advanced data analytics for in-situ monitoring of W-DED.

42 ENGINEERING↗
DOE OSTI2024

Galaxy bias in the era of LSST: perturbative bias expansions

Upcoming imaging surveys will allow for high signal-to-noise measurements of galaxy clustering at small scales. In this work, we present the results of the Rubin Observatory Legacy Survey of Space and Time (LSST) bias challenge, the goal of which is to compare the performance of different nonlinear galaxy bias models in the context of LSST Year 10 (Y10) data. Specifically, we compare two perturbative approaches, Lagrangian perturbation theory (LPT) and Eulerian perturbation theory (EPT) to two variants of Hybrid Effective Field Theory (HEFT), with our fiducial implementation of these models including terms up to second order in the bias expansion as well as nonlocal bias and deviations from Poissonian stochasticity. Here, we consider a variety of different simulated galaxy samples and test the performance of the bias models in a tomographic joint analysis of LSST-Y10-like galaxy clustering, galaxy-galaxy-lensing and cosmic shear. We find both HEFT methods as well as LPT and EPT combined with non-perturbative predictions for the matter power spectrum to yield unbiased constraints on cosmological parameters up to at least a maximal scale of k max = 0.4 Mpc -1 for all samples considered, even in the presence of assembly bias. While we find that we can reduce the complexity of the bias model for HEFT without compromising fit accuracy, this is not generally the case for the perturbative models. We find significant detections of non-Poissonian stochasticity in all cases considered, and our analysis shows evidence that small-scale galaxy clustering predominantly improves constraints on galaxy bias rather than cosmological parameters. These results therefore suggest that the systematic uncertainties associated with current nonlinear bias models are likely to be subdominant compared to other sources of error for tomographic analyses of upcoming photometric surveys, which bodes well for future galaxy clustering analyses using these high signal-to-noise data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

WeakIdent: Weak formulation for identifying differential equation using narrow-fit and trimming

Data-driven identification of differential equations is an interesting but challenging problem, especially when the given data are corrupted by noise. When the governing differential equation is a linear combination of various differential terms, the identification problem can be formulated as solving a linear system, with the feature matrix consisting of linear and nonlinear terms multiplied by a coefficient vector. This product is equal to the time derivative term, and thus generates dynamical behaviors. The goal is to identify the correct terms that form the equation to capture the dynamics of the given data. We propose a general and robust framework to recover differential equations using a weak formulation with two new mechanisms, narrow-fit and trimming, for both ordinary and partial differential equations (ODEs and PDEs). The weak formulation facilitates an efficient and robust way to handle noise, and two new mechanisms, narrow-fit and trimming, improve the coefficient support and value recoveries respectively. For each sparsity level, Subspace Pursuit is utilized to find an initial set of support from the large dictionary. Then, we focus on highly dynamic regions (rows of the feature matrix), and error normalize the feature matrix in the narrow-fit step. The support is further updated via trimming the terms that contribute the least. Finally, the support set of features with the smallest Cross-Validation error is chosen as the result. A comprehensive set of numerical experiments are presented for both systems of ODEs and PDEs with various noise levels. The proposed method gives a robust recovery of the coefficients, and a significant denoising effect which can handle up to 100% noise-to-signal ratio for some equations. We compare the proposed method with several state-of-the-art algorithms for the recovery of differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Evaluation of a pair-based, joint-likelihood association approach for regional infrasound event identification

SUMMARY A Bayesian framework for the association of infrasonic detections is presented and evaluated for analysis at regional propagation scales. A pair-based, joint-likelihood association approach is developed that identifies events by computing the probability that individual detection pairs are attributable to a hypothetical common source and applying hierarchical clustering to identify events from the pair-based analysis. The framework is based on a Bayesian formulation introduced for infrasonic source localization and utilizes the propagation models developed for that application with modifications to improve the numerical efficiency of the analysis. Clustering analysis is completed using hierarchical analysis via weighted linkage for a non-Euclidean distance matrix defined by the negative log-joint-likelihood values. The method is evaluated using regional synthetic data with propagation distances of hundreds of kilometres in order to study the sensitivity of the method to uncertainties and errors in backazimuth and time of arrival. The method is found to be robust and stable for typical uncertainties, able to effectively distinguish noise detections within the data set from those in events, and can be made numerically efficient due to its ease of parallelization.

58 GEOSCIENCES↗
DOE OSTI2025

Identifying Vehicle Signals in Continuous Seismic Data Using Unsupervised Machine-Learning Techniques

Seismic sensors deployed near roadways effectively capture ground vibrations generated by passing vehicles. Although both traditional and machine‐learning algorithms have been utilized for analyzing such signals, independent validation of detected vehicle events remains limited. We applied two unsupervised machine‐learning algorithms, uniform manifold approximation and projection for dimension reduction, and hierarchical density‐based spatial clustering of applications with noise, to continuous seismic data collected along a road on the main campus of Oak Ridge National Laboratory. The algorithms identified seven distinct cluster labels across the entire dataset. By comparing these cluster labels with precipitation records from a nearby weather station and image‐derived labels from a local camera system, we identified one cluster associated with rainfall and another with vehicle activity. Our algorithms identified a greater number of vehicle‐related labels compared to the camera‐derived labels because seismic data are unaffected by poor lighting conditions. The arrival times of the newly detected vehicle signals corresponded well with the road’s speed limit, supporting our findings. Our algorithm outperformed the short‐term average/long‐term average method and k‐means clustering. Our results suggest that seismic data, when analyzed with machine‐learning algorithms, can complement existing vehicle monitoring systems, particularly under challenging environmental conditions.

Chai, Chengping [Oak Ridge National Laboratory (OR↗
DOE OSTI2024

Exploration of signal processing methods for superconducting magnet and quench data

Quenching is the phenomenon of a superconducting magnetic material carrying current transitioning into a regular conducting material. This may cause severe and irreparable damage to the superconductor due to Joule heating. The Magnet Department at Fermi National Accelerator Laboratory (FNAL) has acquired experimental data through quench antenna arrays that are recorded when the quench is detected. These data are in terms of voltage signals that are sampled at 100kHz for several minutes. There are multiple channels and each channel provides a data set of more than 20 million observations, while there is one channel, called the trigger channel which shows the time when quench is detected. Despite some advancements that were made including machine learning, data complexity still shadows the progress. In this work, we studied a multi-resolution analysis of the quench antenna data through the Haar wavelet transform. In particular, we applied the maximally overlapped discrete w avelet transform (MODWT) of a suitable level L to the given data and then projected it onto the wavelet basis. This decomposes a given signal (Original data) $x ϵ \mathbb{R}^N$ into $L + 1$ subspaces of $\mathbb{R}^N$. One of the subspaces called the approximation, captures the trend of the signal, and the others, called the details, capture the fluctuations at different frequency bands. This decomposition provides a clear trend of the data at a suitable level and also various activities (spikes) are seen in the details of the decomposition at every level. These spikes might reveal some information about the quench under investigation but in any case, give information about magnet behavior. Also, this decomposition is seen to be very useful in removing noise present in the data due to the source or mechanism of the experiment.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Mathematical Morphological Filtering with a Self-Adaptive Reconstruction Technique and Application to Local Seismic Data

Recorded seismic data are generally contaminated by noise from different sources, which masks the signals of interest. In the seismology community, frequency filtering (FF) is the standard method for noise suppression. However, when the signal of interest and noise share the same frequency band, the latter cannot be filtered out without infringing on the former. We implemented a noise suppression approach based on the mathematical morphology theorem. The method involves compound operations of dilation and erosion using structuring elements of varying lengths and decomposes an input noisy waveform into several time functions with differing characteristics. Further, the filtered waveform is constructed from the time functions using a self-adaptive reconstruction technique. Application to a data set of >4700 local waveforms suggests that the implemented mathematical morphological filtering (MMF) approach is efficient for data with low signal-to-noise ratio (SNR) and significantly outperforms FF in that SNR range. For most of the dataset, FF, machine learning (ML) denoising, and continuous wavelet transform (CWT) thresholding result in higher SNR values compared with the MMF method. However, for ~42% of the waveforms, MMF outperforms FF, and the SNR gain achieved with MMF is as large as ~23 dB. Compared to ML denoising and CWT thresholding, this proportion drops to only ~10%–14%. Our results suggests that in an operational setting, MMF cannot replace the other noise suppression methods; however, signal detection can be improved if MMF is used to supplement them in some scenarios. MMF could help detect signals in problematic low-SNR data, which are currently being missed particularly when using FF alone.

58 GEOSCIENCES↗
DOE OSTI2025

Unlocking hidden information in sparse small-angle neutron scattering measurements

Hypothesis Small-Angle Neutron Scattering (SANS) is a powerful technique for studying soft matter systems such as colloids, polymers, and lyotropic phases, providing nanoscale structural insights. However, its effectiveness is limited by low neutron flux, leading to long acquisition times and noisy data. Here, we hypothesize that Bayesian statistical inference using Gaussian Process Regression (GPR) can reconstruct high-fidelity scattering data from sparse measurements by leveraging intensity smoothness and continuity. Experiments and Simulations The method was benchmarked computationally and validated through SANS experiments on various soft matter systems, including wormlike micelles, colloidal suspensions, polymeric structures, and lyotropic phases. GPR-based inference was applied to both experimental and synthetic data to evaluate its effectiveness in noise reduction and intensity reconstruction. Findings GPR significantly enhances SANS data quality and therefore reducing measurement times by up to two orders of magnitude. This cost-effective approach maximizes experimental efficiency, enabling high-throughput studies and real-time monitoring of dynamic systems. It is particularly beneficial for weakly scattering and time-sensitive studies. Beyond SANS, this framework applies to other low-SNR techniques, including laboratory-based small-angle X-ray scattering and various dynamical scattering methods. Furthermore, it offers transformative potential for compact neutron sources, enhancing their viability for structural analysis in resource-limited settings.

Small angle neutron scattering↗
DOE OSTI2024

High Yield Xray Imager Final Design Review

The High Yield Xray Imager (HYXI) is a new NIF target diagnostic system currently under development. The goal of HYXI is to provide high-fidelity, high temporal resolution x-ray imaging capability on high yield NIF implosions at 10MJ and above. The HYXI instrument design concept is based on the combination of two technologies that have been successfully utilized at the NIF on previous instruments, electron pulse-dilation and hybrid-CMOS sensor imaging. The combination of these two techniques will give HYXI sufficient data quality to ascertain differences in hot spot formation dynamics between high and low yield implosions. This information will highlight the critical hot spot conditions needed for ignition and burn. The HYXI design leverages the successful operation of the PDIXI x-ray imager at the NIF on multi MJ yield shots. A new radiation tolerant CMOS imaging array (HYPERION) is being developed to eliminate the significant background noise which limits the data quality of PDIXI. We successfully placed the contract with Advanced hCMOS Systems (AHS) to develop the HYPERION sensor, which fulfils our criteria to place long lead time item procurements by end of FY24. The HYXI Final Design Review was completed at the end of Q4 FY24 (Sep 24 th and Sep 30 th ). The HYXI project is a multi-year effort with a phased approach to be bring up system functionality over time in parallel with the development and fabrication effort of the HYPERION CMOS imaging array. In Phase 1, time-integrated x-ray images on NIF DT experiments will be collected starting in Q3 FY25. In Phase 2 of the project, time-resolved imaging with HYXI utilizing a spare microchannel plate detector back-end will begin in Q3 FY26. Phase 3 concludes the project with the installation of the HYPERION sensor array and the final performance qualification of the HYXI instrument which is scheduled for Q3 FY27 as discussed in the PDR and MRT report on this project in FY23.

42 ENGINEERING↗
DOE OSTI2023

Milestone report for MRT 8479: High Yield Xray Imager Preliminary Design Review

The High Yield Xray Imager (HYXI) is a new NIF target diagnostic system currently under development. The goal of HYXI is to provide high-fidelity, high temporal resolution x-ray imaging capability on high yield NIF implosions at 10MJ and above. The HYXI instrument design concept is based on the combination of two technologies that have been successfully utilized at the NIF on previous instruments, electron pulse-dilation and hybrid-CMOS sensor imaging. The combination of these two techniques will give HYXI sufficient data quality to ascertain differences in hot spot formation dynamics between high and low yield implosions. This information will highlight the critical hot spot conditions needed for ignition and burn. The HYXI design leverages the successful operation of the PDIXI x-ray imager at the NIF on multi-MJ yield shots. Also, a new radiation tolerance CMOS imaging array is being developed to eliminate the significant background noise which limits the data quality of PDIXI. The HYXI Preliminary Design Review was completed at the end of Q4 FY23. The HYXI project is a multi-year effort with a phased approach to be bring up system functionality over time in parallel with the development and fabrication effort of the new CMOS imaging array. Initial time integrated NIF data will be collected in Q2 FY25, first time resolved imaging with Daedalus starting in Q2 FY26 and the final performance qualification of the complete HYXI system is scheduled for Q2 FY27.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2026

Full-stack Quantification of Variability in Predicting Ion Transport Properties using Machine-learned Interatomic Potentials

Machine-learned interatomic potentials (MLIPs) have become the state-of-the-art for performing accurate, scalable molecular dynamics (MD) simulations. It is therefore crucial to understand and quantify the reliability of MLIPs for downstream property predictions. Uncertainty in predicted properties can arise from limitations in first-principles training data, intrinsic MLIP model errors in representing the data, and the statistical noise introduced during subsequent MD simulations. Using ion transport in Li7P3S11 as a case study, we systematically assess the impact of training set size and selection, neural network stochasticity, and MD sampling statistics on predicted diffusivity and activation energy. We find that when using equivariant MLIP architectures with standard MD protocols, uncertainty arising from MD sampling dominates over model-induced errors. In contrast, MLIP errors relative to the underlying first-principles data are consistently minor. Given this, there are two main routes to improving the accuracy of predictions based on MLIP potentials: adopting higher accuracy reference data generation methods, and improving the MD sampling statistics.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Filling data analysis gaps in time-resolved crystallography by machine learning

There is a growing understanding of the structural dynamics of biological molecules fueled by x-ray crystallography experiments. Time-resolved serial femtosecond crystallography (TR-SFX) with x-ray Free Electron Lasers allows the measurement of ultrafast structural changes in proteins. Nevertheless, this technique comes with some limitations. One major challenge is the quality of data from TR-SFX measurements, which often faces issues like data sparsity, partial recording of Bragg reflections, timing errors, and pixel noise. To overcome these difficulties, conventionally, large volumes of data are collected and grouped into a few temporal bins. The data in each bin are then averaged and paired with the mean of their corresponding jittered timestamps. This procedure provides one structure per bin, resulting in a limited number of averaged structures for the entire time interval spanned by the experiment. Therefore, the information on ultrafast structural dynamics at high temporal resolution is lost. This has initiated research for advanced methods of analyzing experimental TR-SFX data beyond the standard binning and averaging method. To address this problem, we use a machine learning algorithm called Nonlinear Laplacian Spectral Analysis (NLSA), which has emerged as a promising technique for studying the dynamics of complex systems. In this work, we demonstrate the power of this algorithm using synthetic x-ray diffraction snapshots from a protein with significant data incompleteness, timing uncertainties, and noise. Our study confirms that NLSA is a suitable approach that effectively mitigates the effects of these artifacts in TR-SFX data and recovers accurate structural dynamics information hidden in such data.

Trujillo, Justin (ORCID:0000000285505360)↗
DOE OSTI2020

SYMBA: An end-to-end VLBI synthetic data generation pipeline

Realistic synthetic observations of theoretical source models are essential for our understanding of real observational data. In using synthetic data, one can verify the extent to which source parameters can be recovered and evaluate how various data corruption effects can be calibrated. These studies are the most important when proposing observations of new sources, in the characterization of the capabilities of new or upgraded instruments, and when verifying model-based theoretical predictions in a direct comparison with observational data. We present the SYnthetic Measurement creator for long Baseline Arrays (SYMBA), a novel synthetic data generation pipeline for Very Long Baseline Interferometry (VLBI) observations. SYMBA takes into account several realistic atmospheric, instrumental, and calibration effects. Here, we used SYMBA to create synthetic observations for the Event Horizon Telescope (EHT), a millimetre VLBI array, which has recently captured the first image of a black hole shadow. After testing SYMBA with simple source and corruption models, we study the importance of including all corruption and calibration effects, compared to the addition of thermal noise only. Using synthetic data based on two example general relativistic magnetohydrodynamics (GRMHD) model images of M 87, we performed case studies to assess the image quality that can be obtained with the current and future EHT array for different weather conditions. Our synthetic observations show that the effects of atmospheric and instrumental corruptions on the measured visibilities are significant. Despite these effects, we demonstrate how the overall structure of our GRMHD source models can be recovered robustly with the EHT2017 array after performing calibration steps, which include fringe fitting, a priori amplitude and network calibration, and self-calibration. With the planned addition of new stations to the EHT array in the coming years, images could be reconstructed with higher angular resolution and dynamic range. In our case study, these improvements allowed for a distinction between a thermal and a non-thermal GRMHD model based on salient features in reconstructed images.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Crystallographic variant mapping using precession electron diffraction data

In this work, we developed three methods to map crystallographic variants of samples at the nanoscale by analyzing precession electron diffraction data using a high-temperature shape memory alloy and a VO2 thin film on sapphire as the model systems. The three methods are (I) a user-selecting-reference pattern approach, (II) an algorithm-selecting-reference-pattern approach, and (III) a k-means approach. In the first two approaches, Euclidean distance, Cosine, and Structural Similarity (SSIM) algorithms were assessed for the diffraction pattern similarity quantification. We demonstrated that the Euclidean distance and SSIM methods outperform the Cosine algorithm. We further revealed that the random noise in the diffraction data can dramatically affect similarity quantification. Denoising processes could improve the crystallographic mapping quality. With the three methods mentioned above, we were able to map the crystallographic variants in different materials systems, thus enabling fast variant number quantification and clear variant distribution visualization. The advantages and disadvantages of each approach are also discussed. We expect these methods to benefit researchers who work on martensitic materials, in which the variant information is critical to understand their properties and functionalities.

Crystallographic variant mapping↗
DOE OSTI2023

Online real-time learning of dynamical systems from noisy streaming data

Abstract Recent advancements in sensing and communication facilitate obtaining high-frequency real-time data from various physical systems like power networks, climate systems, biological networks, etc. However, since the data are recorded by physical sensors, it is natural that the obtained data is corrupted by measurement noise. In this paper, we present a novel algorithm for online real-time learning of dynamical systems from noisy time-series data, which employs the Robust Koopman operator framework to mitigate the effect of measurement noise. The proposed algorithm has three main advantages: (a) it allows for online real-time monitoring of a dynamical system; (b) it obtains a linear representation of the underlying dynamical system, thus enabling the user to use linear systems theory for analysis and control of the system; (c) it is computationally fast and less intensive than the popular extended dynamic mode decomposition (EDMD) algorithm. We illustrate the efficiency of the proposed algorithm by applying it to identify the Van der Pol oscillator, the chaotic attractor of the Henon map, the IEEE 68 bus system, and a ring network of Van der Pol oscillators.

97 MATHEMATICS AND COMPUTING↗