Search NASA⌕ Search

SEARCH · Search NASA

Results for “data processing methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Methods of resolving artifacts in Hadamard-transformed data

Apparatus and methods are disclosed for processing data transformed according to an invertible transform (e.g., using a Hadamard transform) multiplexing scheme. In one example of the disclosed technology, a computer-implemented method includes generating transformed data by applying a Hadamard transform to intensity data generated by modulating input of analytes into a mass spectrometer according to a pseudorandom sequence (PRS). The exemplary method further includes identifying at least one pair of symmetric intensity peaks in the transformed data based on the PRS and removing data associated with the pair of symmetric peaks from the transformed data to produce modified data, which can be used to identify, characterize, and/or quantify the composition of the sample. In some examples, the exemplary method further includes validating peaks in the transformed data based on comparing the location of peaks in the untransformed intensity data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

The utility of Bayesian data reconciliation for separations

Data reconciliation methods for separation processes typically rely on classical statistical approaches to generate estimates of true mass flow rates from measurements. Knowledge regarding the uncertainty of these estimates has value in decision making, but is often not acquired. Bayesian approaches intrinsically quantify uncertainty; however, literature for Bayesian data reconciliation of separation processes is scarce. This publication outlines two Bayesian data reconciliation models and provides details for how the models were implemented for the BayesMassBal (V 1.0.0) software package written in R. To demonstrate the advantages of this approach for data reconciliation, the models were first applied to simulated data and then compared to a classical model through a Monte Carlo experiment. In this example, the Bayesian models were found to provide more accurate estimates of the simulated data, while also providing quantitative information on the estimate uncertainty. To demonstrate the use of the technique in a practical problem, the models were also applied to real data collected from a pilot-scale rare earth solvent extraction process. Here, this publication provides a small window into how Bayesian methods can be used for data reconciliation, but findings suggest Bayesian data reconciliation models for separation processes have distinct advantages over classical alternatives.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2022

Phase curves of small bodies from the SLOAN Moving Objects Catalog

Extensive photometric surveys continue to produce enormous stores of data on small bodies. These data are typically sparsely obtained at arbitrary (or unknown) rotational phases. Therefore, new methods for processing such data need to be developed to make the most of these vast catalogs. We aim to produce a method of recreating the phase curves of small bodies by considering the uncertainties introduced by the nominal errors in the magnitudes and the effect introduced by rotational variations. Here, we use the SLOAN Moving Objects Catalog data as a benchmark to construct phase curves of all small bodies in u', g', r', i', and z' filters. From the phase curves, we obtain the absolute magnitudes and we use them to set up the absolute colors, which are the colors of the asteroids that are not affected by changes in the phase angle. We selected objects with ≥3 observations taken in at least one filter and spanning over a minimum of 5 degrees in the phase angle. We developed a method that combines Monte Carlo simulations and Bayesian inference to estimate the absolute magnitudes using the HG 12 * photometric system. We obtained almost 15 000 phase curves, with about 12 000 of these including all five filters. The absolute magnitudes and absolute colors are compatible with previously published data that support our method. The method we developed is fully automatic and well suited for a run based on large amounts of data. Moreover, it includes the nominal uncertainties in the magnitudes and the whole distribution of possible rotational states of the objects producing what are possibly less precise values, that is, larger uncertainties, but more accurate, namely, closer to the actual value. To our knowledge, this work is the first to include the effect of rotational variations in such a manner.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Memory access system

A method and system for accessing a memory for a data processing system. The method comprises sending a read request for a plurality of locations in the memory to read the plurality of locations in parallel based on an upper bound for reading the memory. The upper bound for a number of locations is based on a group of constraints for the memory. The method receives a summed value of a plurality of memory values in the plurality of locations in the memory.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Method and apparatus for scheduling in a non-uniform compute device

A data processing apparatus, and method of operation thereof, for executing instructions. The apparatus includes one or more host processors, each having a first processing unit, and a multi-level memory system. One or more levels of the memory system are tightly coupled to a corresponding second processing unit. At least one of the host processors includes an instruction scheduler that routes instructions selectively to at least one of the first and second processing units, dependent upon the availability of the processing units and the location, within the memory system, of data to be used when executing the instructions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Robust Vector BOTDA Signal Processing with Probabilistic Machine Learning

This paper presents a novel probabilistic machine learning (PML) framework to estimate the Brillouin frequency shift (BFS) from both Brillouin gain and phase spectra of a vector Brillouin optical time-domain analysis (VBOTDA). The PML framework is used to predict the Brillouin frequency shift (BFS) along the fiber and to assess its predictive uncertainty. We compare the predictions obtained from the proposed PML model with a conventional curve fitting method and evaluate the BFS uncertainty and data processing time for both methods. The proposed method is demonstrated using two BOTDA systems: (i) a BOTDA system with a 10 km sensing fiber and (ii) a vector BOTDA with a 25 km sensing fiber. The PML framework provides a pathway to enhance the VBOTDA system performance.

47 OTHER INSTRUMENTATION↗
DOE OSTI2018

A Solar Time-based Analog Ensemble Method for Regional Solar Power Forecasting

This paper presents a new analog ensemble method for day-ahead regional photovoltaic (PV) power forecasting with hourly resolution. By utilizing open weather forecast and power measurement data, this prediction method is processed within a set of historical data with similar meteorological data (temperature and irradiance), and astronomical date (solar time and earth declination angle). Further, clustering and blending strategies are applied to improve its accuracy in regional PV forecasting. The robustness of the proposed method is demonstrated with three different numerical weather prediction models, the North American Mesoscale Forecast System, the Global Forecast System, and the Short-Range Ensemble Forecast, for both region level and single site level PV forecasts. Using real measured data, the new forecasting approach is applied to the load zone in Southeastern Massachusetts as a case study. In conclusion, the normalized root mean square error (NRMSE) has been reduced by 13.80%-61.21% when compared with three tested baselines.

14 SOLAR ENERGY↗
DOE OSTI2024

Exploration of signal processing methods for superconducting magnet and quench data

Quenching is the phenomenon of a superconducting magnetic material carrying current transitioning into a regular conducting material. This may cause severe and irreparable damage to the superconductor due to Joule heating. The Magnet Department at Fermi National Accelerator Laboratory (FNAL) has acquired experimental data through quench antenna arrays that are recorded when the quench is detected. These data are in terms of voltage signals that are sampled at 100kHz for several minutes. There are multiple channels and each channel provides a data set of more than 20 million observations, while there is one channel, called the trigger channel which shows the time when quench is detected. Despite some advancements that were made including machine learning, data complexity still shadows the progress. In this work, we studied a multi-resolution analysis of the quench antenna data through the Haar wavelet transform. In particular, we applied the maximally overlapped discrete w avelet transform (MODWT) of a suitable level L to the given data and then projected it onto the wavelet basis. This decomposes a given signal (Original data) $x ϵ \mathbb{R}^N$ into $L + 1$ subspaces of $\mathbb{R}^N$. One of the subspaces called the approximation, captures the trend of the signal, and the others, called the details, capture the fluctuations at different frequency bands. This decomposition provides a clear trend of the data at a suitable level and also various activities (spikes) are seen in the details of the decomposition at every level. These spikes might reveal some information about the quench under investigation but in any case, give information about magnet behavior. Also, this decomposition is seen to be very useful in removing noise present in the data due to the source or mechanism of the experiment.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

A variable high-order shock-capturing finite difference method with GP-WENO

Here, we present a new finite difference shock-capturing scheme for hyperbolic equations on static uniform grids. The method provides selectable high-order accuracy by employing a kernel-based Gaussian Process (GP) data prediction method which is an extension of the GP high-order method originally introduced in a finite volume framework by the same authors. The method interpolates Riemann states to high order, replacing the conventional polynomial interpolations with polynomial-free GP-based interpolations. For shocks and discontinuities, this GP interpolation scheme uses a nonlinear shock handling strategy similar to Weighted Essentially Non-Oscillatory (WENO), with a novelty consisting in the fact that nonlinear smoothness indicators are formulated in terms of the Gaussian likelihood of the local stencil data, replacing the conventional L 2 -type smoothness indicators of the original WENO method. We demonstrate that these GP-based smoothness indicators play a key role in the new algorithm, providing significant improvements in delivering high – and selectable – order accuracy in smooth flows, while successfully delivering non-oscillatory solution behavior in discontinuous flows.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Tracking Volumetric Units in Modular Factories for Automated Progress Monitoring Using Computer Vision

The construction industry is increasingly adopting off-site and prefabricated methods due to advantages offered in safety, quality, and lead time. Applying industrialized methods for plant management in offsite construction factories requires the collection of large volumes of production process data, which is a tedious task when performed manually. Recent attempts to automate this process have relied on sensor-based data collection methods which are susceptible to noise, expensive, and difficult to validate. Computer vision methods, however, enable process data collection from videos without the limitations of the other sensor-based methods. This technology has not been applied for offsite construction except in very few instances and therefore, this study proposes a novel method to reliably collect the production process data using computer vision method in near real-time from widely used surveillance cameras in offsite construction. The proposed method allows the user to annotate the workstations of interest on the video as ground truths and process these areas throughout the entire video to track the units entering and leaving stations, while continuously updating a near real-time schedule of the production line. This framework was validated by implementing on the surveillance videos of the production process of modular home manufacturing in a factory. The results consistently provided 100% accuracy, after denoising, for all the videos processed including 60 h of work for a station. The developed method enables real-time tracking of station performance, which can enable continuous improvement methods for factory management and resource allocation.

computer vision↗
DOE OSTI2022

A Combined Computer Vision and Deep Learning Approach for Rapid Drone-Based Optical Characterization of Parabolic Troughs

Optical accuracy is a primary driver of parabolic trough concentrating solar power (CSP) plant performance, but can be damaged by wind loads, gravity, error during installation, and regular plant operation. Collecting and analyzing optical measurements over an entire operating parabolic trough plant is difficult, given the large scale of typical installations. Distant Observer, a software tool developed at the National Renewable Energy Laboratory, uses images of the absorber tube reflected in the collector mirror to measure both surface slope in the parabolic mirror and offset of the absorber tube from the ideal focal point. This technology has been adapted for fast data collection using low-cost commercial drones, but until recently still required substantial human labor to process large amounts of data. A new method leveraging advanced deep learning and computer vision tools can drastically reduce the time required to process images. This new method addresses the primary analysis bottleneck, identifying featureless, reflective mirror corner points to a high degree of accuracy. Recent work has shown promising results using computer vision methods. The combined deep learning and computer vision approach presented here proved highly effective and has the potential to further automate data collection and analysis, making the tool more robust. The method presented in this paper automatically identified 74.3% of mirror corners within 2 pixels of their manually marked counterparts and 91.9% within 3 pixels. This level of accuracy is sufficient for practical Distant Observer analysis within a target uncertainty. A commercial drone collected video of over 100 parabolic trough modules at an operating CSP plant to demonstrate the deep learning and computer vision method's usefulness in processing large amounts of data. These troughs were successfully analyzed using Distant Observer, paired with the new deep learning and computer vision algorithm, and can provide plant operators and trough designers with valuable insight about plant performance, operating strategies, and plant-wide optical error trends.

computer vision↗
DOE OSTI2020

Big Data Processing for Power Grid Event Detection

In this paper we present the application of big data processing for the development of machine learning (ML) models to detect relevant events in power grid operations. This is based on almost 20TB of phasor measurement unit data corresponding to up to two years of operation of three grid interconnections which provide power to most of the United States. A significant aspect of the work consists in having all data processing performed on a single standard GPU server, from pre-processing to ML model training and testing. We describe the data and computational infrastructure, challenges faced and methods used in data processing, main findings and results. The ML approach employed for best utilization of the big data is also discussed, including sample results.

Paes Leao, Bruno↗
DOE OSTI2025

Machine Learning-Driven Conservative-to-Primitive Conversion in Hybrid Piecewise Polytropic and Tabulated Equations of State

We present a novel machine learning (ML)-based method to accelerate conservative-to-primitive inversion, focusing on hybrid piecewise polytropic and tabulated equations of state. Traditional root-finding techniques are computationally expensive, particularly for large-scale relativistic hydrodynamics simulations. To address this, we employ feedforward neural networks (NNC2PS and NNC2PL), trained in PyTorch (2.0+) and optimized for GPU inference using NVIDIA TensorRT (8.4.1), achieving significant speedups with minimal accuracy loss. The NNC2PS model achieves 𝐿 1 and 𝐿 ∞ errors of 4.54 × 10 −7 and 3.44 × 10−6, respectively, while the NNC2PL model exhibits even lower error values. TensorRT optimization with mixed-precision deployment substantially accelerates performance compared to traditional root-finding methods. Specifically, the mixed-precision TensorRT engine for NNC2PS achieves inference speeds approximately 400 times faster than a traditional single-threaded CPU implementation for a dataset size of 1,000,000 points. Ideal parallelization across an entire compute node in the Delta supercomputer (dual AMD 64-core 2.45 GHz Milan processors and 8 NVIDIA A100 GPUs with 40 GB HBM2 RAM and NVLink) predicts a 25-fold speedup for TensorRT over an optimally parallelized numerical method when processing 8 million data points. Moreover, the ML method exhibits sub-linear scaling with increasing dataset sizes. We release the scientific software developed, enabling further validation and extension of our findings. By exploiting the underlying symmetries within the equation of state, these findings highlight the potential of ML, combined with GPU optimization and model quantization, to accelerate conservative-to-primitive inversion in relativistic hydrodynamics simulations.

conservative-to-primitive conversion↗
DOE OSTI2022

Data Mining and Visualization of High-Dimensional ICME Data for Additive Manufacturing

Integrated computational materials engineering (ICME) methods combining CALPHAD with process-based simulations can produce rich, high-dimensional data for alloy and process design. In ICME methods for metallurgical applications, the visualization and interpretation of such high-dimensional data has previously been through heat maps represented in 2 or 3 dimensions. While such an approach is ideal when one variable is varied at a time, in the case of high-dimensional data with multiple variables varied simultaneously, as is the case in additive manufacturing, interpreting the trends through two- or three-dimensional heat maps becomes challenging. Here, we propose a strategy of mixed visual data mining and quantitative analysis for high-dimensional metallurgical and process data using high-throughput thermodynamic calculations. Two case studies show the application of the proposed approach. The first case study investigated the effects of feedstock chemistry on the δ ferrite formation in 316L stainless steel powders used for binder jet additive manufacturing. The second case study linked Scheil–Gulliver calculations to a process model for dissimilar joining of aluminum alloys 5356 and 6111 during laser hot-wire additive manufacturing. Both cases contained thousands of calculated data points, showcasing the utility of visual data analysis through parallel coordinate plotting, Pearson correlation coefficient matrices, and scatter matrices compared to traditional process maps. These visualization techniques can be extended to many additive manufacturing problems to capture process–structure–property relationships for additively manufactured components.

36 MATERIALS SCIENCE↗
DOE OSTI2021

An Intelligent Garbage Sorting System Based on Edge Computing and Visual Understanding of Social Internet of Vehicles

In order to enable Social Internet of Vehicles devices to achieve the purpose of intelligent and autonomous garbage classification in a public environment, while avoiding network congestion caused by a large amount of data accessing the cloud at the same time, it is therefore considered to combine mobile edge computing with Social Internet of Vehicles to give full play to mobile edge computing features of high bandwidth and low latency. At the same time, based on cutting-edge technologies such as deep learning, knowledge graph, and 5G transmission, the paper builds an intelligent garbage sorting system based on edge computing and visual understanding of Social Internet of Vehicles. First of all, for the massive multisource heterogeneous Social Internet of Vehicles big data in the public environment, different item modal data adopts different processing methods, aiming to obtain a visual understanding model. Secondly, using the 5G network, the model is deployed on the edge device and the cloud for cloud-side collaborative management, aiming to avoid the waste of edge node resources, while ensuring the data privacy of the edge node. Finally, the Social Internet of Vehicles devices is used to make intelligent decision-making on the big data of the items. First, the items are judged as garbage, and then the category is judged, and finally the task of grabbing and sorting is realized. The experimental results show that the system proposed in this paper can efficiently process the big data of Social Internet of Vehicles and make valuable intelligent decisions. At the same time, it also has a certain role in promoting the promotion of Social Internet of Vehicles devices.

Shen, Xuehao↗
DOE OSTI2024

Special Observing Period (SOP) data for the Year of Polar Prediction site Model Intercomparison Project (YOPPsiteMIP)

The rapid changes occurring in the polar regions require an improved understanding of the processes that are driving these changes. At the same time, increased human activities such as marine navigation, resource exploitation, aviation, commercial fishing, and tourism require reliable and relevant weather information. One of the primary goals of the World Meteorological Organization's Year of Polar Prediction (YOPP) project is to improve the accuracy of numerical weather prediction (NWP) at high latitudes. During YOPP, two Canadian “supersites” were commissioned and equipped with new ground-based instruments for enhanced meteorological and system process observations. Additional pre-existing supersites in Canada, the United States, Norway, Finland, and Russia also provided data from ongoing long-term observing programs. These supersites collected a wealth of observations that are well suited to address YOPP objectives. In order to increase data useability and station interoperability, novel Merged Observatory Data Files (MODFs) were created for the seven supersites over two Special Observing Periods (February to March 2018 and July to September 2018). All observations collected at the supersites were compiled into this standardized NetCDF MODF format, simplifying the process of conducting pan-Arctic NWP verification and process evaluation studies. This paper describes the seven Arctic YOPP supersites, their instrumentation, data collection and processing methods, the novel MODF format, and examples of the observations contained therein. MODFs comprise the observational contribution to the model intercomparison effort, termed YOPP site Model Intercomparison Project (YOPPsiteMIP). All YOPPsiteMIP MODFs are publicly accessible via the YOPP Data Portal (Whitehorse: https://doi.org/10.21343/a33e-j150, Huang et al., 2023a; Iqaluit: https://doi.org/10.21343/yrnf-ck57, Huang et al., 2023b; Sodankylä: https://doi.org/10.21343/m16p-pq17, O'Connor, 2023; Utqiagvik: https://doi.org/10.21343/a2dx-nq55, Akish and Morris, 2023c; Tiksi: https://doi.org/10.21343/5bwn-w881, Akish and Morris, 2023b; Ny-Ålesund: https://doi.org/10.21343/y89m-6393, Holt, 2023; and Eureka: https://doi.org/10.21343/r85j-tc61, Akish and Morris, 2023a), which is hosted by MET Norway, with corresponding output from NWP models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Compactly‐Supported Nonstationary Kernels for Computing Exact Gaussian Processes on Big Data

The Gaussian process (GP) is a widely used method for analyzing large-scale data sets, including spatio-temporal measurements of nonlinear processes that are now commonplace in the environmental sciences. Traditional implementations of GPs involve stationary kernels (also termed covariance functions) that limit their flexibility, and exact methods for inference that prevent application to data sets with more than about 10,000 points. Modern approaches to address stationarity assumptions generally fail to accommodate large data sets, while all attempts to address scalability focus on approximating the Gaussian likelihood, which can involve subjectivity and lead to inaccuracies. In this work, we explicitly derive an alternative kernel that can discover and encode both sparsity and nonstationarity. We embed the kernel within a fully Bayesian GP model and leverage high-performance computing resources to enable the analysis of massive data sets. We demonstrate the favorable performance of our novel kernel relative to existing exact and approximate GP methods across a variety of synthetic data examples. Furthermore, we conduct space–time prediction based on more than 1 million measurements of daily maximum temperature and verify that our results outperform state-of-the-art methods in the Earth sciences. More broadly, having access to exact GPs that use ultra-scalable, sparsity-discovering, nonstationary kernels allows GP methods to truly compete with a wide variety of machine learning methods.

Gaussian processes↗
DOE OSTI2022

Solar data uncertainty impacts on MCMC methods for r-process nucleosynthesis

In recent work, we developed a Markov Chain Monte Carlo (MCMC) procedure to predict the ground state masses capable of forming the observed Solar r-process rare-earth abundance peak. By applying this method to nucleosynthesis calculations which make use of distinct astrophysical conditions and comparing our results to the latest precision mass measurements, we are able to shed light on the conditions/masses capable of producing a rare-earth peak which matches Solar data. Here we examine how our mass predictions change when using a few different sets of r-process Solar abundance residuals that have been reported in the literature. We explore how the differing error estimates of these Solar evaluations propagate through the Markov Chain Monte Carlo to our mass predictions. We find that Solar data which reports the rare-earth peak to have its highest abundance at mass number A = 162 can require distinctly different mass predictions from data with the peak centered at A = 164. Nevertheless, we find that two important general conclusions from past work, regarding the inconsistency of ‘cold’ astrophysical outflows with current mass measurements and the need for local stability at N = 104 in ‘hot’ scenarios, remain robust in the face of differing Solar data evaluations. Additionally, we show that the masses our procedure finds capable of producing a peak at A < 164 are not in line with the latest precision mass measurements.

79 ASTRONOMY AND ASTROPHYSICS↗