Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data reduction methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Monte Carlo simulation of an expanding gas

By application of simple computer graphics techniques, the statistical performance of two Monte Carlo methods used in the simulation of rarefied gas flows are assessed. Specifically, two direct simulation Monte Carlo (DSMC) methods developed by Bird and Nanbu are considered. The graphics techniques are found to be of great benefit in the reduction and interpretation of the large volume of data generated, thus enabling important conclusions to be drawn about the simulation results. Hence, it is discovered that the method of Nanbu suffers from increased statistical fluctuations, thereby prohibiting its use in the solution of practical problems.

Boyd, Iain D.↗

Sensitivity and uncertainty analysis applied to the JHR reactivity prediction

The on-going AMMON program in EOLE reactor at CEA Cadarache (France) provides experimental results to qualify the HORUS-3D/N neutronics calculation scheme used for the design and safety studies of the new Material Testing Jules Horowitz Reactor (JHR). This paper presents the determination of technological and nuclear data uncertainties on the core reactivity and the propagation of the latter from the AMMON experiment to JHR. The technological uncertainty propagation was performed with a direct perturbation methodology using the 3D French stochastic code TRIPOLI4 and a statistical methodology using the 2D French deterministic code APOLLO2-MOC which leads to a value of 289 pcm (1{sigma}). The Nuclear Data uncertainty propagation relies on a sensitivity study on the main isotopes and the use of a retroactive marginalization method applied to the JEFF 3.1.1 {sup 27}Al evaluation in order to obtain a realistic multi-group covariance matrix associated with the considered evaluation. This nuclear data uncertainty propagation leads to a K{sub eff} uncertainty of 624 pcm for the JHR core and 684 pcm for the AMMON reference configuration core. Finally, transposition and reduction of the prior uncertainty were made using the Representativity method which demonstrates the similarity of the AMMON experiment with JHR (the representativity factor is 0.95). The final impact of JEFF 3.1.1 nuclear data on the Begin Of Life (BOL) JHR reactivity calculated by the HORUS-3D/N V4.0 is a bias of +216 pcm with an associated posterior uncertainty of 304 pcm (1{sigma}). (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

On the estimation of statistical uncertainties on powder diffraction and small-angle scattering data from two-dimensional X-ray detectors

Optimal methods are explored for obtaining one-dimensional powder pattern intensities from two-dimensional planar detectors with good estimates of their standard deviations. Methods are described to estimate uncertainties when the same image is measured in multiple frames as well as from a single frame. The importance of considering the correlation of diffraction points during the integration and the resampling process of data analysis is shown. It is found that correlations between adjacent pixels in the image can lead to seriously overestimated uncertainties if such correlations are neglected in the integration process. Off-diagonal entries in the variance–covariance (VC) matrix are problematic as virtually all data processing and modeling programs cannot handle the full VC matrix. It is shown that the off-diagonal terms come mainly from the pixel-splitting algorithm used as the default integration algorithm in many popular two-dimensional integration programs, as well as from rebinning and resampling steps later in the processing. When the full VC matrix can be propagated during the data reduction, it is possible to get accurate refined parameters and their uncertainties at the cost of increasing computational complexity. However, as this is not normally possible, the best approximate methods for data processing in order to estimate uncertainties on refined parameters with the greatest accuracy from just the diagonal variance terms in the VC matrix is explored.

Yang, X.↗

Study to demonstrate the feasibility of and determine the optimum method of remote haze monitoring by satellite

The author has identified the following significant results. Ground truth data has been obtained in order to determine the amount of haze present at the time of ERTS-1 passes over the Los Angeles area. October 21, 1972 was the haziest day, that was essentially cloud free, to date. At the time of the next pass, November 8, 1972 this area was quite clear. Changes in terrain features are minimized because these were consecutive passes. Digital tapes of the ERTS-1 data from these two days are being analyzed to determine the effects of haze on the data, in order to determine the optimum method for monitoring haze from a satellite. Preliminary results indicate that a slight reduction in contrast and some distortion of spectral signatures are caused by heavy haze. This effect is thought to be sufficient to determine the presence of moderate haze without the benefit of ground truth data. It is significant to note that there is very little degradation of ERTS-1 data caused be a rather heavy haze. For many purposes the data obtained on a hazy day are just as useful as that obtained on a clear day.

Rogers, E. H.↗

A Wall Correction Program Based on Classical Methods for the National Transonic Facility (Solid Wall or Slotted Wall) and the 14x22-Ft Subsonic Tunnel at NASA LaRC

A Fortran subroutine CMWALL is described, which is an implementation of the collective information from classical methods-based wall corrections. These methods use established closed-form expressions which were developed based on simple linear potential-based methods. This is a simple and rapid tool to calculate corrections due to wall interference in the National Transonic Facility (Solid Wall or Slotted Wall) or the 14x22-Ft Subsonic Tunnel at NASA LaRC. It is designed to be easily implemented in the existing tunnel data reduction programs, either as real-time or post-point. It is however important to realize that the method is based on the simplifying assumptions of linearity, small model and attached flow. The computed results are thus to be viewed as first-cut estimates, to be refined further using more complex methods based on measured wall pressures (known as wall signature methods).

Hemsch, Michael J.↗

The reduction of airplane flight test data to standard atmosphere conditions

This report was prepared for the National Advisory Committee for Aeronautics in order to supply the need of practical methods of reducing observed performance to standard conditions with a minimum of labor. The first part gives a very simple approximate method of reducing performance in climb, and is particularly adapted to work not requiring extreme accuracy. The second part gives a somewhat more elaborate and more accurate method which is well suited to general flight test reduction. The third part gives the conventional method of calibrating air-speed indicators and reducing the indicated speeds to true air speeds. An appendix gives working tables and charts for the standard atmosphere. (author)

Diehl, Walter S↗

Electronic damage in S atoms in a native protein crystal induced by an intense X-ray free-electron laser pulse

Current hard X-ray free-electron laser (XFEL) sources can deliver doses to biological macromolecules well exceeding 1 GGy, in timescales of a few tens of femtoseconds. During the pulse, photoionization can reach the point of saturation in which certain atomic species in the sample lose most of their electrons. This electronic radiation damage causes the atomic scattering factors to change, affecting, in particular, the heavy atoms, due to their higher photoabsorption cross sections. Here, it is shown that experimental serial femtosecond crystallography data collected with an extremely bright XFEL source exhibit a reduction of the effective scattering power of the sulfur atoms in a native protein. Quantitative methods are developed to retrieve information on the effective ionization of the damaged atomic species from experimental data, and the implications of utilizing new phasing methods which can take advantage of this localized radiation damage are discussed.

Galli, L.↗

Crystalline Silicon Photovoltaic Module Manufacturing Costs and Sustainable Pricing: 1H 2018 Benchmark and Cost Reduction Road Map

In this paper we provide an overview of the accounting methods and most recent input data used within NREL's bottom-up crystalline silicon (c-Si) solar photovoltaic (PV) module supply chain cost models. The costs accounting framework that is detailed in this paper provides a methodology to prepare bottom-up c-Si PV supply chain cost models in accordance with the U.S. Generally Accepted Accounting Principles (GAAP) and the International Financial Reporting Standards (IFRS). For the polysilicon, ingot, wafer, cell conversion, and module assembly steps of the c-Si supply chain, we outline what is relevant for calculations of the costs of goods sold (consisting of the materials, labor, electricity, maintenance, and depreciation of capital equipment); as well as the additional operating expenses of research and development (R&D); and sales, general, and business administration (S, G, &A). This 2017 benchmark analysis is applied to current state-of-the-art manufacturing sites located within the United States, China, Taiwan, South Korea, Malaysia, the Philippines, and Germany. Primarily due to country-dependent labor and electricity rates, tax rates, and cost-of-capital considerations, we calculate differences in minimum sustainable pricing for all of these countries. In this paper we also include technical descriptions and bottom-up cost model results for several emerging c-Si technologies including larger Czochralski (Cz) ingots, kerfless wafers, reduced silicon utilization by thinner wafers and advanced diamond wire wafering, new encapsulants and backsheets, shingled cells, and the cell architectures passivated emitter and rear cells (PERC), bifacial passivated emitter and rear totally diffused (PERT) cells, bifacial heterojunction intrinsic thin layer (HIT) cells, and interdigitated all back-contact cells (IBC).

14 SOLAR ENERGY↗

Random projection using random quantum circuits

The random sampling task performed by Google's Sycamore processor gave us a glimpse of the “quantum supremacy era.” This has definitely shed some light on the power of random quantum circuits in this abstract task of sampling outputs from the (pseudo)random circuits. In this paper, we explore a practical near-term use of local random quantum circuits in dimensional reduction of large low-rank data sets. We make use of the well-studied dimensionality reduction technique called the random projection method. This method has been extensively used in various applications such as image processing, logistic regression, entropy computation of low-rank matrices, etc. We prove that the matrix representations of local random quantum circuits with sufficiently shorter depths [ ∼ O ( n ) ] serve as good candidates for random projection. We demonstrate numerically that their projection abilities are not far off from the computationally expensive classical principal components analysis on MNIST and CIFAR-100 image datasets. We also benchmark the performance of quantum random projection against the commonly used classical random projection in the tasks of dimensionality reduction of image data sets and computing von Neumann entropies of large low-rank density matrices. And finally, using variational quantum singular value decomposition, we demonstrate a near-term implementation of extracting the singular vectors with dominant singular values after quantum random projecting a large low-rank matrix to lower dimensions. All such numerical experiments unequivocally demonstrate the ability of local random circuits to randomize a large Hilbert space at sufficiently shorter depths with robust retention of properties of large data sets in reduced dimensions. Published by the American Physical Society 2024

Kumaran, Keerthi (ORCID:0009000949125721)↗

Constitutive modeling for isotropic materials

The first year of progress on a NASA-Lewis contract with the General Electric Co is documented. The purpose of this contract (NAS3-23927) is to develop and evaluate unified constitutive equations for applications to hot-path components of aircraft gas turbine engines such as high pressure turbine blades and vanes. To accomplish this goal, uniaxial, notched, and multiaxial specimens made of conventionally cast Rene 80 are being tested under conditions that simulate engine operating conditions. To reduce the raw data, automated data reduction techniques are being developed that produce computer files containing the information needed to analyze proposed constitutive theories. Described are the analytical methods being developed to determine the parameters for these nonlinear unified theories by using the reduced data files. In another activity, a dedicated finite-element computer code is being developed to use unified theories in the structural analysis of hot-section components. This code was extensively verified for one such theory by successfully predicting the strain histories measured experimentally at the notch root of complex specimens taken from complex laboratory specimens.

Ramaswamy, V. G.↗

Denoising Autoencoder for Reconstructing Sensor Observation Data and Predicting Evapotranspiration: Noisy and Missing Values Repair and Uncertainty Quantification

Abstract Machine learning (ML) methods applied in scientific research often deal with interrelated features in high‐dimensional data. Reducing data noise and redundancy is needed to increase prediction accuracy and efficiency especially when dealing with data from field sensors. We explored an unsupervised learning method, the denoising autoencoder (DAE), to extract the underlying data structure from noisy raw data in the context of predicting hydrologic quantities from multiple field sensors. These sensors have intrinsic instrumental noise and occasional malfunctions that cause missing values. Our DAE neural network reconstructed meteorological sensor data containing noise and missing values to predict evapotranspiration in a mountainous watershed. The DAE reconstructed the sensor variables with a mean coefficient of determination value of 0.77 across 15 dimensions representing individual sensors. It reduced variance and bias uncertainties compared to a classical autoencoder model. The reconstruction quality varied across dimensions depending on their cross‐correlation and alignment with the underlying data structure. Uncertainties arising from the model structure were overall higher than those resulting from data corruption. We attached the DAE structure to a downstream ET‐prediction neural network in three formats and achieved reasonably accurate ET predictions . The use of the DAE notably reduced variance uncertainty in ET prediction. However, excessive variance reduction may be accompanied by an increase in bias due to the intrinsic bias‐variance tradeoff. Our method of evaluating and reducing uncertainties in aggregated data from different sources can be used to improve predictive models, process understanding, and uncertainty quantification for better water resource management. Plain Language Summary We present a machine learning method, namely the denoising autoencoder, which reduces the effects of data noise and missing values typically present in scientific data sets collected through sensor measurements. This method selects the most relevant information from noisy raw data collected by the instruments and fills in missing values. To demonstrate the effectiveness of our method, we applied it to predict evapotranspiration, a hydrologic variable that represents the water moved from the land surface to the atmosphere through a combination of evaporation and plant water use (transpiration). We also used a random sampling technique (the Monte Carlo method) to compare the uncertainty in the predictions when using the raw and noisy data versus the reconstructed data. The denoising process produced more accurate predictions of evapotranspiration with less uncertainty. Improved predictions of evapotranspiration can lead to a better understanding and accounting of water budgets. This ML approach is broadly suitable for a wide variety of applications that involve noisy sensor data with missing values. Key Points We used a denoising autoencoder (DAE) neural network to reduce noise in meteorological and soil sensor observations by on average We used Monte Carlo sampling to estimate the bias and variance of all model outputs, including uncertainty sources from data and the model We attached the DAE component to a downstream neural network to predict ET with the variance reduced by , compared to that without the DAE

denoising autoencoder↗

Trend Detection of Atmospheric Time Series: Incorporating Appropriate Uncertainty Estimates and Handling Extreme Events

This paper is aimed at atmospheric scientists without formal training in statistical theory. Its goal is to, 1) provide a critical review of the rationale for trend analysis of the time series typically encountered in the field of atmospheric chemistry; 2) describe a range of trend-detection methods; and 3) demonstrate effective means of conveying the results to a general audience. Trend detections in atmospheric chemical composition data are often challenged by a variety of sources of uncertainty, which often behave differently to other environmental phenomena such as temperature, precipitation rate, or stream flow, and may require specific methods depending on the science questions to be addressed. Some sources of uncertainty can be explicitly included in the model specification, such as autocorrelation and seasonality, but some inherent uncertainties are difficult to quantify, such as data heterogeneity and measurement uncertainty due to the combined effect of short- and long-term natural variability, instrumental stability, and aggregation of data from sparse sampling frequency. Failure to account for these uncertainties might result in an inappropriate inference of the trends and their estimation errors. On the other hand, the variation in extreme events might be interesting for different scientific questions, for example, the frequency of extremely high surface ozone events and their relevance to human health. In this study we aim to, 1) review trend detection methods for addressing different levels of data complexity in different chemical species; 2) demonstrate that the incorporation of scientifically interpretable covariates can outperform pure numerical curve fitting techniques in terms of uncertainty reduction and improved predictability; 3) illustrate the study of trends based on extreme quantiles that can provide insight beyond standard mean or median based trend estimates; and 4) present an advanced method of quantifying regional trends based on the inter-site correlations of multi-site data. All demonstrations are based on time series of observed trace gases relevant to atmospheric chemistry, but the methods can be applied to other environmental data sets.

Trace gas↗

KEPLER ECLIPSING BINARY STARS. III. CLASSIFICATION OF KEPLER ECLIPSING BINARY LIGHT CURVES WITH LOCALLY LINEAR EMBEDDING

We present an automated classification of 2165 Kepler eclipsing binary (EB) light curves that accompanied the second Kepler data release. The light curves are classified using locally linear embedding, a general nonlinear dimensionality reduction tool, into morphology types (detached, semi-detached, overcontact, ellipsoidal). The method, related to a more widely used principal component analysis, produces a lower-dimensional representation of the input data while preserving local geometry and, consequently, the similarity between neighboring data points. We use this property to reduce the dimensionality in a series of steps to a one-dimensional manifold and classify light curves with a single parameter that is a measure of 'detachedness' of the system. This fully automated classification correlates well with the manual determination of morphology from the data release, and also efficiently highlights any misclassified objects. Once a lower-dimensional projection space is defined, the classification of additional light curves runs in a negligible time and the method can therefore be used as a fully automated classifier in pipeline structures. The classifier forms a tier of the Kepler EB pipeline that pre-processes light curves for the artificial intelligence based parameter estimator.

79 ASTRONOMY AND ASTROPHYSICS↗

SNS: A Solution-Based Nonlinear Subspace Method for Time-Dependent Model Order Reduction

Several reduced order models have been successfully developed for nonlinear dynamical systems. To achieve a considerable speed-up, a hyper-reduction step is needed to reduce the computational complexity due to nonlinear terms. Many hyper-reduction techniques require the construction of nonlinear term basis, which introduces a computationally expensive offline phase. A novel way of constructing nonlinear term basis within the hyper-reduction process is introduced. In contrast to the traditional hyper-reduction techniques where the collection of nonlinear term snapshots is required, the SNS method avoids collecting the nonlinear term snapshots. Instead, it uses the solution snapshots that are used for building a solution basis, which enables avoiding an extra data compression of nonlinear term snapshots. As a result, the SNS method provides a more efficient offline strategy than the traditional model order reduction techniques, such as the DEIM, GNAT, and ST-GNAT methods. The SNS method is theoretically justified by the conforming subspace condition and the subspace inclusion relation. It is useful for model order reduction of large-scale nonlinear dynamical problems to reduce the offline cost. It is especially useful for ST-GNAT that has shown promising results, such as a good accuracy with a considerable online speed-up for hyperbolic problems in a recent paper by Choi and Carlberg [SIAM J. Sci. Comput., 41 (2019), pp. A26--A58], because ST-GNAT involves an expensive offline cost related to collecting nonlinear term snapshots. Error analysis for the SNS method is presented. Numerical results support that the accuracy of the solution from the SNS method is comparable to the traditional methods and a considerable speed-up (i.e., a factor of two to a hundred) is achieved in the offline phase.

97 MATHEMATICS AND COMPUTING↗

From microbial diversity to functional potential using dimensionality reduction

The high dimensionality of microbial diversity data from ‘omics observations can be reduced using Machine Learning, with many recent studies showcasing ML utility for exploratory ecological feature finding and process prediction. Here, we compare the Self Organizing Map (SOM) dimensionality reduction method to the well-documented sample-based Principal Coordinate Analysis (PCoA) and taxa-based Weighted Gene Correlation Network Analysis (WGCNA) using near daily 16S rRNA gene amplicon sequencing data from the 2019 to 2020 MOSAiC International Arctic Drift Expedition. We then map k-means clustering outputs from each method to available metagenomes, extracting functionally distinct seasonal microbial ecotypes in the surface Arctic Ocean. Our results indicate the SOM method better represented expected seasonal transitions and identified a greater number of metabolically distinct functional groups than the more traditional PCoA ordination. Ultimately, we identified four community ecotypes with distinct taxonomic and functional cut-offs driven by seasonality, water mass, and substrate turnover, highlighting the importance of succession in functional diversity for the central Arctic Ocean. These results reinforce ML dimensionality reduction as a meaningful translator in the mining of historical amplicon datasets to address modern mechanistic questions and potentially provide ’omics informed ecotype diversity to leverage in mechanistic biogeochemical models.

Arctic Ocean↗

Deep Hierarchical Super Resolution for Scientific Data

We present a novel technique for hierarchical super resolution (SR) with neural networks (NNs), which upscales volumetric data represented with an octree data structure to a high-resolution uniform gridwith minimal seam artifacts on octree node boundaries. Our method uses existing state-of-the-art SR models and adds flexibility to upscale input data with varying levels of detail across the domain, instead of only uniform grid data that are supported in previous approaches.The key is to use a hierarchy of SR NNs, each trained to perform 2x SR between two levels of detail, with a hierarchical SR algorithm that minimizes seam artifacts by starting from the coarsest level of detail and working up.We show that our hierarchical approach outperforms baseline interpolation and hierarchical upscaling methods, and demonstrate the usefulness of our proposed approach across three use cases including data reduction using hierarchical downsampling+SR instead of uniform downsampling+SR, computation savings for hierarchical finite-time Lyapunov exponent field calculation, and super-resolving low-resolution simulation results for a high-resolution approximation visualization.

97 MATHEMATICS AND COMPUTING↗

CV4Quantum

CV4Quantum is a statistical technique for reducing the sampling overhead in probabilistic error cancellation, which is an error mitigation technique used in quantum computing. CV4Quantum is based on the control variates method, which is a Monte Carlo variance reduction technique. This repository contains the code and data associated with a demonstration of CV4Quantum using simulation experiments.

Shyamsundar, Prasanth [Fermi National Accelerator ↗

Visual enhancement of images of natural resources: Applications in geology

The principal components technique for use in multispectral scanner LANDSAT data processing results in optimum dimensionality reduction. A powerful tool for MSS IMAGE enhancement, the method provides a maximum impression of terrain ruggedness; this fact makes the technique well suited for geological analysis.

Dejesusparada, N.↗