Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data reduction methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29

ClassNMSW- a real-time classification approach for non-recycled municipal solid waste using hyperspectral imaging

Real-time classification of non-recycled municipal solid waste (NMSW) is essential for efficient valorization. This study introduces ClassNMSW, a comprehensive framework for classifying 22 NMSW subclasses under industrial constraints by using hyperspectral imaging (HSI). A primary innovation of this work is the development of a variance-controlled spectral extraction algorithm. Unlike traditional methods that rely on simple averaging, this approach systematically investigates the extent of pixel extraction to minimize the loss of critical chemical information while maximizing data reduction thus ensuring high spectral fidelity with low computational cost. The approach developed in this work integrates automated, computer-vision-based background removal, eliminating the need for the manual thresholding common in current literature. To resolve ambiguities among chemically similar subclasses, a tiered classification and multi-camera fusion strategy (NIR17 and NIR22) is implemented. Results demonstrate that ClassNMSW achieves an object-wise weighted accuracy of 98.70% for single-sensor configurations and 100% under sensor fusion. A novel rolling-window strategy satisfies desired end-to-end latency of <2 s, satisfying the strict deterministic requirements of high-speed industrial sorting environments. The ClassNMSW framework provides a scalable foundation for advancing circularity and resource recovery in large-scale waste valorization operations.

99 - GENERAL AND MISCELLANEOUS↗

Scientific Data Compression for Large Scale Computational Fluid Dynamics (CFD) Simulations

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and General Electric (GE) investigated methods for reducing the size of large computational fluid dynamics (CFD) simulation datasets using scientific data compression techniques. The work focused on adapting the MultiGrid Adaptive Reduction of Data (MGARD) compression framework and integrating it with high-performance I/O and visualization tools used in CFD workflows. MGARD uses hierarchical multilevel decomposition to enable error-controlled compression of floating-point scientific data while preserving quantities of interest. During the project, MGARD compression was integrated with the ADIOS I/O framework and visualization tools such as ParaView to enable efficient storage, transfer, and analysis of simulation data. The collaboration also explored approaches for improving compression performance for CFD data defined on unstructured meshes. Results demonstrate that scientific data compression can significantly reduce storage requirements and improve data management for large-scale CFD simulations.

97 MATHEMATICS AND COMPUTING↗

Automatic discovery of optimal classes

A criterion, based on Bayes' theorem, is described that defines the optimal set of classes (a classification) for a given set of examples. This criterion is transformed into an equivalent minimum message length criterion with an intuitive information interpretation. This criterion does not require that the number of classes be specified in advance, this is determined by the data. The minimum message length criterion includes the message length required to describe the classes, so there is a built in bias against adding new classes unless they lead to a reduction in the message length required to describe the data. Unfortunately, the search space of possible classifications is too large to search exhaustively, so heuristic search methods, such as simulated annealing, are applied. Tutored learning and probabilistic prediction in particular cases are an important indirect result of optimal class discovery. Extensions to the basic class induction program include the ability to combine category and real value data, hierarchical classes, independent classifications and deciding for each class which attributes are relevant.

Cheeseman, Peter↗

Estimation of full modal damping matrices from complex test modes

This paper describes the refinements of a previously published method for estimating a full modal damping matrix from complex test modes. It also documents application of the refined method to a structure where complex test modes were derived by the ERA method from multi-input random vibration test data. A numerical example based on simulated test data is presented to demonstrate the validity of the method. The application using real data was not successful, presumably because of noise in the small phase angles of the measured complex modes. Alternative test and data reduction procedures are suggested as possible remedies to the problem. A careful analysis of measurement and data processing errors should be made to examine basic feasibility before implementing the alternative procedures. The ability to estimate a full modal damping matrix is considered important for the preflight estimation of on-orbit damping, and for the synthesis of structural damping from substructure tests.

Hasselman, T. K.↗

Smithsonian stratospheric far-infrared spectrometer and data reduction system

The Smithsonian far-infrared spectrometer (FIRS) is a remote sensing Fourier transform spectrometer that measures the mid- and far-infrared thermal emission spectrum of the stratosphere from balloon and aircraft platforms. The spectrometer has had nine successful balloon flights from 1987 to 1994, flying at float altitudes of 36 - 39 km and collecting 131 hours of midlatitude stratospheric limb spectra. The spectrometer also flew on a NASA CD-8 aircraft, as part of the second Airborne Arctic Stratospheric Expedition (AASE-2), collecting 140 hours of overhead spectra at latitudes ranging from the equator to the north pole. We present here a brief description of the instrument, a discussion of data reduction procedures, an estimation of both random and systematic errors, an outline of the procedure for retrieving mixing ratio profiles, and an explanation of the method of deriving temperature and pressure from the far- and mid-infrared spectra.

Johnson, D. G.↗

Hyperspectral Remote Sensing and Ecological Modeling Research and Education at Mid America Remote Sensing Center (MARC): Field and Laboratory Enhancement

The purpose of this project was to establish a new hyperspectral remote sensing laboratory at the Mid-America Remote sensing Center (MARC), dedicated to in situ and laboratory measurements of environmental samples and to the manipulation, analysis, and storage of remotely sensed data for environmental monitoring and research in ecological modeling using hyperspectral remote sensing at MARC, one of three research facilities of the Center of Reservoir Research at Murray State University (MSU), a Kentucky Commonwealth Center of Excellence. The equipment purchased, a FieldSpec FR portable spectroradiometer and peripherals, and ENVI hyperspectral data processing software, allowed MARC to provide hands-on experience, education, and training for the students of the Department of Geosciences in quantitative remote sensing using hyperspectral data, Geographic Information System (GIS), digital image processing (DIP), computer, geological and geophysical mapping; to provide field support to the researchers and students collecting in situ and laboratory measurements of environmental data; to create a spectral library of the cover types and to establish a World Wide Web server to provide the spectral library to other academic, state and Federal institutions. Much of the research will soon be published in scientific journals. A World Wide Web page has been created at the web site of MARC. Results of this project are grouped in two categories, education and research accomplishments. The Principal Investigator (PI) modified remote sensing and DIP courses to introduce students to ii situ field spectra and laboratory remote sensing studies for environmental monitoring in the region by using the new equipment in the courses. The PI collected in situ measurements using the spectroradiometer for the ER-2 mission to Puerto Rico project for the Moderate Resolution Imaging Spectrometer (MODIS) Airborne Simulator (MAS). Currently MARC is mapping water quality in Kentucky Lake and vegetation in the Land-Between-the Lakes (LBL) using Landsat-TM data. A Landsat-TM scene of the same day was obtained to relate ground measurements to the satellite data. A spectral library has been created for overstory species in LBL. Some of the methods, such as NPDF and IDFD techniques for spectral unmixing and reduction of effects of shadows in classifications- comparison of hyperspectral classification techniques, and spectral nonlinear and linear unmixing techniques, are being tested using the laboratory.

Cetin, Haluk↗

Spatially Varying Spectrally Thresholds for MODIS Cloud Detection

The EOS science team has developed an elaborate global MODIS cloud detection procedure, and the resulting MODIS product (MOD35) is used in the retrieval process of several geophysical parameters to mask out clouds. While the global application of the cloud detection approach appears quite robust, the product has some shortcomings on the regional scale, often over determining clouds in a variety of settings, particularly at night. This over-determination of clouds can cause a reduction in the spatial coverage of MODIS derived clear-sky products. To minimize this problem, a new regional cloud detection method for use with MODIS data has been developed at NASA's Global Hydrology and Climate Center (GHCC). The approach is similar to that used by the GHCC for GOES data over the continental United States. Several spatially varying thresholds are applied to MODIS spectral data to produce a set of tests for detecting clouds. The thresholds are valid for each MODIS orbital pass, and are derived from 20-day composites of GOES channels with similar wavelengths to MODIS. This paper and accompanying poster will introduce the GHCC MODIS cloud mask, provide some examples, and present some preliminary validation.

Haines, S. L.↗

Upset Simulation and Training Initiatives for U.S. Navy Commercial Derived Aircraft

Militarized versions of commercial platforms are growing in popularity due to many logistical benefits in the form of commercial off-the-shelf (COTS) parts, established production methods, and commonality for different certifications. Commercial data and best practices are often leveraged to reduce procurement and engineering development costs. While the developmental and cost reduction benefits are clear, these militarized aircraft are routinely operated in flight at significantly different conditions and in significantly different manners than for routine commercial flight. Therefore they are at a higher risk of flight envelope exceedance. This risk may lead to departure from controlled flight and/or aircraft loss1. Historically, the risk of departure from controlled flight for military aircraft has been mitigated by piloted simulation training and engineering analysis of typical aircraft response. High-agility military aircraft simulation databases are typically developed to include high angles of attack (AoA) and sideslip due to the dynamic nature of their missions and have been developed for many tactical configurations over the previous decades. These aircraft simulations allow for a more thorough understanding of the vehicle flight dynamics characteristics at high AoA and sideslip. In recent years, government sponsored research on transport airplane aerodynamic characteristics at high angles of attack has produced a growing understanding of stall/post-stall behavior. This research along with recent commercial airline training initiatives has resulted in improved understanding of simulator-based training requirements and simulator model fidelity.2-5 In addition, inflight training research over the past decade has produced a database of pilot performance and recurrency metrics6. Innovative solutions to aerodynamically model large commercial aircraft for upset conditions such as high AoA, high sideslip, and ballistic damage, as well as capability to accurately account for scaling factors, is necessary to develop realistic engineering and training simulations. Such simulations should significantly reduce the risk of departure from controlled flight, loss of aircraft, and ease the airworthiness certification process. The characteristics of commercial derivative aircraft are exemplified by the P-8A Multi-mission Maritime Aircraft (MMA) aircraft, and the largest benefits of initial investigation are likely to be yielded from this platform. The database produced would also be utilized by flight dynamics engineers as a means to further develop and investigate vehicle flight characteristics as mission tactics evolve through the years ahead. This paper will describe ongoing efforts by the U.S. Navy to develop a methodology for simulation and training for large commercial-derived transport aircraft at unusual attitudes, typically experienced during an aircraft upset. This methodology will be applied to a representative Navy aircraft (P-8A) and utilized to develop a robust simulation that should accurately represent aircraft response in these extremes. Simulation capabilities would then extend to flight dynamics analysis and simulation, as well as potential training applications. Recent evaluations of integrated academic, ground-based simulation, and in-flight upset training will be described along with important lessons learned, specific to military requirements.

Donaldson, Steven↗

MSL Telecom Automated Anomaly Detection

The Mars Science Laboratory (MSL) Telecom Operations Team at the Jet Propulsion Laboratory (JPL) has implemented a machine learning system in order to automate the anomaly detection process as a part of daily operations. Machine learning enables reliable detection of anomalies in Telecom-related telemetry and automated reporting of Telecom subsystem status, resulting in an 90% reduction in team workload and improved anomaly detection reliability. At present, machine learning methods are used to detect: 1. Anomalous long-term trends in telemetry data 2. Anomalous time-domain evolution of telemetry values Both types of anomalies pose their own unique challenges that are addressed in different ways. In the first case, long term trending of daily minima, maximum, and mean telemetry values in temperatures, currents, voltages, and radio frequency (RF) power levels is used in addition to hard threshold safety checks to look for changes in long-term equipment health and performance. Long-term trending methods allow for ordinary seasonal variations in these quantities caused by temperature changes over the course of the Martian year while allowing operators to determine whether current performance remains in line with historical values from previous years. Changes in long-term trends can provide important insights into the health and status of the rover's on-board systems as well as valuable early warning if subtle degradation begins to take hold. But while trending of daily statistics is valuable, it does not detect anomalies in the short-term time evolution of data over the course of minutes or hours during a day, and this task is handled with short-term shape analysis. Principal components analysis (PCA) has been found to provide robust detection of short-term anomalies, and several examples of the use of PCA to detect actual anomalous events will be provided here. In using PCA, we use both the percentage of explained variance and also a log likelihood test on the PCA expansion coefficients to flag telemetry data for human review. Previous work in the field of spacecraft anomaly detection includes [1] for MSL and [2] for some other JPL missions.

Mukai, Ryan↗

Systematic characterization of unknown compounds via dimensionality reduction of time series

Analysis of ambient aerosols provides valuable insight into particle sources and formation chemistry. However, due to the complexity of atmospheric data and the dynamic nature of aerosol composition, a substantial fraction of data often become discarded by conventional analysis methods. Furthermore, a large fraction of chemical species within those data are unidentifiable due to a lack of matching spectral information, resulting in suboptimal characterization of chemical composition. Previous work has demonstrated techniques for cataloging analytes in a chromatographic dataset by deconvolution of mass spectra, but integration of these analytes throughout a large dataset remains time consuming. Here, we present a method to automatically identify an ion for quantitation for single-ion chromatogram based peak fitting and integration, enabling comprehensive integration of analytes with minimal user interaction. The resulting time series are clustered with a machine-learning based dimensionality reduction technique to systematically investigate the underlying characteristics of the categorized analytes and gain new insights into the chemical composition and physicochemical properties of the unidentifiable analytes. We apply these methods to existing atmospheric datasets collected in Manacapuru, Brazil during the GoAmazon2014/5 campaign to identify new analytes and interpret their variability and transformations in the atmosphere. The analysis results generate 408 time series from cataloged analytes of interest, and the clustering of those time series with spherical k-means results in 8 distinct clusters. We find the analytes form clusters based on their distinct physicochemical properties, demonstrating the method’s ability to systematically identify and selectively filter contaminants and instrumental analytes and characterize the unidentifiable analytes.

54 ENVIRONMENTAL SCIENCES↗

Optimum radars and filters for the passive sphere system

Studies have been conducted to determine the influence of the tracking radar and data reduction technique on the accuracy of the meteorological measurements made in the 30 to 100 kilometer altitude region by the ROBIN passive falling sphere. A survey of accuracy requirements was made of agencies interested in data from this region of the atmosphere. In light of these requirements, various types of radars were evaluated to determine the tracking system most applicable to the ROBIN, and methods were developed to compute the errors in wind and density that arise from noise errors in the radar supplied data. The effects of launch conditions on the measurements were also examined. Conclusions and recommendations have been made concerning the optimum tracking and data reduction techniques for the ROBIN falling sphere system.

Luers, J. K.↗

Application of Temperature Sensitivities During Iterative Strain-Gage Balance Calibration Analysis

A new method is discussed that may be used to correct wind tunnel strain-gage balance load predictions for the influence of residual temperature effects at the location of the strain-gages. The method was designed for the iterative analysis technique that is used in the aerospace testing community to predict balance loads from strain-gage outputs during a wind tunnel test. The new method implicitly applies temperature corrections to the gage outputs during the load iteration process. Therefore, it can use uncorrected gage outputs directly as input for the load calculations. The new method is applied in several steps. First, balance calibration data is analyzed in the usual manner assuming that the balance temperature was kept constant during the calibration. Then, the temperature difference relative to the calibration temperature is introduced as a new independent variable for each strain--gage output. Therefore, sensors must exist near the strain--gages so that the required temperature differences can be measured during the wind tunnel test. In addition, the format of the regression coefficient matrix needs to be extended so that it can support the new independent variables. In the next step, the extended regression coefficient matrix of the original calibration data is modified by using the manufacturer specified temperature sensitivity of each strain--gage as the regression coefficient of the corresponding temperature difference variable. Finally, the modified regression coefficient matrix is converted to a data reduction matrix that the iterative analysis technique needs for the calculation of balance loads. Original calibration data and modified check load data of NASA's MC60D balance are used to illustrate the new method.

Ulbrich, N.↗

A transfer learning approach to energy-efficient control of small and medium-sized commercial buildings

Model-free reinforcement learning (RL) provides a data-driven and adaptive approach to optimize building energy use while satisfying occupant comfort. This powerful tool does not need any prior knowledge about the environment and system it is optimizing and can adapt its policy based on the changes in captures. Like any other data-driven tool, it faces high training costs due to the extensive agent-environment interactions required to capture long-term building dynamics and user comfort. Transfer learning, particularly policy distillation, offers a promising way to accelerate training by leveraging pretrained RL agents in different building and system types. Here, this study investigates online student distillation, in which the student model updates its neural network weights using outputs from teacher models. The work introduces a student distillation strategy designed for efficient knowledge transfer, along with a teacher selection method that ensures high-quality guidance. The approach is validated using a highly calibrated whole building energy model for a small/medium commercial building test facility. Results show substantial reductions in training time and data requirements while surpassing the performance of ASHRAE Guideline 36, an advanced rule-based control strategy. The distilled RL model required 45% less data and achieved 20% higher cumulative rewards than a state-of-the-art RL model, with faster convergence and lower energy consumption. These outcomes demonstrate that effective transfer learning enables a scalable and data-efficient energy management solution for commercial buildings.

ASHRAE guideline 36↗

Digital computer modeling of the process of reduction of redundancy in multichannel telemetry information by the method of adaptive discretization with associative sorting

Digital computer modeling of the process of adaptive discretization with associative sorting of actual multichannel telemetry information is discussed. The main task in modeling is production of initial data for determination of dependences describing the operation of the on-board information compression device. Conclusions are presented including the shortcomings of telemetric information used in modeling.

Tolmadzheva, T. A.↗

Microstructural Coarsening Kinetics and Mechanical Property Changes in Long-Term Aged Sn–Pb–Sb Solder Joints

Tin-lead-antimony (50Sn–47Pb–3Sb wt.%) soldered assemblies were mechanically tested approximately 30 years after initial production and found to have solder joints of reduced strength. The microstructure of this solder alloy exhibits a ternary eutectic structure with Sn-rich, Pb-rich, and SnSb phases. Accelerated aging was performed to evaluate solder microstructural coarsening and associated strength of laboratory solder joints to correlate these properties to the “naturally aged” solder joints. Isothermal aging was conducted at room temperature, 55, 70, 100, and 135 °C and aging times that ranged from 0.1 to 365 days. The coarsening kinetics of the Pb-rich phase were determined through optical microscopy and image analysis methods established in previous studies on binary Sn–Pb solder. A kinetic equation was developed with time exponent n of 0.43 and activation energy of 24000 J/mol, suggesting grain boundary diffusion or other fast diffusion pathways controlling the microstructural evolution. Compression testing and Vickers microhardness showed significant strength loss within the first 20–30 days after soldering; then, the microstructure and mechanical properties changed more slowly over long periods of time. Further, by combining accelerated aging data and the microstructure-based kinetics, strength predictions were made that match well with the properties of the actual soldered assemblies naturally aged for 30 years. However, aging at the highest temperature of 135 °C produced anomalous behavior suggesting that extraneous aging mechanisms are active. Therefore, data obtained at this temperature or higher should not be used. Overall, the combined microstructural and mechanical property methods used in this study confirmed that the observed reduction in strength of ~ 30-year-old solder joints can be accounted for by the microstructural coarsening that takes place during long-term solid-state aging.

36 MATERIALS SCIENCE↗

Exploring Continuous Seismic Data at an Industry Facility Using Unsupervised Machine Learning

Seismic data recorded at industrial sites contain valuable information on anthropogenic activities. With advances in machine learning and computing power, new opportunities have emerged to explore the seismic wavefield in these complex environments. We applied two unsupervised machine learning algorithms to analyze continuous seismic data collected from an industrial facility in Texas, United States. The Uniform Manifold Approximation and Projection for Dimension Reduction algorithm was used to reduce the dimensionality of the data and generate 2D embeddings. Then, the Hierarchical Density-Based Spatial Clustering of Applications with Noise method was employed to automatically group these embeddings into distinct signal clusters. Our analysis of over 1400 hr (around 59 days) of continuous seismic data revealed five and seven signal clusters at two separate stations. At both stations, we identified clusters associated with background noise and vehicle traffic, with the latter’s temporal patterns aligning closely with the facility’s work schedule. Furthermore, the algorithms detected signal clusters from unknown sources and underline the ability of unsupervised machine learning for uncovering previously unrecognized patterns. Our analysis demonstrates the effectiveness of unsupervised approaches in examining continuous seismic data without requiring prior knowledge or pre-existing labels.

58 GEOSCIENCES↗

Positive ion densities and mobilities in the upper stratosphere and mesosphere

A brief sketch of the theory concerning the use of the Gerdien condenser as a mobility spectrometer is presented. Data reduction of three parachute borne Gerdien condenser probes is given, as well as that of one blunt conductivity probe. Comparisons of concentrations calculated by two different methods indicate consistency of results. Mobility profiles demonstrating remarkable fine structure are discussed in detail. Finally, theoretical implications of the results on ionospheric structure, including possible night-day differences and latitudinal variations, are considered.

Leiden, S.↗

A miniature high frequency sphere probe

A spherical probe having five surface mounted silicon pressure sensors is described. The probe is 0.2 inches in diameter yielding a frequency response above 30 kHz. A scheme is described for retrieving from the five pressure signals the stagnation and static pressures, Mach number and flow angles in two planes. The calibrations forming the base for this procedure, obtained from steady state tests, are given in the form of polynomials in the two flow angles. A method for compensating the silicon pressure sensors for thermal effects is given. Typical data obtained in the MIT Blowdown Compressor are presented in raw form and after reduction to flow angles, Mach number and stagnation pressure.

Kerrebrock, J. L.↗