Search NASA⌕ Search

SEARCH · Search NASA

Results for “data reduction algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

Scattered light in the echelle modes of the Goddard High Resolution Spectrograph aboard the Hubble Space Telescope. II - Analysis of in-flight spectroscopic observations

We report on an analysis of observations made with the GHRS aboard the HST for the purpose of calibrating the effects of scattered light. The data were obtained during the first 2 years of flight operations as part of the Science Verification, Science Assessment Observations, and Guaranteed Time Observations programs. Both the general level of background light near the spectra and the filling-in of the cores of saturated interstellar absorption lines have been determined as functions of grating, order, wavelength, and aperture. The behavior is in accordance with our detailed analysis of prelaunch laboratory data and validates both the observing procedures and the data reduction developed on the basis of that experience. We present and discuss the use of a background (scattered light) removal algorithm designed to produce net spectra properly corrected for the effects of grating scattered light. The coefficients of the scattered light removal algorithm have been determined, and recommended numerical values are presented.

Cardelli, Jason A.↗
NASA NTRS2010

The Geostationary Lighting Mapper (GLM) for GOES-R: A New Operational Capability to Improve Storm Forecasts and Warnings

The next generation Geostationary Operational Environmental Satellite (GOES-R) series is a follow on to the existing GOES system currently operating over the Western Hemisphere. Superior spacecraft and instrument technology will support expanded detection of environmental phenomena, resulting in more timely and accurate forecasts and warnings. Advancements over current GOES capabilities include a new capability for total lightning detection (cloud and cloud-to-ground flashes) from the Geostationary Lightning Mapper (GLM), and improved spectral (3x), spatial (4x), and temporal (5x) resolution for the Advanced Baseline Imager (ABI). The GLM, an optical transient detector and imager operating in the near-IR at 777.4 nm will map all (in-cloud and cloud-to-ground) lighting flashes continuously day and night with near-uniform spatial resolution of 8 km with a product refresh rate of less than 20 sec over the Americas and adjacent oceanic regions, from the west coast of Africa (GOES-E) to New Zealand (GOES-W) when the constellation is fully operational. This will aid in forecasting severe storms and tornado activity, and convective weather impacts on aviation safety and efficiency. In parallel with the instrument development (a prototype and 4 flight models), a GOES-R Risk Reduction Team and Algorithm Working Group Lightning Applications Team have begun to develop the Level 2 algorithms and applications. Proxy total lightning data from the NASA Lightning Imaging Sensor on the Tropical Rainfall Measuring Mission (TRMM) satellite and regional test beds are being used to develop the pre-launch algorithms and applications, and also improve our knowledge of thunderstorm initiation and evolution. Real time lightning mapping data are being provided in an experimental mode to selected National Weather Service (NWS) national centers and forecast offices via the GOES-R Proving Ground to help improve our understanding of the application of these data in operational settings and facilitate Day-1 user readiness for this new capability.

Goodman, Steven J.↗
DOE OSTI2025

FedOSAA: Improving Federated Learning with One-Step Anderson Acceleration

Federated learning (FL) is a distributed machine learning approach that enables multiple local clients and a central server to collaboratively train a model while keeping the data on their own devices. First-order methods, particularly those incorporating variance reduction techniques, are the most widely used FL algorithms due to their simple implementation and stable performance. However, these methods tend to be slow and require a large number of communication rounds to reach the global minimizer. We propose FedOSAA, a novel approach that preserves the simplicity of first-order methods while achieving the rapid convergence typically associated with second-order methods. Our approach applies one Anderson acceleration (AA) step following classical local updates based on first-order methods with variance reduction, such as FedSVRG and SCAFFOLD, during local training. This AA step is able to leverage curvature information from the history points and gives a new update that approximates the Newton-GMRES direction, thereby significantly improving the convergence. We establish a local linear convergence rate to the global minimizer of FedOSAA for smooth and strongly convex loss functions. Numerical comparisons show that FedOSAA substantially improves the communication and computation efficiency of the original first-order methods, achieving performance comparable to second-order methods like GIANT.

Feng, Xue [University of California, Davis]↗
NASA NTRS2001

The Role of Clear Sky Identification in the Study of Cloud Radiative Effects: Combine Analysis from ISCCP and the Scanner of Radiation Budget (ScaRaB)

Since the effect of clouds on the earth's radiation balance is often estimated as the difference of net radiative fluxes at the top of the atmosphere between all situations and monthly averaged clear sky situations of the same regions, a reliable identification of clear sky is important for the study of cloud radiative effects. The Scanner for Radiation Balance (ScaRaB) radiometer on board the Russian Meteor-3/7 satellite provided earth radiation budget observations from March 1994 to February 1995 with two ERBE-Re broad-band longwave and shortwave channels. Two narrow-band channels, in the infrared atmospheric window and in the visible band, have been added to the ScaRaB instrument to improve the cloud scene identification. The International Satellite Cloud Climatology Project (ISCCP) method for cloud detection and determination of cloud and surface properties uses the same narrow-band channels as ScaRaB, but is employed to a collection of measurements at a better spatial resolution of about 5 km. By applying the original ISCCP algorithms to the ScaRaB data, the clear sky frequency is about 5% lower than the one over quasi-simultaneous original ISCCP data, an indication that the ISCCP cloud detection is quite stable. However, one would expect an about 10 to 20% smaller clear sky occurrence over the larger ScaRaB pixels. Adapting the ISCCP algorithms to the reduced spatial resolution of 60 km and to the different time sampling of the ScaRaB data leads therefore to a reduction of a residual cloud contamination. A sensitivity study with time-space collocated ScaRaB and original ISCCP data at a spatial resolution of 1deg longitude x 1deg latitude shows that the effect of clear sky identification method plays a higher role on the clear sky frequency and therefore on the statistics than on the zonal mean values of the clear sky fluxes. Nevertheless, the zonal outgoing longwave fluxes corresponding to ERBE clear sky are in general about 2 to 10 W/sq m higher than those obtained from the ScaRaB adapted ISCCP clear sky identifications. The latter are close to (about 1 W/sq m higher) fluxes corresponding to clear sky regions from original ISCCP data, whereas ScaRaB clear sky LW fluxes obtained with the original ISCCP identification lie about 1 to 2 W/sq m below. Especially in the tropics where water vapor abundance is high, the ERBE clear sky LW fluxes seem to be systematically overestimated by about 4 W/sq m, and SW fluxes are lower by about 5 to 10 W/sq m. However, the uncertainty in the analysis of monthly mean zonal cloud radiative effects is also produced by the low frequency of clear sky occurrence, illustrated when averaging over pixels or even over regions of 4deg longitude x 5deg latitude, corresponding to the spatial resolution of General Circulation Models. The systematic bias in the clear sky fluxes is not reflected in the zonal cloud radiative effects, because the clear sky regions selected by the different algorithms can occur in different geographic regions with different cloud properties.

Rossow, W. B.↗
NASA NTRS1985

Theoretical and software considerations for general dynamic analysis using multilevel substructured models

The dynamic analysis of complex structural systems using the finite element method and multilevel substructured models is presented. The fixed-interface method is selected for substructure reduction because of its efficiency, accuracy, and adaptability to restart and reanalysis. This method is extended to reduction of substructures which are themselves composed of reduced substructures. The implementation and performance of the method in a general purpose software system is emphasized. Solution algorithms consistent with the chosen data structures are presented. It is demonstrated that successful finite element software requires the use of software executives to supplement the algorithmic language. The complexity of the implementation of restart and reanalysis porcedures illustrates the need for executive systems to support the noncomputational aspects of the software. It is shown that significant computational efficiencies can be achieved through proper use of substructuring and reduction technbiques without sacrificing solution accuracy. The restart and reanalysis capabilities and the flexible procedures for multilevel substructured modeling gives economical yet accurate analyses of complex structural systems.

Schmidt, R. J.↗
NASA NTRS2016

NASADEM Global Elevation Model: Methods and Progress

NASADEM (NASA Digital Elevation Model) is a near-global elevation model that is being produced primarily by completely reprocessing the Shuttle Radar Topography Mission (SRTM) radar data and then merging it with refined ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) GDEM (Global Digital Elevation Model) elevations. The new and improved SRTM elevations in NASADEM result from better vertical control of each SRTM data swath via reference to ICESat (Ice, Cloud, and land Elevation Satellite) elevations and from SRTM void reductions using advanced interferometric unwrapping algorithms. Remnant voids will be filled primarily by GDEM3, but with reduction of GDEM glitches (mostly related to clouds) and therefore with only minor need for secondary sources of fill.

GDEM↗
NASA NTRS1993

An analytic modeling and system identification study of rotor/fuselage dynamics at hover

A combination of analytic modeling and system identification methods have been used to develop an improved dynamic model describing the response of articulated rotor helicopters to control inputs. A high-order linearized model of coupled rotor/body dynamics including flap and lag degrees of freedom and inflow dynamics with literal coefficients is compared to flight test data from single rotor helicopters in the near hover trim condition. The identification problem was formulated using the maximum likelihood function in the time domain. The dynamic model with literal coefficients was used to generate the model states, and the model was parametrized in terms of physical constants of the aircraft rather than the stability derivatives resulting in a significant reduction in the number of quantities to be identified. The likelihood function was optimized using the genetic algorithm approach. This method proved highly effective in producing an estimated model from flight test data which included coupled fuselage/rotor dynamics. Using this approach it has been shown that blade flexibility is a significant contributing factor to the discrepancies between theory and experiment shown in previous studies. Addition of flexible modes, properly incorporating the constraint due to the lag dampers, results in excellent agreement between flight test and theory, especially in the high frequency range.

Hong, Steven W.↗
NASA NTRS1993

An analytic modeling and system identification study of rotor/fuselage dynamics at hover

A combination of analytic modeling and system identification methods have been used to develop an improved dynamic model describing the response of articulated rotor helicopters to control inputs. A high-order linearized model of coupled rotor/body dynamics including flap and lag degrees of freedom and inflow dynamics with literal coefficients is compared to flight test data from single rotor helicopters in the near hover trim condition. The identification problem was formulated using the maximum likelihood function in the time domain. The dynamic model with literal coefficients was used to generate the model states, and the model was parametrized in terms of physical constants of the aircraft rather than the stability derivatives, resulting in a significant reduction in the number of quantities to be identified. The likelihood function was optimized using the genetic algorithm approach. This method proved highly effective in producing an estimated model from flight test data which included coupled fuselage/rotor dynamics. Using this approach it has been shown that blade flexibility is a significant contributing factor to the discrepancies between theory and experiment shown in previous studies. Addition of flexible modes, properly incorporating the constraint due to the lag dampers, results in excellent agreement between flight test and theory, especially in the high frequency range.

Hong, Steven W.↗
NASA NTRS1994

Implementation of a fully-balanced periodic tridiagonal solver on a parallel distributed memory architecture

While parallel computers offer significant computational performance, it is generally necessary to evaluate several programming strategies. Two programming strategies for a fairly common problem - a periodic tridiagonal solver - are developed and evaluated. Simple model calculations as well as timing results are presented to evaluate the various strategies. The particular tridiagonal solver evaluated is used in many computational fluid dynamic simulation codes. The feature that makes this algorithm unique is that these simulation codes usually require simultaneous solutions for multiple right-hand-sides (RHS) of the system of equations. Each RHS solutions is independent and thus can be computed in parallel. Thus a Gaussian elimination type algorithm can be used in a parallel computation and the more complicated approaches such as cyclic reduction are not required. The two strategies are a transpose strategy and a distributed solver strategy. For the transpose strategy, the data is moved so that a subset of all the RHS problems is solved on each of the several processors. This usually requires significant data movement between processor memories across a network. The second strategy attempts to have the algorithm allow the data across processor boundaries in a chained manner. This usually requires significantly less data movement. An approach to accomplish this second strategy in a near-perfect load-balanced manner is developed. In addition, an algorithm will be shown to directly transform a sequential Gaussian elimination type algorithm into the parallel chained, load-balanced algorithm.

Eidson, T. M.↗
NASA NTRS1995

Knowledge-Based Scheduling of Arrival Aircraft in the Terminal Area

A knowledge based method for scheduling arrival aircraft in the terminal area has been implemented and tested in real time simulation. The scheduling system automatically sequences, assigns landing times, and assign runways to arrival aircraft by utilizing continuous updates of aircraft radar data and controller inputs. The scheduling algorithm is driven by a knowledge base which was obtained in over two thousand hours of controller-in-the-loop real time simulation. The knowledge base contains a series of hierarchical 'rules' and decision logic that examines both performance criteria, such as delay reductions, as well as workload reduction criteria, such as conflict avoidance. The objective of the algorithm is to devise an efficient plan to land the aircraft in a manner acceptable to the air traffic controllers. This paper describes the scheduling algorithms, gives examples of their use, and presents data regarding their potential benefits to the air traffic system.

Krzeczowski, K. J.↗
NASA NTRS2016

New Methods for Retrieval of Chlorophyll Red Fluorescence from Hyperspectral Satellite Instruments: Simulations and Application to GOME-2 and SCIAMACHY

Global satellite measurements of solar-induced fluorescence (SIF) from chlorophyll over land and ocean have proven useful for a number of different applications related to physiology, phenology, and productivity of plants and phytoplankton. Terrestrial chlorophyll fluorescence is emitted throughout the red and far-red spectrum, producing two broad peaks near 683 and 736nm. From ocean surfaces, phytoplankton fluorescence emissions are entirely from the red region (683nm peak). Studies using satellite-derived SIF over land have focused almost exclusively on measurements in the far red (wavelengths greater than 712nm), since those are the most easily obtained with existing instrumentation. Here, we examine new ways to use existing hyperspectral satellite data sets to retrieve red SIF (wavelengths less than 712nm) over both land and ocean. Red SIF is thought to provide complementary information to that from the far red for terrestrial vegetation. The satellite instruments that we use were designed to make atmospheric trace-gas measurements and are therefore not optimal for observing SIF; they have coarse spatial resolution and only moderate spectral resolution (0.5nm). Nevertheless, these instruments, the Global Ozone Monitoring Instrument 2 (GOME-2) and the SCanning Imaging Absorption spectroMeter for Atmospheric CHartographY (SCIAMACHY), offer a unique opportunity to compare red and far-red terrestrial SIF at regional spatial scales. Terrestrial SIF has been estimated with ground-, aircraft-, or satellite-based instruments by measuring the filling-in of atmospheric andor solar absorption spectral features by SIF. Our approach makes use of the oxygen (O2) gamma band that is not affected by SIF. The SIF-free O2 gamma band helps to estimate absorption within the spectrally variable O2 B band, which is filled in by red SIF. SIF also fills in the spectrally stable solar Fraunhofer lines (SFLs) at wavelengths both inside and just outside the O2 B band, which further helps to estimate red SIF emission. Our approach is then an extension of previous approaches applied to satellite data that utilized only the filling-in of SFLs by red SIF. We conducted retrievals of red SIF using an extensive database of simulated radiances covering a wide range of conditions. Our new algorithm produces good agreement between the simulated truth and retrievals and shows the potential of the O2 bands for noise reduction in red SIF retrievals as compared with approaches that rely solely on SFL filling. Biases seen with existing satellite data, most likely due to instrumental artifacts that vary in time, space, and with instrument, must be addressed in order to obtain reasonable results. Our 8-year record of red SIF observations over land with the GOME-2 allows for the first time reliable global mapping of monthly anomalies. These anomalies are shown to have similar spatiotemporal structure as those in the far red, particularly for drought-prone regions. There is a somewhat larger percentage response in the red as compared with the far red for these areas that are drought sensitive. We also demonstrate that good-quality ocean fluorescence line height retrievals can be achieved with GOME-2, SCIAMACHY, and similar instruments by utilizing the full complement of radiance measurements that span the red SIF emission feature.

spatiotemporal structure↗
NASA NTRS1995

Knowledge-based scheduling of arrival aircraft

A knowledge-based method for scheduling arrival aircraft in the terminal area has been implemented and tested in real-time simulation. The scheduling system automatically sequences, assigns landing times, and assigns runways to arrival aircraft by utilizing continuous updates of aircraft radar data and controller inputs. The scheduling algorithms is driven by a knowledge base which was obtained in over two thousand hours of controller-in-the-loop real-time simulation. The knowledge base contains a series of hierarchical 'rules' and decision logic that examines both performance criteria, such as delay reduction, as well as workload reduction criteria, such as conflict avoidance. The objective of the algorithms is to devise an efficient plan to land the aircraft in a manner acceptable to the air traffic controllers. This paper will describe the scheduling algorithms, give examples of their use, and present data regarding their potential benefits to the air traffic system.

Krzeczowski, K.↗
NASA NTRS1976

Classification improvement by optimal dimensionality reduction when training sets are of small size

A computer simulation was performed to test the conjecture that, when the sizes of the training sets are small, classification in a subspace of the original data space may give rise to a smaller probability of error than the classification in the data space itself; this is because the gain in the accuracy of estimation of the likelihood functions used in classification in the lower dimensional space (subspace) offsets the loss of information associated with dimensionality reduction (feature extraction). A number of pseudo-random training and data vectors were generated from two four-dimensional Gaussian classes. A special algorithm was used to create an optimal one-dimensional feature space on which to project the data. When the sizes of the training sets are small, classification of the data in the optimal one-dimensional space is found to yield lower error rates than the one in the original four-dimensional space.

Starks, S. A.↗
NASA NTRS1995

Marine optical characterizations

During the past three months, the MOCE Team conducted two field experiments in Mill Creek,Chesapeake Bay, from July 24 to August 4, and at the MOBY operations site at Snug Harbor, Honolulu, Hawaii, from August 15-30, prepared two technical memoranda, and continued MOCE-2 and MOCE-3 data reduction. The primary purposes of the experiments were to test the SeaWiFS 'remote sensing reflectance' protocol, obtain turbid water data for ocean color satellite algorithm development, perform calibration for both Near Infrared (NIR) and Visible Rainbow Spectrometer system, continue assembling the operational Marine Optical Buoy, and to test the MOBY cellular phone communications link at the Lanai mooring site.

Clark, Dennis K.↗
NASA NTRSDate not supplied

Fostering Better Collaboration in Software Development Cycles Between Scientists and Programmers to Ensure the Integrity of and Promote the Development of New Scientific Data Products.

Misaligned incentives lead to reduced interaction between scientists and programmers on modern NASA science data-product development teams. Typically, situations arise where the scientist is not incentivized to learn modern coding practices and the programmer does not understand the science algorithms in the code. A programmer is responsible for the deliverable code thus setting a tradeoff between the desire for code improvement versus fear of compromising the integrity of data-product while the scientist continues to rely on their legacy codebases owing to the complexity of using the delivered code outside the processing environment and lack of validation modules. The NASA/CERES-TISA project has adopted a collaborative approach, with scientists and programmers both utilizing the same software repository with multiple branches, some optimized for delivery to a processing datacenter and others for scientific product development and validation. A team of scientists and programmers jointly review any new science code updates for integration into the codebase and strive to improve practices through promoting algorithm understanding, better institutional knowledge exchange and documentation, modularization, and developing data processing flow-dictated validation and debugging methods. This leads to a reduction in the personnel single point failures and reduced development time for creation of new science data-products.

CERES↗
DOE OSTI2025

In-pixel integration of signal processing and AI/ML based data filtering for particle tracking detectors

We present the first physical realization of in-pixel signal processing with integrated AI-based data filtering for particle tracking detectors. Building on prior work that demonstrated a physics-motivated edge-AI algorithm suitable for ASIC implementation, this work marks a significant milestone toward intelligent silicon trackers. Our prototype readout chip performs real-time data reduction at the sensor level while meeting stringent requirements on power, area, and latency. The chip is taped-out in 28nm TSMC CMOS bulk process, which has been shown to have sufficient radiation hardness for particle experiments. This development represents a key step toward enabling fully on-detector edge AI, with broad implications for data throughput and discovery potential in high-rate, high-radiation environments such as the High-Luminosity LHC.

Parpillon, Benjamin [Fermilab; Illinois U., Chicag↗
NASA NTRS2014

Identification of Flights for Cost-Efficient Climate Impact Reduction

The aircraft-induced climate impact has drawn attention in recent years. Aviation operations affect the environment mainly through the release of carbon-dioxide, nitrogen-oxides, and by the formation of contrails. Recent research has shown that altering trajectories can reduce aviation environmental cost by reducing Absolute Global Temperature Change Potential, a climate assessment metric that adapts a linear system for modeling the global temperature response to aviation emissions and contrails. However, these methods will increase fuel consumption that leads to higher fuel costs imposed on airlines. The goal of this work is to identify ights for which the environmental cost of climate impact reduction outweighs the increase in operational cost on an individual aircraft basis. Environmental cost is quanti ed using the monetary social cost of carbon. The increase in operational cost is considering cost of additional fuel usage only. For this paper, an algorithm has been developed that modi es the trajectories of ights to evaluate the e ect of environ- mental cost and operational cost of ights in the United States National Airspace System. The algorithm identi es ights for which the environmental cost of climate impact can be reduced and modi es their trajectories to achieve maximum environmental net bene t, which is the di erence between reduction in environmental cost and additional operational cost. The result shows on a selected day, 16% of the ights among eight major airlines, or 2,043 ights, can achieve environmental net bene t using weather forecast data, resulting in net bene t of around $500,000. The result also suggests that the long-haul ights would be better candidates for cost-ecient climate impact reduction than the short haul ights. The algorithm will help to identify the characteristics of ights that are capable of applying cost-ecient climate impact reduction strategy.

climate impact reduction↗
NASA NTRS2015

Optimization of Selected Remote Sensing Algorithms for Embedded NVIDIA Kepler GPU Architecture

This paper evaluates the potential of embedded Graphic Processing Units (GPUs) in Nvidia's Tegra K1 (based on Kepler (TM) hardware) for onboard processing. The performance is compared to a general purpose multi-core CPU (Central Processing Unit) and a fully-fledged GPU accelerator. This study uses two algorithms: Wavelet Spectral Dimension Reduction of Hyperspectral Imagery and Automated Cloud-Cover Assessment (ACCA) Algorithm. Tegra K1 achieved 51 for the ACCA algorithm and 20 for the dimension reduction algorithm, as compared to the performance of the high-end 8-core server Intel Xeon CPU with 13.5 times higher power consumption.

data reduction↗