SEARCH · Search NASA

Results for “Data Remapping”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS1997

Efficient Load Balancing and Data Remapping for Adaptive Grid Calculations

Mesh adaption is a powerful tool for efficient unstructured- grid computations but causes load imbalance among processors on a parallel machine. We present a novel method to dynamically balance the processor workloads with a global view. This paper presents, for the first time, the implementation and integration of all major components within our dynamic load balancing strategy for adaptive grid calculations. Mesh adaption, repartitioning, processor assignment, and remapping are critical components of the framework that must be accomplished rapidly and efficiently so as not to cause a significant overhead to the numerical simulation. Previous results indicated that mesh repartitioning and data remapping are potential bottlenecks for performing large-scale scientific calculations. We resolve these issues and demonstrate that our framework remains viable on a large number of processors.

Oliker, Leonid↗

NASA NTRS1990

Optimal dynamic remapping of data parallel computations

A large class of data parallel computations is characterized by a sequence of phases, with phase changes occurring unpredictably. Dynamic remapping of the workload to processors may be required to maintain good performance. The problem considered, for which the utility of remapping and the future behavior of the workload are uncertain, arises when phases exhibit stable execution requirements during a given phase, but requirements change radically between phases. For these situations, a workload assignment generated for one phase may hinder performance during the next phase. This problem is treated formally for a probabilistic model of computation with at most two phases. The authors address the fundamental problem of balancing the expected remapping performance gain against the delay cost, and they derive the optimal remapping decision policy. The promise of the approach is shown by application to multiprocessor implementations of an adaptive gridding fluid dynamics program and to a battlefield simulation program.

Nicol, David M.↗

NASA NTRS1982

Computational aspects of remapping digital imagery

One of the advantages of automated cartography is that map data stored in the digital computer can be plotted or displayed at any scale or projection by recomputing the coordinates of the data. This is especially easy in the case of vector (graphics) data but in the case of digital image (raster) data, remapping is a more difficult operation. Examples of the remapping of digital imagery would include rectification of a LANDSAT MSS to an orthographic or Mercator projection, warping of one image to register with another, or rotation, scale, or aspect changes of a digital image. Use of general purpose computers and array processors for this task will be covered. Data processing error will be discussed for each modelling/warping approach.

Zobrist, A. L.↗

NASA NTRS1998

PLUM: Parallel Load Balancing for Unstructured Adaptive Meshes

Dynamic mesh adaption on unstructured grids is a powerful tool for computing large-scale problems that require grid modifications to efficiently resolve solution features. Unfortunately, an efficient parallel implementation is difficult to achieve, primarily due to the load imbalance created by the dynamically-changing nonuniform grid. To address this problem, we have developed PLUM, an automatic portable framework for performing adaptive large-scale numerical computations in a message-passing environment. First, we present an efficient parallel implementation of a tetrahedral mesh adaption scheme. Extremely promising parallel performance is achieved for various refinement and coarsening strategies on a realistic-sized domain. Next we describe PLUM, a novel method for dynamically balancing the processor workloads in adaptive grid computations. This research includes interfacing the parallel mesh adaption procedure based on actual flow solutions to a data remapping module, and incorporating an efficient parallel mesh repartitioner. A significant runtime improvement is achieved by observing that data movement for a refinement step should be performed after the edge-marking phase but before the actual subdivision. We also present optimal and heuristic remapping cost metrics that can accurately predict the total overhead for data redistribution. Several experiments are performed to verify the effectiveness of PLUM on sequences of dynamically adapted unstructured grids. Portability is demonstrated by presenting results on the two vastly different architectures of the SP2 and the Origin2OOO. Additionally, we evaluate the performance of five state-of-the-art partitioning algorithms that can be used within PLUM. It is shown that for certain classes of unsteady adaption, globally repartitioning the computational mesh produces higher quality results than diffusive repartitioning schemes. We also demonstrate that a coarse starting mesh produces high quality load balancing, at a fraction of the cost required a fine initial mesh. Results indicate that our parallel load balancing strategy will remain viable on large numbers of processors.

Oliker, Leonid↗

NASA NTRS1997

Load Balancing Sequences of Unstructured Adaptive Grids

Mesh adaption is a powerful tool for efficient unstructured grid computations but causes load imbalance on multiprocessor systems. To address this problem, we have developed PLUM, an automatic portable framework for performing adaptive large-scale numerical computations in a message-passing environment. This paper makes several important additions to our previous work. First, a new remapping cost model is presented and empirically validated on an SP2. Next, our load balancing strategy is applied to sequences of dynamically adapted unstructured grids. Results indicate that our framework is effective on many processors for both steady and unsteady problems with several levels of adaption. Additionally, we demonstrate that a coarse starting mesh produces high quality load balancing, at a fraction of the cost required for a fine initial mesh. Finally, we show that the data remapping overhead can be significantly reduced by applying our heuristic processor reassignment algorithm.

Biswas, Rupak↗

NASA NTRS2012

Globally Gridded Satellite (GridSat) Observations for Climate Studies

Geostationary satellites have provided routine, high temporal resolution Earth observations since the 1970s. Despite the long period of record, use of these data in climate studies has been limited for numerous reasons, among them: there is no central archive of geostationary data for all international satellites, full temporal and spatial resolution data are voluminous, and diverse calibration and navigation formats encumber the uniform processing needed for multi-satellite climate studies. The International Satellite Cloud Climatology Project set the stage for overcoming these issues by archiving a subset of the full resolution geostationary data at approx.10 km resolution at 3 hourly intervals since 1983. Recent efforts at NOAA s National Climatic Data Center to provide convenient access to these data include remapping the data to a standard map projection, recalibrating the data to optimize temporal homogeneity, extending the record of observations back to 1980, and reformatting the data for broad public distribution. The Gridded Satellite (GridSat) dataset includes observations from the visible, infrared window, and infrared water vapor channels. Data are stored in the netCDF format using standards that permit a wide variety of tools and libraries to quickly and easily process the data. A novel data layering approach, together with appropriate satellite and file metadata, allows users to access GridSat data at varying levels of complexity based on their needs. The result is a climate data record already in use by the meteorological community. Examples include reanalysis of tropical cyclones, studies of global precipitation, and detection and tracking of the intertropical convergence zone.

Knapp, Kenneth R.↗

NASA NTRS1983

An interactive system for compositing digital radar and satellite data

This paper describes an approach for compositing digital radar data and GOES satellite data for meteorological analysis. The processing is performed on a user-oriented image processing system, and is designed to be used in the research mode. It has a capability to construct PPIs and three-dimensional CAPPIs using conventional as well as Doppler data, and to composite other types of data. In the remapping of radar data to satellite coordinates, two steps are necessary. First, PPI or CAPPI images are remapped onto a latitude-longitude projection. Then, the radar data are projected into satellite coordinates. The exact spherical trigonometric equations, and the approximations derived for simplifying the computations are given. The use of these approximations appears justified for most meteorological applications. The largest errors in the remapping procedure result from the satellite viewing angle parallax, which varies according to the cloud top height. The horizontal positional error due to this is of the order of the error in the assumed cloud height in mid-latitudes. Examples of PPI and CAPPI data composited with satellite data are given for Hurricane Frederic on 13 September 1979 and for a squall line on 2 May 1979 in Oklahoma.

Heymsfield, G. M.↗

NASA NTRS1995

TOMS total ozone trends in potential vorticity coordinates

Global total ozone measurements from the Nimbus 7 Total Ozone Mapping Spectrometer (TOMS) are analyzed using potential vorticity (PV) as an approximate vortex-following coordinate. We analyze the time period November 1978-May 1991, prior to the volcanic eruption of Mt. Pinatubo. The TOMS data are remapped into PV coordinates and trends are calculated, thereby characterizing ozone losses inside and outside the winter polar vortices. These analyses show large regions of ozone loss outside of the vortex in both hemispheres. Furthermore, these data suggest that midlatitude losses in the NH during winter-spring do not result solely from the transport of ozone depleted air from inside to outside the vortex.

Randel, William J.↗

NASA NTRS1995

Using the application visualization system to view HALOE three-dimensional satellite data

The Application Visualization System (AVS) is used to view a three-dimensional data field containing the volume mixing ratios of a chemical species in the middle atmosphere obtained by the Halogen Occultation Experiment (HALOE) aboard the Upper Atmosphere Research Satellite (UARS). Since launch in September 1991, HALOE has been collecting data on approximately 30 sunrise/sunset events in two narrow latitude bands each day. The vertical volume mixing ratio profiles are retrieved for eight species for each event. The accumulated data for approximately 30 days cover most of the globe (limited by sunlit latitudes), and this monthly data block can be described as the volume mixing ratio of a specific species in the atmosphere as a function of latitude, longitude, and height. The data were remapped using linear interpolation for pressure levels and Gaussian weighted binning from sampling locations to a three-dimensional grid. An AVS network is constructed that allows for viewing the three-dimensional field with rendered slices at constant latitudes, longitudes or pressure levels. Discussions are given on the advantages and some disadvantages learned about from experiences applying AVS to visualize HALOE three dimensional data.

Luo, Mingzhao↗

NASA NTRS1994

Scalability study of parallel spatial direct numerical simulation code on IBM SP1 parallel supercomputer

The implementation and the performance of a parallel spatial direct numerical simulation (PSDNS) code are reported for the IBM SP1 supercomputer. The spatially evolving disturbances that are associated with laminar-to-turbulent in three-dimensional boundary-layer flows are computed with the PS-DNS code. By remapping the distributed data structure during the course of the calculation, optimized serial library routines can be utilized that substantially increase the computational performance. Although the remapping incurs a high communication penalty, the parallel efficiency of the code remains above 40% for all performed calculations. By using appropriate compile options and optimized library routines, the serial code achieves 52-56 Mflops on a single node of the SP1 (45% of theoretical peak performance). The actual performance of the PSDNS code on the SP1 is evaluated with a 'real world' simulation that consists of 1.7 million grid points. One time step of this simulation is calculated on eight nodes of the SP1 in the same time as required by a Cray Y/MP for the same simulation. The scalability information provides estimated computational costs that match the actual costs relative to changes in the number of grid points.

Hanebutte, Ulf R.↗

NASA NTRS1995

Scalability of Parallel Spatial Direct Numerical Simulations on Intel Hypercube and IBM SP1 and SP2

The implementation and performance of a parallel spatial direct numerical simulation (PSDNS) approach on the Intel iPSC/860 hypercube and IBM SP1 and SP2 parallel computers is documented. Spatially evolving disturbances associated with the laminar-to-turbulent transition in boundary-layer flows are computed with the PSDNS code. The feasibility of using the PSDNS to perform transition studies on these computers is examined. The results indicate that PSDNS approach can effectively be parallelized on a distributed-memory parallel machine by remapping the distributed data structure during the course of the calculation. Scalability information is provided to estimate computational costs to match the actual costs relative to changes in the number of grid points. By increasing the number of processors, slower than linear speedups are achieved with optimized (machine-dependent library) routines. This slower than linear speedup results because the computational cost is dominated by FFT routine, which yields less than ideal speedups. By using appropriate compile options and optimized library routines on the SP1, the serial code achieves 52-56 M ops on a single node of the SP1 (45 percent of theoretical peak performance). The actual performance of the PSDNS code on the SP1 is evaluated with a "real world" simulation that consists of 1.7 million grid points. One time step of this simulation is calculated on eight nodes of the SP1 in the same time as required by a Cray Y/MP supercomputer. For the same simulation, 32-nodes of the SP1 and SP2 are required to reach the performance of a Cray C-90. A 32 node SP1 (SP2) configuration is 2.9 (4.6) times faster than a Cray Y/MP for this simulation, while the hypercube is roughly 2 times slower than the Y/MP for this application. KEY WORDS: Spatial direct numerical simulations; incompressible viscous flows; spectral methods; finite differences; parallel computing.

Joslin, Ronald D.↗

NASA NTRS2017

Development of Multi-Sensor Global Cloud and Radiance Composites for Earth Radiation Budget Monitoring from DSCOVR

The Deep Space Climate Observatory (DSCOVR) enables analysis of the daytime Earth radiation budget via the onboard Earth Polychromatic Imaging Camera (EPIC) and National Institute of Standards and Technology Advanced Radiometer (NISTAR). Radiance observations and cloud property retrievals from low earth orbit and geostationary satellite imagers have to be co-located with EPIC pixels to provide scene identification in order to select anisotropic directional models needed to calculate shortwave and longwave fluxes. A new algorithm is proposed for optimal merging of selected radiances and cloud properties derived from multiple satellite imagers to obtain seamless global hourly composites at 5-kilometer resolution. An aggregated rating is employed to incorporate several factors and to select the best observation at the time nearest to the EPIC measurement. Spatial accuracy is improved using inverse mapping with gradient search during reprojection and bicubic interpolation for pixel resampling. The composite data are subsequently remapped into EPIC-view domain by convolving composite pixels with the EPIC point spread function (PSF) defined with a half-pixel accuracy. PSF-weighted average radiances and cloud properties are computed separately for each cloud phase. The algorithm has demonstrated contiguous global coverage for any requested time of day with a temporal lag of under 2 hours in over 95 percent of the globe.

Khlopenkov, Konstantin↗

NASA NTRS1992

The Airborne Ocean Color Imager - System description and image processing

The Airborne Ocean Color Imager was developed as an aircraft instrument to simulate the spectral and radiometric characteristics of the next generation of satellite ocean color instrumentation. Data processing programs have been developed as extensions of the Coastal Zone Color Scanner algorithms for atmospheric correction and bio-optical output products. The latter include several bio-optical algorithms for estimating phytoplankton pigment concentration, as well as one for the diffuse attenuation coefficient of the water. Additional programs have been developed to geolocate these products and remap them into a georeferenced data base, using data from the aircraft's inertial navigation system. Examples illustrate the sequential data products generated by the processing system, using data from flightlines near the mouth of the Mississippi River: from raw data to atmospherically corrected data, to bio-optical data, to geolocated data, and, finally, to georeferenced data.

Wrigley, Robert C.↗

NASA NTRS1981

A general interactive system for compositing digital radar and satellite data

Reynolds and Smith (1979) have considered the combined use of digital weather radar and satellite data in interactive systems for case study analysis and forecasting. Satellites view the top of clouds, whereas radar is capable of observing the detailed internal structure of clouds. The considered approach requires the use of a common coordinate system. In the present investigation, it was decided to use the satellite coordinate system as the base system in order to maintain the fullest resolution of the satellite data. The investigation is concerned with the development of a general interactive software system called RADPAK for remapping and analyzing conventional and Doppler radar data. RADPAK is implemented as a part of a minicomputer-based image processing system, called Atmospheric and Oceanographic Image Processing System. Attention is given to a general description of the RADPAK system, remapping methodology, and an example of satellite remapping.

Ghosh, K. K.↗

NASA NTRS2019

External Geolocation for Swath Data

Swath products (called Level 1 and Level 2 by NASA) are in the original instrument geometry (not gridded). Geolocation information is needed for remapping but also understanding the product. Geolocation data can be as large or larger than some L2 products. Storing this information in each L1/L2 product increases archive volume and for users who work with multiple products, the information would be duplicated. In this talk, MODIS is used as an example –other satellite and aircraft instruments have similar issues.

Geolocation↗

NASA NTRS1986

Dynamic remapping of parallel computations with varying resource demands

A large class of computational problems is characterized by frequent synchronization, and computational requirements which change as a function of time. When such a problem must be solved on a message passing multiprocessor machine, the combination of these characteristics lead to system performance which decreases in time. Performance can be improved with periodic redistribution of computational load; however, redistribution can exact a sometimes large delay cost. We study the issue of deciding when to invoke a global load remapping mechanism. Such a decision policy must effectively weigh the costs of remapping against the performance benefits. We treat this problem by constructing two analytic models which exhibit stochastically decreasing performance. One model is quite tractable; we are able to describe the optimal remapping algorithm, and the optimal decision policy governing when to invoke that algorithm. However, computational complexity prohibits the use of the optimal remapping decision policy. We then study the performance of a general remapping policy on both analytic models. This policy attempts to minimize a statistic W(n) which measures the system degradation (including the cost of remapping) per computation step over a period of n steps. We show that as a function of time, the expected value of W(n) has at most one minimum, and that when this minimum exists it defines the optimal fixed-interval remapping policy. Our decision policy appeals to this result by remapping when it estimates that W(n) is minimized. Our performance data suggests that this policy effectively finds the natural frequency of remapping. We also use the analytic models to express the relationship between performance and remapping cost, number of processors, and the computation's stochastic activity.

Nicol, D. M.↗

NASA NTRS1998

PLUM: Parallel Load Balancing for Adaptive Unstructured Meshes

Mesh adaption is a powerful tool for efficient unstructured-grid computations but causes load imbalance among processors on a parallel machine. We present a novel method called PLUM to dynamically balance the processor workloads with a global view. This paper presents the implementation and integration of all major components within our dynamic load balancing strategy for adaptive grid calculations. Mesh adaption, repartitioning, processor assignment, and remapping are critical components of the framework that must be accomplished rapidly and efficiently so as not to cause a significant overhead to the numerical simulation. A data redistribution model is also presented that predicts the remapping cost on the SP2. This model is required to determine whether the gain from a balanced workload distribution offsets the cost of data movement. Results presented in this paper demonstrate that PLUM is an effective dynamic load balancing strategy which remains viable on a large number of processors.

Oliker, Leonid↗

NASA NTRS1998

Parallel Load Balancing for Adaptive Unstructured Meshes

Mesh adaption is a powerful tool for efficient unstructured-grid computations but causes load imbalance among processors on a parallel machine. We describe a novel method to dynamically balance the processor workloads with a global view. Mesh question, repartitioning, processor assignment, and remapping are critical components of the framework that must be accomplished rapidly and efficiently so as not to cause a significant overhead to the numerical simulation. A data redistribution model will also be presented that predicts the remapping cost. This model is required to determine whether the gain from a balanced workload distribution offsets the cost of data movement. Results presented will demonstrate that this is an effective dynamic load balancing strategy which remains viable on a large number of processors.

Biswas, Rupak↗