Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data compression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Detecting Anomalies from End-to-End Internet Performance Measurements (PingER) Using Cluster Based Local Outlier Factor

PingER (Ping End-to-End Reporting) is a worldwide end-to-end Internet performance measurement framework. It was developed by the SLAC National Accelerator Laboratory, Stanford, USA and running from the last 20 years. It has more than 700 monitoring agents and remote sites which monitor the performance of Internet links around 170 countries of the world. At present, the size of the compressed PingER data set is about 60 GB comprising of 100,000 flat files. The data is publicly available for valuable Internet performance analyses. However, the data sets suffer from missing values and anomalies due to congestion, bottleneck links, queuing overflow, network software misconfiguration, hardware failure, cable cuts, and social upheavals. Therefore, the objective of this paper is to detect such performance drops or spikes labeled as anomalies or outliers for the PingER data set. In the proposed approach, the raw text files of the data set are transformed into a PingER dimensional model. The missing values are imputed using the k-NN algorithm. The data is partitioned into similar instances using the k-means clustering algorithm. Afterward, clustering is integrated with the Local Outlier Factor (LOF) using the Cluster Based Local Outlier Factor (CBLOF) algorithm to detect the anomalies or outliers from the PingER data. Lastly, anomalies are further analyzed to identify the time frame and location of the hosts generating the major percentage of the anomalies in the PingER data set ranging from 1998 to 2016.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization

There presently exist two dominant strategies to reduce data sizes in analysis and visualization: reducing the precision of the data, e.g., through quantization, or reducing its resolution, e.g., by subsampling. Both have advantages and disadvantages and both face fundamental limits at which the reduced information ceases to be useful. The paper explores the additional gains that could be achieved by combining both strategies. In particular, we present a common framework that allows us to study the trade-off in reducing precision and/or resolution in a principled manner. We represent data reduction schemes as progressive streams of bits and study how various bit orderings such as by resolution, by precision, etc., impact the resulting approximation error across a variety of data sets as well as analysis tasks. Additionally, we compute streams that are optimized for different tasks to serve as lower bounds on the achievable error. Scientific data management systems can use the results proposed in this paper as guidance on how to store and stream data to make efficient use of the limited storage and bandwidth in practice.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Unified many-worlds browsing of arbitrary physics-based animations

Manually tuning physics-based animation parameters to explore a simulation outcome space or achieve desired motion outcomes can be notoriously tedious. This problem has motivated many sophisticated and specialized optimization-based methods for fine-grained (keyframe) control, each of which are typically limited to specific animation phenomena, usually complicated, and, unfortunately, not widely used. In this paper, we propose Unified Many-Worlds Browsing (UMWB), a practical method for sample-level control and exploration of physics-based animations. Our approach supports browsing of large simulation ensembles of arbitrary animation phenomena by using a unified volumetric WORLDPACK representation based on spatiotemporally compressed voxel data associated with geometric occupancy and other low-fidelity animation state. Beyond memory reduction, the WORLDPACK representation also enables unified query support for interactive browsing: it provides fast evaluation of approximate spatiotemporal queries, such as occupancy tests that find ensemble samples ("worlds") where material is either IN or NOT IN a user-specified spacetime region. WORLDPACKS also support real-time hardware-accelerated voxel rendering by exploiting the spatially hierarchical and temporal RLE raster data structure. Our UMWB implementation supports interactive browsing (and offline refinement) of ensembles containing thousands of simulation samples, and fast spatiotemporal queries and ranking. We show UMWB results using a wide variety of physics-based animation phenomena---not just JELL-O ® .

Computer Science↗
DOE OSTI2021

H5TurboPFor v0.0.1

H5TurboPFor is a wrapper for the TurboPFor-Integer-Compression for HDF5 library. Basically, it enables the HDF5 library to compress integer data using the TurboPFor.

Dong, Bin↗
DOE OSTI2024

Weak-Form Latent Space Dynamics Identification

This software showcases the enhanced capabilities of the Latent Space Dynamics Identification (LaSDI) algorithm through the application of the weak form, resulting in WLaSDI. WLaSDI first compresses the data, then projects it onto test functions, and subsequently learns the local latent space models. Notably, WLaSDI demonstrates significantly improved robustness to noise. Using weak-form equation learning techniques, WLaSDI achieves local latent space modeling. Compared to the standard sparse identification of nonlinear dynamics (SINDy) used in LaSDI, the variance reduction of the weak form ensures robust and precise latent space recovery, enabling fast, robust, and accurate simulations. We demonstrate the efficacy of WLaSDI against LaSDI using several common benchmark examples, including viscid and inviscid Burgers', radial advection, and heat conduction. For instance, in 1D inviscid Burgers' simulations with up to 100% Gaussian white noise, WLaSDI maintains relative errors consistently below 6%, whereas LaSDI errors can exceed 10,000%. Similarly, in radial advection simulations, WLaSDI keeps relative errors below 16%, compared to potential errors of up to 10,000% with LaSDI. Additionally, WLaSDI achieves significant speedups, such as a 140X speedup in 1D Burgers' simulations compared to the corresponding full order model.

Choi, Youngsoo↗
DOE OSTI2026

NLR Data Processing Pipeline for MADIS [SWR-26-050]

The NLR Data Processing Pipeline for MADIS software package is for downloading, processing, and performing QA/QC on MADIS data. Designed to handle the following steps: 1) Download all MADIS data as compressed netcdf files for a given time period. 2) Unpack netcdf files into timeseries csvs for each coordinate within the given bounding box. 3) Process the csvs to filter according to quality control checks and convert variables to correct units. 4) Write processed csvs to a single nc file.

Benton, Brandon [National Laboratory of the Rockie↗
DOE OSTI2018

Graphite Characterization: Baseline Variability Analysis Report

The Idaho National INL’s Baseline Material Property program provides many measurements of different material properties on a variety of nuclear graphite grades being investigated for irradiation-induced changes in material properties. The baseline material properties provide an unirradiated reference for comparison with measurements of those properties after irradiation. This study describes how uncertainty, or variance, in estimates of important material properties, like compressive strength, may depend on sample size, sample distribution and parameter estimation methods. Compressive strength data from two large collections of specimens from a billet of PCEA graphite (sample size = 230) and a billet of IG-110 graphite (sample size = 48) are used to illustrate these dependencies. The grades of graphite used represent approximate end members in terms of relative variability in strength measurements. Monte Carlo simulation and bootstrapping methods were used to examine variance in Weibull-parameter estimates as a function of sample size. Results were consistent with what previous statistical analyses have shown about the relationship between sample size and width of the confidence interval for maximum likelihood (ML) estimated parameters when samples are repeatedly drawn from a Weibull distribution. As a function of sample size, the precision of the modulus estimate increases faster than that of the characteristic strength. The modulus maximum likelihood estimates (MLEs) are biased for low sample sizes. The American Society for Testing and Materials document ASTM D7846-16 provides equations that describe the dependence of the variance of parameter MLEs on sample size. Using the same methodology, we show that the polynomials provided for variance dependence on sample size are inaccurate for sample sizes greater than ~120. We provide approximations to that relationship that illustrate relative constancy for sample size > ~10, in the logspace relationship of the modulus variance and the log transformed characteristic strength variance. Bootstrapping provided a means of comparing Weibull parameters obtained using ASME’s guidance for specimen collection to those obtained using different weighting in a case where strength is related to position in a billet and to grain orientation and a single distribution is used to characterize the billet. ASME specifies that samples be collected with equal representation in all locations and grain orientations. Parameter values obtained using ASME’s method were different from those obtained using random sampling of the complete data sets, and this was particularly true for the PCEA data. In addition, while the ASME approach yielded more conservative parameters for the PCEA data, the opposite was true for the IG-110 data. The difference observed for the PCEA data set likely reflects the larger number of against-grain specimens collected, which had generally higher strength than the with-grain specimens. This emphasizes the importance of the dependence of strength on location or grain orientation and suggests that it may be prudent to weight specimen collection toward the weaker subgroups if a conservative estimate as desired, as the equal weighting method recommended by ASME would will not always provide the most conservative Weibull parameters. Because spatial correlations commonly exist in certain grades of graphite, future work may involve developing a methodology for parameter inference that relaxes the assumption of independent specimens. ASTM and ASME codes recommend different parameter estimation methods and models of the Weibull distribution in different situations. Descriptions of the effects on parameter estimates for the example billets used in this study provide an example of how the method may affect the parameter estimate, and how—in this case—that compares to differences due to other factors, such as sample size and dependence on grain orientation or sample location.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS1986

Engineering Voyager 2's encounter with Uranus

Changes made by radio control from the ground in the Voyager 2 spacecraft as it approached Uranus are described. Reduced power required that subsystems and heaters had to be switched on and off in carefully synchronized fashion. Low light levels required increased exposure times, so the jiggling of the spacecraft had to be minimized. Coding changes were made and image data were compressed to cope with the reduced bit rate at larger distances. Successful efforts to cope with failures in the primary radio receiver and in the computer instructions for image compression are described, as are changes made on the ground in the spacecraft navigation.

Laeser, Richard P.↗
NASA NTRS1990

Innovative operating modes and techniques for the spaceborne imaging radar-C instrument

The operation of the spaceborne imaging radar-C (SIR-C) is discussed. The SIR-C instrument has been designed to obtain simultaneous multifrequency and simultaneous multipolarization radar images from a low earth orbit. It is a multiparameter imaging radar which will be flown during at least two different seasons. The instrument has been designed to operate in innovative modes such as the squint alignment mode, the extended aperture mode, the scansar mode, and the interferometry mode. The instrument has been designed to demonstrate innovative engineering techniques such as beam nulling for echo tracking, pulse-repetition frquency hopping for Doppler centroid tracking, generating the frequency step chirp for radar parameter flexibility, block floating point quantizing for data rate compression, and elevation beamwidth broadening for increasing the swath illumination.

Huneycutt, Bryan L.↗
NASA NTRS1993

Flux-difference split parabolized Navier-Stokes algorithm for non-equilibrium chemically reacting flows

A flux-difference split explicit finite-difference algorithm is presented for solving the parabolized form of the equations governing three-dimensional nonequilibrium chemically reacting flows. The algorithm is based on an explicit noniterative, upwind space-marching scheme developed by Korte, but differs in that the unsteady Riemann problem, rather than the steady Riemann problem, is solved. The algorithm allows either a second or an approximately third-order accurate upwind treatment of the convection terms by employing the unsteady approximate Riemann solver of Roe. The source terms of the species transport equations are treated in either an explicit or implicit manner, and the species diffusion terms are modeled with either a Fickian or a multicomponent model. A validation of the algorithm is performed by comparing computational results with the 2-D Mach 14, 15 degree compression-corner data of Holden. The three-dimensional capability of the algorithm is demonstrated by computing Mach 2.7 flow over a swept wedge scramjet fuel injector, and three-dimensional reacting flow capability is demonstrated by a computing a shock-jet interaction concept for mixing and combustion enhancement.

White, J. A.↗
NASA NTRS1994

Optimality Of Variable-Length Codes

Report presents analysis of performances of conceptual Rice universal noiseless coders designed to provide efficient compression of data over wide range of source-data entropies. Includes predictive preprocessor that maps source data into sequence of nonnegative integers and variable-length-coding processor, which adapts to varying entropy of source data by selecting whichever one of number of optional codes yields shortest codeword.

Yeh, Pen-Shu↗
NASA NTRS1993

Spaceborne Imaging Radar-C instrument

The Spaceborne Imaging Radar-C is the next radar in the series of spaceborne radar experiments, which began with Seasat and continued with SIR-A and SIR-B. The SIR-C instrument has been designed to obtain simultaneous multifrequency and simultaneous multipolarization radar images from a low earth orbit. It is a multiparameter imaging radar that will be flown during at least two different seasons. The instrument operates in the squint alignment mode, the extended aperture mode, the scansar mode, and the interferometry mode. The instrument uses engineering techniques such as beam nulling for echo tracking, pulse repetition frequency hopping for Doppler centroid tracking, generating the frequency step chirp for radar parameter flexibility, block floating-point quantizing for data rate compression, and elevation beamwidth broadening for increasing the swath illumination.

Huneycutt, Bryan L.↗
NASA NTRS1993

The Brewster angle effect in SAR polarimetry

For the double bounce case, where the radar signal is reflected twice before returning to the radar antenna, some polarization effects may be observed related to the dielectric constant of the two surfaces causing the reflections. The most noticeable effect would be that the returned signal would be preferentially H polarized. In fact, it may be possible to discern the Brewster angle for both surfaces. The locations of the Brewster angle will depend on the dielectric constant and permittivity of each surface. If it is assumed that both reflections are in the same plane of incidence, and that both surfaces are smooth and flat, there is a straightforward relationship between the degree of linear polarization m and both the dielectric constants of the two reflecting surfaces and the angle of incidence of the illuminating wave: m carat = cos 2(arccot (square root of (R(sub v) / R(sub h)))) where R(sub v,h) are the V and H polarized Fresnel reflection coefficients for two surfaces perpendicular to each other. The degree of linear polarization may be calculated from AIRSAR compressed Stokes data and compared with the given equation. The degree of linear polarization may also be calculated using tree models and compared with AIRSAR data. With further work, it may be possible to use the degree of linear polarization to determine surface parameters of certain imaged areas.

Chapman, B.↗
NASA NTRS1997

A Three-Line Stereo Camera Concept for Planetary Exploration

This paper presents a low-weight stereo camera concept for planetary exploration. The camera uses three CCD lines within the image plane of one single objective. Some of the main features of the camera include: focal length-90 mm, FOV-18.5 deg, IFOV-78 (mu)rad, convergence angles-(+/-)10 deg, radiometric dynamics-14 bit, weight-2 kg, and power consumption-12.5 Watts. From an orbit altitude of 250 km the ground pixel size is 20m x 20m and the swath width is 82 km. The CCD line data is buffered in the camera internal mass memory of 1 Gbit. After performing radiometric correction and application-dependent preprocessing the data is compressed and ready for downlink. Due to the aggressive application of advanced technologies in the area of microelectronics and innovative optics, the low mass and power budgets of 2 kg and 12.5 Watts is achieved, while still maintaining high performance. The design of the proposed light-weight camera is also general purpose enough to be applicable to other planetary missions such as the exploration of Mars, Mercury, and the Moon. Moreover, it is an example of excellent international collaboration on advanced technology concepts developed at DLR, Germany, and NASA's Jet Propulsion Laboratory, USA.

Sandau, Rainer↗
NASA NTRS1999

Integration of a Decentralized Linear-Quadratic-Gaussian Control into GSFC's Universal 3-D Autonomous Formation Flying Algorithm

A decentralized control is investigated for applicability to the autonomous formation flying control algorithm developed by GSFC for the New Millenium Program Earth Observer-1 (EO-1) mission. This decentralized framework has the following characteristics: The approach is non-hierarchical, and coordination by a central supervisor is not required; Detected failures degrade the system performance gracefully; Each node in the decentralized network processes only its own measurement data, in parallel with the other nodes; Although the total computational burden over the entire network is greater than it would be for a single, centralized controller, fewer computations are required locally at each node; Requirements for data transmission between nodes are limited to only the dimension of the control vector, at the cost of maintaining a local additional data vector. The data vector compresses all past measurement history from all the nodes into a single vector of the dimension of the state; and The approach is optimal with respect to standard cost functions. The current approach is valid for linear time-invariant systems only. Similar to the GSFC formation flying algorithm, the extension to linear LQG time-varying systems requires that each node propagate its filter covariance forward (navigation) and controller Riccati matrix backward (guidance) at each time step. Extension of the GSFC algorithm to non-linear systems can also be accomplished via linearization about a reference trajectory in the standard fashion, or linearization about the current state estimate as with the extended Kalman filter. To investigate the feasibility of the decentralized integration with the GSFC algorithm, an existing centralized LQG design for a single spacecraft orbit control problem is adapted to the decentralized framework while using the GSFC algorithm's state transition matrices and framework. The existing GSFC design uses both reference trajectories of each spacecraft in formation and by appropriate choice of coordinates and simplified measurement modeling is formulated as a linear time-invariant system. Results for improvements to the GSFC algorithm and a multiple satellite formation will be addressed. The goal of this investigation is to progressively relax the assumptions that result in linear time-invariance, ultimately to the point of linearization of the non-linear dynamics about the current state estimate as in the extended Kalman filter. An assessment will then be made about the feasibility of the decentralized approach to the realistic formation flying application of the EO-1/Landsat 7 formation flying experiment.

Folta, David C.↗