Utilization of information theory in digital data guidance systems Quarterly progress report no. 1, 1 Sep. - 30 Nov. 1965
Information theory in improving digital data guidance system
GUIDANCE SYSTEM↗
SEARCH · Search NASA
Results for “INFORMATION THEORY”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Application of information theory to the design of line-scan imaging systems
Information theory is used to formulate a single figure of merit for assessing the performance of line scan imaging systems as a function of their spatial response (point spread function or modulation transfer function), sensitivity, sampling and quantization intervals, and the statistical properties of a random radiance field. Computational results for the information density and efficiency (i.e., the ratio of information density to data density) are intuitively satisfying and compare well with experimental and theoretical results obtained by earlier investigators concerned with the performance of TV systems.
Huck, F. O.↗
Information theory analysis of sensor-array imaging systems for computer vision
Information theory is used to assess the performance of sensor-array imaging systems, with emphasis on the performance obtained with image-plane signal processing. By electronically controlling the spatial response of the imaging system, as suggested by the mechanism of human vision, it is possible to trade-off edge enhancement for sensitivity, increase dynamic range, and reduce data transmission. Computational results show that: signal information density varies little with large variations in the statistical properties of random radiance fields; most information (generally about 85 to 95 percent) is contained in the signal intensity transitions rather than levels; and performance is optimized when the OTF of the imaging system is nearly limited to the sampling passband to minimize aliasing at the cost of blurring, and the SNR is very high to permit the retrieval of small spatial detail from the extensively blurred signal. Shading the lens aperture transmittance to increase depth of field and using a regular hexagonal sensor-array instead of square lattice to decrease sensitivity to edge orientation also improves the signal information density up to about 30 percent at high SNRs.
Huck, F. O.↗
An information theory approach for evaluating earth radiation budget (ERB) measurements - Nonuniform sampling of diurnal longwave flux variations
Satellite measurements are subject to a wide range of uncertainties due to their temporal, spatial, and directional sampling characteristics. An information-theory approach is suggested to examine the nonuniform temporal sampling of ERB measurements. The information (i.e., its entropy or uncertainty) before and after the measurements is determined, and information gain (IG) is defined as a reduction in the uncertainties involved. A stochastic model for the diurnal outgoing flux variations that affect the ERB is developed. Using Gaussian distributions for the a priori and measured radiant exitance fields, the IG is obtained by computing the a posteriori covariance. The IG for the monthly outgoing flux measurements is examined for different orbital parameters and orbital tracks, using the Earth Observing System orbital parameters as specific examples. Variations in IG due to changes in the orbit's inclination angle and the initial ascending node local time are investigated.
Halyo, Nesim↗
What Information Theory Says About Best Response and About Binding Contracts
Product Distribution (PD) theory is the information-theoretic extension of conventional full- rationality game theory to bounded rational games. Here PD theory is used to investigate games in which the players use bounded rational best-response strategies. This investigation illuminates how to determine the optimal organization chart for a corporation, or more generally how to order the sequence of moves of the players / employees so as to optimize an overall objective function. It is then shown that in the continuum-time limit, bounded rational best response games result in a variant of the replicator dynamics of evolutionary game theory. This variant is then investigated for team games, in which the players share the same utility function, by showing that such continuum- limit bounded rational best response is identical to Newton-Raphson iterative optimization of the shared utility function. Next PD theory is used to investigate changing the coordinate system of the game, i.e., changing the mapping from the joint move of the players to the arguments in the utility functions. Such a change couples those arguments, essentially by making each players move be an offered binding contract.
Wolpert, David H.↗
Generalised squeezing and information theory approach to quantum entanglement
It is shown that the usual one- and two-mode squeezing are based on reducible representations of the SU(1,1) group. Generalized squeezing is introduced with the use of different SU(1,1) rotations on each irreducible sector. Two-mode squeezing entangles the modes and information theory methods are used to study this entanglement. The entanglement of three modes is also studied with the use of the strong subadditivity property of the entropy.
Vourdas, A.↗
The measurement of the earth's radiation budget as a problem in information theory - A tool for the rational design of earth observing systems
The measurement of the earth's radiation budget has been chosen to illustrate the technique of objective system design. The measurement process is an approximately linear transformation of the original field of radiant exitances, so that linear statistical techniques may be employed. The combination of variability, measurement strategy, and error propagation is presently made with the help of information theory, as suggested by Kondratyev et al. (1975) and Peckham (1974). Covariance matrices furnish the quantitative statement of field variability.
Barkstrom, B. R.↗
Maximizing efficiency of dataset compression for machine learning potentials with information theory
Machine learning interatomic potentials (MLIPs) balance high accuracy and lower costs compared to density functional theory calculations, but their performance often depends on the size and diversity of training datasets. Large datasets improve model accuracy and generalization but are computationally expensive to produce and train on, while smaller datasets risk discarding rare but important atomic environments and compromising MLIP accuracy/reliability. Here, we develop an information-theoretical framework to quantify the efficiency of dataset compression methods and propose an algorithm that maximizes this efficiency. By framing atomistic dataset compression as an instance of the minimum set cover (MSC) problem over atom-centered environments, our method identifies the smallest subset of structures that contains as much information as possible from the original dataset while pruning redundant information. The approach is extensively demonstrated on the GAP-20 and TM23 datasets and validated on 64 varied datasets from the ColabFit repository. Across all cases, MSC consistently retains outliers, preserves dataset diversity, and reproduces the long-tail distributions of forces even at high compression rates, outperforming other subsampling methods. Furthermore, MLIPs trained on MSC-compressed datasets exhibit reduced error for out-of-distribution data even in low-data regimes. We explain these results using an outlier analysis and show that such quantitative conclusions could not be achieved with conventional dimensionality reduction methods. The algorithm is implemented in the open-source QUESTS package and can be used for several tasks in atomistic modeling, from data subsampling, outlier detection, and training improved MLIPs at a lower cost.
36 MATERIALS SCIENCE↗
Model-free estimation of completeness, uncertainties, and outliers in atomistic machine learning using information theory
Abstract An accurate description of information is relevant for a range of problems in atomistic machine learning (ML), such as crafting training sets, performing uncertainty quantification (UQ), or extracting physical insights from large datasets. However, atomistic ML often relies on unsupervised learning or model predictions to analyze information contents from simulation or training data. Here, we introduce a theoretical framework that provides a rigorous, model-free tool to quantify information contents in atomistic simulations. We demonstrate that the information entropy of a distribution of atom-centered environments explains known heuristics in ML potential developments, from training set sizes to dataset optimality. Using this tool, we propose a model-free UQ method that reliably predicts epistemic uncertainty and detects out-of-distribution samples, including rare events in systems such as nucleation. This method provides a general tool for data-driven atomistic modeling and combines efforts in ML, simulations, and physical explainability.
36 MATERIALS SCIENCE↗
Relativistic quantum information theory
Explore the source record for details and available documents.
information theory quantum theory relativity↗
Debates: Does Information Theory Provide a New Paradigm for Earth Science? Emerging Concepts and Pathways of Information Physics
Explore the source record for details and available documents.
Rui A.P. Perdigão↗
What Information Theory Says about Bounded Rational Best Response
Probability Collectives (PC) provides the information-theoretic extension of conventional full-rationality game theory to bounded rational games. Here an explicit solution to the equations giving the bounded rationality equilibrium of a game is presented. Then PC is used to investigate games in which the players use bounded rational best-response strategies. Next it is shown that in the continuum-time limit, bounded rational best response games result in a variant of the replicator dynamics of evolutionary game theory. It is then shown that for team (shared-payoff) games, this variant of replicator dynamics is identical to Newton-Raphson iterative optimization of the shared utility function.
Wolpert, David H.↗
The problem of applying information theory to efficient image transmission.
The main ideas of Shannon's (1948, 1960) theory of source encoding with a fidelity constraint, more commonly known as rate distortion theory, are summarized. The theory was specifically intended to provide a theoretical basis for efficient transmission of information such as images. What the theory has to contribute to the problem is demonstrated. Difficulties that impeded application of the theory to image transmission, and current efforts to solve these difficulties are discussed.
Sakrison, D. J.↗
An information theory of image gathering
Shannon's mathematical theory of communication is extended to image gathering. Expressions are obtained for the total information that is received with a single image-gathering channel and with parallel channels. It is concluded that the aliased signal components carry information even though these components interfere with the within-passband components in conventional image gathering and restoration, thereby degrading the fidelity and visual quality of the restored image. An examination of the expression for minimum mean-square-error, or Wiener-matrix, restoration from parallel image-gathering channels reveals a method for unscrambling the within-passband and aliased signal components to restore spatial frequencies beyond the sampling passband out to the spatial frequency response cutoff of the optical aperture.
Fales, Carl L.↗
On the assessment of visual communication by information theory
This assessment of visual communication integrates the optical design of the image-gathering device with the digital processing for image coding and restoration. Results show that informationally optimized image gathering ordinarily can be relied upon to maximize the information efficiency of decorrelated data and the visual quality of optimally restored images.
Huck, Friedrich O.↗
Information Theory - The Bridge Connecting Bounded Rational Game Theory and Statistical Physics
A long-running difficulty with conventional game theory has been how to modify it to accommodate the bounded rationality of all red-world players. A recurring issue in statistical physics is how best to approximate joint probability distributions with decoupled (and therefore far more tractable) distributions. This paper shows that the same information theoretic mathematical structure, known as Product Distribution (PD) theory, addresses both issues. In this, PD theory not only provides a principle formulation of bounded rationality and a set of new types of mean field theory in statistical physics; it also shows that those topics are fundamentally one and the same.
Wolpert, David H.↗
An information theory characterization of radar images and a new definition for radiometric resolution
The noise properties of the radar image formation process are used in the present modeling of a communication channel in which the desired target properties are the information transmitted, and the final image represents the received signal. The average information rate over this communication channel is calculated together with appropriate bounds and approximations, and is found to be small on a per-sample basis. As a result, many samples must be averaged to allow for the discrimination, or classification, of several levels of target reflectivity. These information rate properties are consistent with known results concerning target detection and image quality in speckle, and the rate is applicable to the definition of radar image radiometric resolution. Radiometric resolution is functionally related to the degree of noncoherent averaging performed by the sensor.
Frost, V. S.↗
Using information Theory in Optimal Test Point Selection for Health Management in NASA's Exploration Vehicles
In this paper, we will present a new methodology that measures the "worth" of deploying an additional testing instrument (sensor) in terms of the amount of information that can be retrieved from such measurement. This quantity is obtained using a probabilistic model of RLV's that has been partially developed in the NASA Ames Research Center. A number of correlated attributes are identified and used to obtain the worth of deploying a sensor in a given test point from an information-theoretic viewpoint. Once the information-theoretic worth of sensors is formulated and incorporated into our general model for IHM performance, the problem can be formulated as a constrained optimization problem where reliability and operational safety of the system as a whole is considered. Although this research is conducted specifically for RLV's, the proposed methodology in its generic form can be easily extended to other domains of systems health monitoring.
Mehr, Ali Farhang↗