Search NASA⌕ Search

SEARCH · Search NASA

Results for “Information theory entropy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Transient anisotropic kernel for probabilistic learning on manifolds

PLoM (Probabilistic Learning on Manifolds) is a method introduced in 2016 for handling small training datasets by projecting an Itô equation from a stochastic dissipative Hamiltonian dynamical system, acting as the MCMC generator, for which the KDE-estimated probability measure with the training dataset is the invariant measure. PLoM performs a projection on a reduced-order vector basis related to the training dataset, using the diffusion maps (DMAPS) basis constructed with a time-independent isotropic kernel. In this paper, we propose a new ISDE projection vector basis built from a transient anisotropic kernel, providing an alternative to the DMAPS basis to improve statistical surrogates for stochastic manifolds with heterogeneous data. The construction ensures that for times near the initial time, the DMAPS basis coincides with the transient basis. For larger times, the differences between the two bases are characterized by the angle of their spanned vector subspaces. The optimal instant yielding the optimal transient basis is determined using an estimation of mutual information from Information Theory, which is normalized by the entropy estimation to account for the effects of the number of realizations used in the estimations. Consequently, this new vector basis better represents statistical dependencies in the learned probability measure for any dimension. Three applications with varying levels of statistical complexity and data heterogeneity validate the proposed theory, showing that the transient anisotropic kernel improves the learned probability measure.

Diffusion maps↗
DOE OSTI2022

Misanthropic entropy and renormalization as a communication channel

A central physical question is the extent to which infrared (IR) observations are sufficient to reconstruct a candidate ultraviolet (UV) completion. We recast this question as a problem of communication, with messages encoded in field configurations of the UV being transmitted to the IR degrees of freedom via a noisy channel specified by renormalization group (RG) flow, with noise generated by coarse graining/decimation. We present an explicit formulation of these considerations in terms of lattice field theory, where we show that the “misanthropic entropy” — the mutual information obtained from decimating neighbors — encodes the extent to which information is lost in marginalizing over/tracing out UV degrees of freedom. Our considerations apply both to statistical field theories as well as density matrix renormalization of quantum systems, where in the quantum case, the statistical field theory analysis amounts to a leading-order approximation. As a concrete example, we focus on the case of the 2D Ising model, where we show that the misanthropic entropy detects the onset of the phase transition at the Ising model critical point.

Physics↗
DOE OSTI2023

The entropy of galaxy spectra: how much information is encoded?

Abstract The inverse problem of extracting the stellar population content of galaxy spectra is analysed here from a basic standpoint based on information theory. By interpreting spectra as probability distribution functions, we find that galaxy spectra have high entropy, thus leading to a rather low effective information content. The highest variation in entropy is unsurprisingly found in regions that have been well studied for decades with the conventional approach. We target a set of six spectral regions that show the highest variation in entropy – the 4000 Å break being the most informative one. As a test case with real data, we measure the entropy of a set of high-quality spectra from the Sloan Digital Sky Survey, and contrast entropy-based results with the traditional method based on line strengths. The data are classified into star-forming (SF), quiescent (Q), and active galactic nucleus (AGN) galaxies, and show – independently of any physical model – that AGN spectra can be interpreted as a transition between SF and Q galaxies, with SF galaxies featuring a more diverse variation in entropy. The high level of entanglement complicates the determination of population parameters in a robust, unbiased way, and affects traditional methods that compare models with observations, as well as machine learning (especially deep learning) algorithms that rely on the statistical properties of the data to assess the variations among spectra. Entropy provides a new avenue to improve population synthesis models so that they give a more faithful representation of real galaxy spectra.

Ferreras, Ignacio (ORCID:0000000345843127)↗
NASA NTRS1982

Entropy, instrument scan and pilot workload

Correlation and information theory which analyze the relationships between mental loading and visual scanpath of aircraft pilots are described. The relationship between skill, performance, mental workload, and visual scanning behavior are investigated. The experimental method required pilots to maintain a general aviation flight simulator on a straight and level, constant sensitivity, Instrument Landing System (ILS) course with a low level of turbulence. An additional periodic verbal task whose difficulty increased with frequency was used to increment the subject's mental workload. The subject's looppoint on the instrument panel during each ten minute run was computed via a TV oculometer and stored. Several pilots ranging in skill from novices to test pilots took part in the experiment. Analysis of the periodicity of the subject's instrument scan was accomplished by means of correlation techniques. For skilled pilots, the autocorrelation of instrument/dwell times sequences showed the same periodicity as the verbal task. The ability to multiplex simultaneous tasks increases with skill. Thus autocorrelation provides a way of evaluating the operator's skill level.

Tole, J. R.↗
NASA NTRS2022

Molecular information theory meets protein folding

We propose an application of molecular information theory to analyze the folding of single domain proteins. We analyze results from various areas of protein science, such as sequence-based potentials, reduced amino acid alphabets, backbone configurational entropy, secondary structure content, residue burial layers, and mutational studies of protein stability changes. We found that the average information contained in the sequences of evolved proteins is very close to the average information needed to specify a fold ~2.2 ± 0.3 bits/(site operation). The effective alphabet size in evolved proteins equals the effective number of conformations of a residue in the compact unfolded state at around 5. We calculated an energy-to-information conversion efficiency upon folding of around 50%, lower than the theoretical limit of 70%, but much higher than human built macroscopic machines. We propose a simple mapping between molecular information theory and energy landscape theory and explore the connections between sequence evolution, configurational entropy and the energetics of protein folding.

Ignacio E. Sánchez↗
DOE OSTI2021

Entanglement in Gravity and Quantum Field Theory (Final Report)

It is becoming increasingly clear that ideas from quantum information theory, particularly the notion of quantum entanglement, play a fundamental role in some of the deepest aspects of our modern theories of quantum fields and gravity. The aim of this research was to explore the role that quantum entanglement plays in quantum field theories and in the nature of space-time and gravity. Building on a variety of earlier results obtained in these regards at the University of Illinois, we explored the constraints on the dynamical content of quantum field theories that follow from their entanglement properties. Topological field theories are important examples of particularly simple quantum field theories whose patterns of entanglement make connections between high energy physics, condensed matter physics and mathematics. These theories are directly relevant to low energy properties of certain materials. The study of such theories allowed us to investigate ideas that are relevant to quantum information research, such as new notions of entanglement between multiple parties and the quantum properties of interfaces between different phases of such materials. In addition, we employed new results in mathematics which strengthen monotonicity constraints on relative entropy to study their ramifications in quantum field theories, and we used quantum information methods to study the emergence of quantum gravity and string theory in holographic quantum field theories.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1985

Information theoretic comparisons of original and transformed data from Landsat MSS and TM

The dispersion and concentration of signal values in transformed data from the Landsat-4 MSS and TM instruments are analyzed using a communications theory approach. The definition of entropy of Shannon was used to quantify information, and the concept of mutual information was employed to develop a measure of information contained in several subsets of variables. Several comparisons of information content are made on the basis of the information content measure, including: system design capacities; data volume occupied by agricultural data; and the information content of original bands and Tasseled Cap variables. A method for analyzing noise effects in MSS and TM data is proposed.

Malila, W. A.↗
DOE OSTI2020

Time dependence of reflected entropy in rational and holographic conformal field theories

We calculate the time dependence of the reflected entropy of two disconnected regions after a global quench in (1 + 1)-dimensional conformal field theories and in large temperature limit. For rational conformal field theories, we find that the time evolution of the reflected entropy is the same as that of the mutual information. We get the same result for holographic theories in the limit where the separation between disconnected regions is much smaller than their respective sizes. We discuss how this result is consistent with the quasi-particle picture of Calabrese and Cardy.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS2004

Measuring Questions: Relevance and its Relation to Entropy

The Boolean lattice of logical statements induces the free distributive lattice of questions. Inclusion on this lattice is based on whether one question answers another. Generalizing the zeta function of the question lattice leads to a valuation called relevance or bearing, which is a measure of the degree to which one question answers another. Richard Cox conjectured that this degree can be expressed as a generalized entropy. With the assistance of yet another important result from Janos Acz6l, I show that this is indeed the case; and that the resulting inquiry calculus is a natural generalization of information theory. This approach provides a new perspective of the Principle of Maximum Entropy.

Knuth, Kevin H.↗
DOE OSTI2024

Maximum Entropy Theory of Multiscale Coarse-Graining via Matching Thermodynamic Forces: Application to a Molecular Crystal (TATB)

The MSCG/FM (multiscale coarse-graining via force-matching) approach is an efficient supervised machine learning method to develop microscopically informed coarse-grained (CG) models. Here we present a theory based on the principle of maximum entropy (PME) enveloping the existing MSCG/FM approaches. This theory views the MSCG/FM method as a special case of matching the thermodynamic forces from the extended ensemble described by the set of thermodynamic (relevant) system coordinates. This set may include CG coordinates, the stress tensor, applied external fields, and so forth, and may be characterized by nonequilibrium conditions. Following the presentation of the theory, we discuss the consistent matching of both bonded and nonbonded interactions. The proposed PME formulation is used as a starting point to extend the MSCG/FM method to the constant strain ensemble, which together with the explicit matching of the bonded forces is better suited for coarse-graining anisotropic media at a submolecular resolution. The theory is demonstrated by performing the fine coarse-graining of crystalline 1,3,5-triamino-2,4,6-trinitrobenzene (TATB), a well-known insensitive molecular energetic material, which exhibits highly anisotropic mechanical properties.

1,3,5-triamino-2,4,6-trinitrobenzene↗
NASA NTRS1991

A safety-based decision making architecture for autonomous systems

Engineering systems designed specifically for space applications often exhibit a high level of autonomy in the control and decision-making architecture. As the level of autonomy increases, more emphasis must be placed on assimilating the safety functions normally executed at the hardware level or by human supervisors into the control architecture of the system. The development of a decision-making structure which utilizes information on system safety is detailed. A quantitative measure of system safety, called the safety self-information, is defined. This measure is analogous to the reliability self-information defined by McInroy and Saridis, but includes weighting of task constraints to provide a measure of both reliability and cost. An example is presented in which the safety self-information is used as a decision criterion in a mobile robot controller. The safety self-information is shown to be consistent with the entropy-based Theory of Intelligent Machines defined by Saridis.

Musto, Joseph C.↗
NASA NTRS1991

Fuzzy geometry, entropy, and image information

Presented here are various uncertainty measures arising from grayness ambiguity and spatial ambiguity in an image, and their possible applications as image information measures. Definitions are given of an image in the light of fuzzy set theory, and of information measures and tools relevant for processing/analysis e.g., fuzzy geometrical properties, correlation, bound functions and entropy measures. Also given is a formulation of algorithms along with management of uncertainties for segmentation and object extraction, and edge detection. The output obtained here is both fuzzy and nonfuzzy. Ambiguity in evaluation and assessment of membership function are also described.

Pal, Sankar K.↗
DOE OSTI2020

The dual of non-extremal area: differential entropy in higher dimensions

The Ryu-Takayanagi formula relates entanglement entropy in a field theory to the area of extremal surfaces anchored to the boundary of a dual AdS space. It is interesting to ask if there is also an information theoretic interpretation of the areas of non-extremal surfaces that are not necessarily boundary-anchored. In general, the physics outside such surfaces is associated to observers restricted to a time-strip in the dual boundary field theory. When the latter is two-dimensional, it is known that the differential entropy associated to the strip computes the length of the dual bulk curve, and has an interpretation in terms of the information cost in Bell pairs of restoring correlations inaccessible to observers in the strip. A general realization of this formalism in higher dimensions is unknown. We first prove a no-go theorem eliminating candidate expressions for higher dimensional differential entropy based on entropic c-theorems. Then we propose a new formula in terms of an integral of shape derivatives of the entanglement entropy of ball shaped regions. Our proposal stems from the physical requirement that differential entropy must be locally finite and conformally invariant. Demanding cancelation of the well-known UV divergences of entanglement entropy in field theory guides us to our conjecture, which we test for surfaces in AdS 4 . Our results suggest a candidate c-function for field theories in arbitrary dimensions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS2007

Understanding Local Structure Globally in Earth Science Remote Sensing Data Sets

Empirical probability distributions derived from the data are the signatures of physical processes generating the data. Distributions defined on different space-time windows can be compared and differences or changes can be attributed to physical processes. This presentation discusses on ways to reduce remote sensing data in a way that preserves information, focusing on the rate-distortion theory and using the entropy-constrained vector quantization algorithm.

massive data sets↗
DOE OSTI2025

Environmental Controls on Water Vapor Deuterium Excess in the Coastal Boundary Layer: An Information Theory Perspective

We use information theory to quantify the environmental controls on water vapor deuterium excess (D-excess) in coastal Southern California from June 2023 through February 2024. Using Shannon entropy, mutual information (MI), and joint mutual information, metrics that capture both linear and nonlinear relationships, we identify the most informative variables and variable combinations governing D-excess across contrasting marine and continental regimes. Relative humidity with respect to sea surface temperature (RHS) is consistently the strongest individual predictor, explaining up to 27% of D-excess variability during marine conditions but only 10% in continental air masses. The Relative humidity(RHS) + sea surface temperature (SST) combination demonstrates synergistic effects, where their joint influence (explaining up to 36% of D-excess variability) exceeds what either variable achieves individually, confirming their coupled influence on deuterium excess. Wind direction complements RHS most effectively during continental conditions. The best three-variable combination (RHS + SST + Planetary Boundary Layer height) explains 38% of D-excess variability in marine air, while no combination exceeds 20% explanatory power during continental periods. Information theory shows that heteroscedasticity in D-excess relationships indicates regime shifts in controlling processes and quantifies fundamental constraints on predictor variables: some environmental factors like surface pressure or water vapor flux contain insufficient information content to explain D-excess variability regardless of their physical relevance. These results highlight the different predictability limits between marine and continental regimes, challenging the adequacy of linear models and providing a rigorous framework for quantifying the information content of isotope-climate relationships with implications for both modern and paleoclimate applications.

information theory↗
DOE OSTI2024

The expressivity of classical and quantum neural networks on entanglement entropy

Abstract Analytically continuing the von Neumann entropy from Rényi entropies is a challenging task in quantum field theory. While then-th Rényi entropy can be computed using the replica method in the path integral representation of quantum field theory, the analytic continuation can only be achieved for some simple systems on a case-by-case basis. In this work, we propose a general framework to tackle this problem using classical and quantum neural networks with supervised learning. We begin by studying several examples with known von Neumann entropy, where the input data is generated by representing$${\text {Tr}}\rho _A^n$$ Tr ρ A n with a generating function. We adopt KerasTuner to determine the optimal network architecture and hyperparameters with limited data. In addition, we frame a similar problem in terms of quantum machine learning models, where the expressivity of the quantum models for the entanglement entropy as a partial Fourier series is established. Our proposed methods can accurately predict the von Neumann and Rényi entropies numerically, highlighting the potential of deep learning techniques for solving problems in quantum information theory.

Physics↗
NASA NTRS1985

Comparison of the information contents of Landsat TM and MSS data

A communications-theory approach is taken to analyze the dispersion and concentration of signal values in various data spaces, irrespective of specific class membership. Entropy is used to quantify information, and mutual information is used to measure the information represented by subsets of spectral variables. Several different comparisons of information content are made. These include comparisons of system design capacities, of data volumes occupied by agricultural data in the spaces defined by original bands and by transformed spectral (Tasseled Cap) variables, of the information contents of original bands and Tasseled Cap variables, and of the information contents of TM and MSS for the given agricultural data sets. Also, the effects of sample size, scene content, and quantization level are examined.

Malila, W. A.↗