Engineering topics

Dembo, Amir

Publications and source records attributed to Dembo, Amir.

4 records

NASA NTRS1991

A general weight matrix formulation using optimal control

Classical methods from optimal control theory are used in deriving general forms for neural network weights. The network learning or application task is encoded in a performance index of a general structure. Consequently, different instances of this performance index lead to special cases of weight rules, including some well-known forms. Comparisons are made with the outer product rule, spectral methods, and recurrent back-propagation. Simulation results and comparisons are presented.

Farotimi, Oluseyi↗

NASA NTRS1990

Model-free distributed learning

Model-free learning for synchronous and asynchronous quasi-static networks is presented. The network weights are continuously perturbed, while the time-varying performance index is measured and correlated with the perturbation signals; the correlation output determines the changes in the weights. The perturbation may be either via noise sources or orthogonal signals. The invariance to detailed network structure mitigates large variability between supposedly identical networks as well as implementation defects. This local, regular, and completely distributed mechanism requires no central control and involves only a few global signals. Thus it allows for integrated on-chip learning in large analog and optical networks.

Dembo, Amir↗

NASA NTRS1989

On Gaussian feedback capacity

Pinsker and Ebert (1970) proved that in channels with additive Gaussian noise, feedback at most doubles the capacity. Cover and Pombra (1989) proved that feedback at most adds half a bit per transmission. Following their approach, the author proves that in the limit as signal power approaches either zero (very low SNR) or infinity (very high SNR), feedback does not increase the finite block-length capacity (which for nonstationary Gaussian channels replaces the standard notion of capacity that may not exist). Tighter upper bounds on the capacity are obtained in the process. Specializing these results to stationary channels, the author recovers some of the bounds recently obtained by Ozarow.

Dembo, Amir↗

NASA NTRS1989

Simple proof of the concavity of the entropy power with respect to Gaussian noise

A very simple proof of M. H. Costa's result that the entropy power of Xt = X + N (O, tI) is concave in t, is derived as an immediate consequence of an inequality concerning Fisher information. This relationship between Fisher information and entropy is found to be useful for proving the central limit theorem. Thus, one who seeks new entropy inequalities should try first to find new inequalities about Fisher information, or at least to exploit the existing ones in new ways.

Dembo, Amir↗