Project 3
In information theory it is often important to compare different measures of divergence, i.e. functions which measure how different two probability measures are. The most important class of divergence measures are the so-called f-divergences defined by
where f is
a convex function with f(1)=0, and pi and qi
are the point probabilities of P and Q. Important examples of f-divergences
are total variation, information divergence, Hellinger divergence, Jensen Shannon
divergence and 
. We are interested in the joint range 
for pairs of
functions 
. In particular we are interested in lower and upper bounds
of one f-divergence given another. Tight bounds are known for the
following pairs:
- total variation/information divergence
- total variation/Hellinger divergence
- total variation/Jensen-Shannon divergence
- information divergence/Hellinger divergence
It is known that the problem can be reduced to distributions on a two-element set, which makes numerical computations very simple. In the project one should explore the theory of f-divergences and compare one or more pairs of f-divergences.
Litteratur
Csiszár, On Topological Properties
of F-Divergences. Studia Sci. Math. Hungar., vol. 2, pp. 329-339, 1967.