Interpretations of directed information in portfolio theory, data compression, and hypothesis testing
Haim H. Permuter, Young-Han Kim, and Tsachy Weissman
IEEE Transactions on Information Theory, vol. 57, no. 3, pp. 3248–3259, June 2011.
Preliminary results appeared in Proceedings of IEEE International Symposium on Information Theory, pp. 1403–1407, Toronto, Canada, July 2008.
We investigate the role of directed information in portfolio theory,
data compression, and statistics with causality constraints. In
particular, we show that directed information is an upper bound on the
increment in growth rates of optimal portfolios in a stock market due
to causal side information. This upper bound is tight for gambling in
a horse race, which is an extreme case of stock markets. Directed
information also characterizes the value of causal side information in
instantaneous compression and quantifies the benefit of causal
inference in joint compression of two stochastic processes. In
hypothesis testing, directed information evaluates the best error
exponent for testing whether a random process 
causally influences
another process 
or not. These results lead to a natural
interpretation of directed information 
as the amount
of information that a random sequence 
causally provides about another random sequence 
. A new measure, directed lautum information, is also
introduced and interpreted in portfolio theory, data compression, and
hypothesis testing.