On the Optimality of Sufficient Statistics-Based Quantizers.

Abstract

Let X be a random variable taking values in a set X, and let {P_θ; θ ∈ Θ} be a family of distributions indexed by the parameter vector θ taking values in a set Θ. A quantized random variable γ(X) is obtained by employing a quantizer γ: X→ {1,…,K}. It is shown that any extreme point of the set of all possible probability distributions of γ(X) can be achieved by a deterministic quantizer that decides based only on the sufficient statistics. Using this fact, optimality properties of deterministic sufficient statistics-based quantizers are established for the problem of parameter estimation. It is proven that there always exists an optimal partitioning of sufficient statistics into K convex polytopes which maximizes the trace of the Fisher information matrix when {P_θ; θ ∈ Θ} belongs to the exponential family. Furthermore, the optimality of likelihood ratio statistic for simple hypothesis testing follows as a consequence of this result, thereby demonstrating a link between parameter estimation and hypothesis testing.