Title:Consistent order estimation and the local geometry of mixtures

Abstract: Consider an i.i.d. sequence of random variables whose distribution $f^\star$ lies in one of a nested family of models $(\mathcal{M}_q)_{q\in\mathbb{N}}$, $\mathcal{M}_q\subset \mathcal{M}_{q+1}$. The smallest index $q^\star$ such that $\mathcal{M}_{q^\star}$ contains $f^\star$ is called the model order. We establish strong consistency of the penalized likelihood order estimator in a general setting with penalties of order $\eta(q)\log\log n$, where $\eta(q)$ is a dimensional quantity. Moreover, such penalties are shown to be minimal. In contrast to previous work, an a priori upper bound on the model order is not assumed. The local dimension $\eta(q)$ of the model $\mathcal{M}_q$ is defined in terms of the bracketing entropy of a class of weighted densities, whose computation is a nonstandard problem which is of independent interest. We perform the requisite computations for the case of one-dimensional location mixtures, thus demonstrating the consistency of the penalized likelihood mixture order estimator. The proof requires a delicate analysis of the local geometry of the mixture family $\mathcal{M}_q$ in a neighborhood of $f^\star$, for $q>q^\star$. The extension to more general mixture models remains an open problem.