Mathematics > Statistics Theory
[Submitted on 23 Nov 2023]
Title:Semidefinite programming on population clustering: a local analysis
View PDF HTML (experimental)Abstract:In this paper, we consider the problem of partitioning a small data sample of size $n$ drawn from a mixture of $2$ sub-gaussian distributions. In particular, we design and analyze two computational efficient algorithms to partition data into two groups approximately according to their population of origin given a small sample in a recent paper (Zhou 2023a). Our work is motivated by the application of clustering individuals according to their population of origin using markers, when the divergence between any two of the populations is small. Moreover, we are interested in the case that individual features are of low average quality $\gamma$, and we want to use as few of them as possible to correctly partition the sample. Here we use $p \gamma$ to denote the $\ell_2^2$ distance between two population centers (mean vectors), namely, $\mu^{(1)}$, $\mu^{(2)}$ $\in$ ${\mathbb R}^p$. We allow a full range of tradeoffs between $n, p, \gamma$ in the sense that partial recovery (success rate $< 100\%$) is feasible once the signal to noise ratio $s^2 := \min\{np \gamma^2, p \gamma\}$ is lower bounded by a constant. Our work builds upon the semidefinite relaxation of an integer quadratic program that is formulated essentially as finding the maximum cut on a graph, where edge weights in the cut represent dissimilarity scores between two nodes based on their $p$ features in Zhou (2023a). More importantly, we prove that the misclassification error decays exponentially with respect to the SNR $s^2$ in the present paper. The significance of such an exponentially decaying error bound is: when $s^2 =\Omega(\log n)$, perfect recovery of the cluster structure is accomplished. This result was introduced in Zhou (2023a) without a proof. We therefore present the full proof in the present work.
Current browse context:
stat
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.