Statistics > Machine Learning
[Submitted on 27 Dec 2019 (v1), revised 21 Feb 2020 (this version, v4), latest version 14 May 2021 (v5)]
Title:The Chi-Square Test of Distance Correlation
View PDFAbstract:Distance correlation has gained much recent attention in the data science community: the sample statistic is straightforward to compute and asymptotically equals zero if and only if independence, making it an ideal choice to test any type of dependency structure given sufficient sample size. One major bottleneck is the testing process: because the null distribution of distance correlation depends on the underlying random variables and metric choice, it typically requires a permutation test to estimate the null and compute the p-value, which is very costly for large amount of data. To overcome the difficulty, we propose a centered chi-square distribution, demonstrate it well-approximates the limiting null distribution of unbiased distance correlation, and prove upper tail dominance and distribution bound. The resulting distance correlation chi-square test is a nonparametric test for independence, is valid and universally consistent using any strong negative type metric or characteristic kernel, enjoys a similar finite-sample testing power as the standard permutation test, is provably most powerful among all valid tests of distance correlation using known distributions, and is also applicable to K-sample and partial testing.
Submission history
From: Cencheng Shen [view email][v1] Fri, 27 Dec 2019 15:16:40 UTC (144 KB)
[v2] Tue, 7 Jan 2020 15:08:41 UTC (142 KB)
[v3] Wed, 22 Jan 2020 21:53:47 UTC (143 KB)
[v4] Fri, 21 Feb 2020 21:35:39 UTC (143 KB)
[v5] Fri, 14 May 2021 18:09:51 UTC (154 KB)
Current browse context:
stat.ML
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.