Statistics > Methodology
[Submitted on 7 Jul 2022]
Title:Comparing Confidence Intervals for a Binomial Proportion with the Interval Score
View PDFAbstract:There are over 55 different ways to construct a confidence respectively credible interval (CI) for the binomial proportion. Methods to compare them are necessary to decide which should be used in practice. The interval score has been suggested to compare prediction intervals. This score is a proper scoring rule that combines the coverage as a measure of calibration and the width as a measure of sharpness. We evaluate eleven CIs for the binomial proportion based on the expected interval score and propose a summary measure which can take into account different weighting of the underlying true proportion. Under uniform weighting, the expected interval score recommends the Wilson CI or Bayesian credible intervals with a uniform prior. If extremely low or high proportions receive more weight, the score recommends Bayesian credible intervals based on Jeffreys' prior. While more work is needed to theoretically justify the use of the interval score for the comparison of CIs, our results suggest that it constitutes a useful method to combine coverage and width in one measure. This novel approach could also be used in other applications.
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.