Applications
See recent articles
- [1] arXiv:2408.11874 [pdf, html, other]
-
Title: Investigating Mode Effects in Interviewer Variances Using Two Representative Multi-mode SurveysSubjects: Applications (stat.AP)
This study examines whether interviewer variances remain consistent across different modes in mixed-mode studies, using data from two distinct designs. In the first design, when interviewers are responsible for either face-to-face or telephone mode, we examine whether there are mode differences in interviewer variances for 1) sensitive political questions, 2) international items, 3) and item missing indicators on international items, using the Arab Barometer wave 6 Jordan data. In the second design, we draw on Health and Retirement Study (HRS) 2016 core survey data to examine the question on three topics when interviewers are responsible for both modes. The topics cover 1) the CESD depression scale, 2) interviewer observations, and 3) the physical activity scale. To account for the lack of interpenetrated designs in both data sources, we include respondent-level covariates in our models. We find significant differences in interviewer variances on one item (twelve items in total) in the Arab Barometer study; whereas for HRS, the results are three out of eighteen. Overall, we find the magnitude of the interviewer variances larger in FTF than TEL on sensitive items. We conduct simulations to understand the power to detect mode effects in the typically modest interviewer sample sizes.
- [2] arXiv:2408.11922 [pdf, other]
-
Title: Evaluating Four Methods for Detecting Differential Item Functioning in Large-Scale Assessments with More Than Two GroupsComments: preprint, 16 pages (excluding figures, references, and title page)Subjects: Applications (stat.AP)
This study evaluated four multi-group differential item functioning (DIF) methods (the root mean square deviation approach, Wald-1, generalized logistic regression procedure, and generalized Mantel-Haenszel method) via Monte Carlo simulation of controlled testing conditions. These conditions varied in the number of groups, the ability and sample size of the DIF-contaminated group, the parameter associated with DIF, and the proportion of DIF items. When comparing Type-I error rates and powers of the methods, we showed that the RMSD approach yielded the best Type-I error rates when it was used with model-predicted cutoff values. Also, this approach was found to be overly conservative when used with the commonly used cutoff value of 0.1. Implications for future research for educational researchers and practitioners were discussed.
- [3] arXiv:2408.12332 [pdf, html, other]
-
Title: Simplifying Random Forests' Probabilistic ForecastsSubjects: Applications (stat.AP); Machine Learning (stat.ML)
Since their introduction by Breiman, Random Forests (RFs) have proven to be useful for both classification and regression tasks. The RF prediction of a previously unseen observation can be represented as a weighted sum of all training sample observations. This nearest-neighbor-type representation is useful, among other things, for constructing forecast distributions (Meinshausen, 2006). In this paper, we consider simplifying RF-based forecast distributions by sparsifying them. That is, we focus on a small subset of nearest neighbors while setting the remaining weights to zero. This sparsification step greatly improves the interpretability of RF predictions. It can be applied to any forecasting task without re-training existing RF models. In empirical experiments, we document that the simplified predictions can be similar to or exceed the original ones in terms of forecasting performance. We explore the statistical sources of this finding via a stylized analytical model of RFs. The model suggests that simplification is particularly promising if the unknown true forecast distribution contains many small weights that are estimated imprecisely.
- [4] arXiv:2408.12521 [pdf, html, other]
-
Title: Disclosure risk assessment with Bayesian non-parametric hierarchical modellingSubjects: Applications (stat.AP); Computation (stat.CO)
Micro and survey datasets often contain private information about individuals, like their health status, income or political preferences. Previous studies have shown that, even after data anonymization, a malicious intruder could still be able to identify individuals in the dataset by matching their variables to external information. Disclosure risk measures are statistical measures meant to quantify how big such a risk is for a specific dataset. One of the most common measures is the number of sample unique values that are also population-unique. \cite{Man12} have shown how mixed membership models can provide very accurate estimates of this measure. A limitation of that approach is that the number of extreme profiles has to be chosen by the modeller. In this article, we propose a non-parametric version of the model, based on the Hierarchical Dirichlet Process (HDP). The proposed approach does not require any tuning parameter or model selection step and provides accurate estimates of the disclosure risk measure, even with samples as small as 1$\%$ of the population size. Moreover, a data augmentation scheme to address the presence of structural zeros is presented. The proposed methodology is tested on a real dataset from the New York census.
New submissions for Friday, 23 August 2024 (showing 4 of 4 entries )
- [5] arXiv:2408.11967 (cross-list from cs.LG) [pdf, html, other]
-
Title: Valuing an Engagement Surface using a Large Scale Dynamic Causal ModelAbhimanyu Mukerji, Sushant More, Ashwin Viswanathan Kannan, Lakshmi Ravi, Hua Chen, Naman Kohli, Chris Khawand, Dinesh MandalapuComments: 10 pages, 5 figures. Accepted at Applied Data Science track of KDD 2024, Barcelona, SpainSubjects: Machine Learning (cs.LG); Econometrics (econ.EM); Applications (stat.AP)
With recent rapid growth in online shopping, AI-powered Engagement Surfaces (ES) have become ubiquitous across retail services. These engagement surfaces perform an increasing range of functions, including recommending new products for purchase, reminding customers of their orders and providing delivery notifications. Understanding the causal effect of engagement surfaces on value driven for customers and businesses remains an open scientific question. In this paper, we develop a dynamic causal model at scale to disentangle value attributable to an ES, and to assess its effectiveness. We demonstrate the application of this model to inform business decision-making by understanding returns on investment in the ES, and identifying product lines and features where the ES adds the most value.
Cross submissions for Friday, 23 August 2024 (showing 1 of 1 entries )
- [6] arXiv:2204.06544 (replaced) [pdf, other]
-
Title: Features of the Earth's seasonal hydroclimate: Characterizations and comparisons across the Koppen-Geiger climates and across continentsJournal-ref: Progress in Earth and Planetary Science 10 (2023) 46Subjects: Applications (stat.AP); Methodology (stat.ME); Machine Learning (stat.ML)
Detailed investigations of time series features across climates, continents and variable types can progress our understanding and modelling ability of the Earth's hydroclimate and its dynamics. They can also improve our comprehension of the climate classification systems appearing in their core. Still, such investigations for seasonal hydroclimatic temporal dependence, variability and change are currently missing from the literature. Herein, we propose and apply at the global scale a methodological framework for filling this specific gap. We analyse over 13 000 earth-observed quarterly temperature, precipitation and river flow time series. We adopt the Koppen-Geiger climate classification system and define continental-scale geographical regions for conducting upon them seasonal hydroclimatic feature summaries. The analyses rely on three sample autocorrelation features, a temporal variation feature, a spectral entropy feature, a Hurst feature, a trend strength feature and a seasonality strength feature. We find notable differences to characterize the magnitudes of these features across the various Koppen-Geiger climate classes, as well as between continental-scale geographical regions. We, therefore, deem that the consideration of the comparative summaries could be beneficial in water resources engineering contexts. Lastly, we apply explainable machine learning to compare the investigated features with respect to how informative they are in distinguishing either the main Koppen-Geiger climates or the continental-scale regions. In this regard, the sample autocorrelation, temporal variation and seasonality strength features are found to be more informative than the spectral entropy, Hurst and trend strength features at the seasonal time scale.
- [7] arXiv:2207.04369 (replaced) [pdf, html, other]
-
Title: Balancing Producer Fairness and Efficiency via Prior-Weighted Rating System DesignComments: 12 pages, 8 figures, accepted to ICWSM 2025Subjects: Applications (stat.AP)
Online marketplaces use rating systems to promote the discovery of high-quality products. However, these systems also lead to high variance in producers' economic outcomes: a new producer who sells high-quality items, may unluckily receive a low rating early, severely impacting their future popularity. We investigate the design of rating systems that balance the goals of identifying high-quality products (``efficiency'') and minimizing the variance in outcomes of producers of similar quality (individual ``producer fairness'').
We show that there is a trade-off between these two goals: rating systems that promote efficiency are necessarily less individually fair to producers. We introduce prior-weighted rating systems as an approach to managing this trade-off. Informally, the system we propose sets a system-wide prior for the quality of an incoming product; subsequently, the system updates that prior to a posterior for each product's quality based on user-generated ratings over time. We show theoretically that in markets where products accrue reviews at an equal rate, the strength of the rating system's prior determines the operating point on the identified trade-off: the stronger the prior, the more the marketplace discounts early ratings data (increasing individual fairness), but the slower the platform is in learning about true item quality (so efficiency suffers). We further analyze this trade-off in a responsive market where customers make decisions based on historical ratings. Through calibrated simulations in 19 different real-world datasets sourced from large online platforms, we show that the choice of prior strength mediates the same efficiency-consistency trade-off in this setting. Overall, we demonstrate that by tuning the prior as a design choice in a prior-weighted rating system, platforms can be intentional about the balance between efficiency and producer fairness. - [8] arXiv:2407.15401 (replaced) [pdf, html, other]
-
Title: Data Space Inversion for Efficient Predictions and Uncertainty Quantification for Geothermal ModelsAlex de Beer, Andrew Power, Daniel Wong, Ken Dekkers, Michael Gravatt, Elvar K. Bjarkason, John P. O'Sullivan, Michael J. O'Sullivan, Oliver J. Maclaren, Ruanui NicholsonSubjects: Applications (stat.AP)
The ability to make accurate predictions with quantified uncertainty provides a crucial foundation for the successful management of a geothermal reservoir. Conventional approaches for making predictions using geothermal reservoir models involve estimating unknown model parameters using field data, then propagating the uncertainty in these estimates through to the predictive quantities of interest. However, the unknown parameters are not always of direct interest; instead, the predictions are of primary importance. Data space inversion (DSI) is an alternative methodology that allows for the efficient estimation of predictive quantities of interest, with quantified uncertainty, that avoids the need to estimate model parameters entirely. In this paper, we evaluate the applicability of DSI to geothermal reservoir modelling. We first review the processes of model calibration, prediction and uncertainty quantification from a Bayesian perspective, and introduce data space inversion as a simple, efficient technique for approximating the posterior predictive distribution. We then apply the DSI framework to two model problems in geothermal reservoir modelling. We evaluate the accuracy and efficiency of DSI relative to other common methods for uncertainty quantification, study how the number of reservoir model simulations affects the resulting approximation to the posterior predictive distribution, and demonstrate how the framework can be enhanced through the use of suitable reparametrisations. Our results support the idea that data space inversion is a simple, robust and efficient technique for making predictions with quantified uncertainty using geothermal reservoir models, providing a useful alternative to more conventional approaches.
- [9] arXiv:2408.03741 (replaced) [pdf, html, other]
-
Title: Varying coefficients correlated velocity models in complex landscapes with boundaries applied to narwhal responses to noise exposureComments: 29 pages, 9 figures, 7 tablesSubjects: Applications (stat.AP)
Narwhals in the Arctic are increasingly exposed to human activities that can temporarily or permanently threaten their survival by modifying their behavior. We examine GPS data from a population of narwhals exposed to ship and seismic airgun noise during a controlled experiment in 2018 in the Scoresby Sound fjord system in Southeast Greenland. The fjord system has a complex shore line, restricting the behavioral response options for the narwhals to escape the threats. We propose a new continuous-time correlated velocity model with varying coefficients that includes spatial constraints on movement. To assess the sound exposure effect we compare a baseline model for the movement before exposure to a response model for the movement during exposure. Our model, applied to the narwhal data, suggests increased tortuosity of the trajectories as a consequence of the spatial constraints, and further indicates that sound exposure can disturb narwhal motion up to a couple of tens of kilometers. Specifically, we found an increase in velocity and a decrease in the movement persistence.
- [10] arXiv:2303.03080 (replaced) [pdf, html, other]
-
Title: Defining and comparing SICR-events for classifying impaired loans under IFRS 9Comments: 41 pages (including appendix), 10601 words, 15 figuresSubjects: Risk Management (q-fin.RM); Statistical Finance (q-fin.ST); Applications (stat.AP)
The IFRS 9 accounting standard requires the prediction of credit deterioration in financial instruments, i.e., significant increases in credit risk (SICR). However, the definition of such a SICR-event is inherently ambiguous, given its current reliance on evaluating the change in the estimated probability of default (PD) against some arbitrary threshold. We examine the shortcomings of this PD-comparison approach and propose an alternative framework for generating SICR-definitions based on three parameters: delinquency, stickiness, and the outcome period. Having varied these framework parameters, we obtain 27 unique SICR-definitions and fit logistic regression models accordingly using rich South African mortgage and macroeconomic data. For each definition and corresponding model, the resulting SICR-rates are analysed at the portfolio-level on their stability over time and their responsiveness to economic downturns. At the account-level, we compare both the accuracy and dynamicity of the SICR-predictions, and discover several interesting trends and trade-offs. These results can help any bank with appropriately setting the three framework parameters in defining SICR-events for prediction purposes. We demonstrate this process by comparing the best-performing SICR-model to the PD-comparison approach, and show the latter's inferiority as an early-warning system. Our work can therefore guide the formulation, modelling, and testing of any SICR-definition, thereby promoting the timeous recognition of credit losses; the main imperative of IFRS 9.
- [11] arXiv:2308.01156 (replaced) [pdf, html, other]
-
Title: A new adaptive local polynomial density estimation procedure on complicated domainsComments: 43 pages, 4 figuresSubjects: Statistics Theory (math.ST); Probability (math.PR); Applications (stat.AP); Methodology (stat.ME)
This paper presents a novel approach for pointwise estimation of multivariate density functions on known domains of arbitrary dimensions using nonparametric local polynomial estimators. Our method is highly flexible, as it applies to both simple domains, such as open connected sets, and more complicated domains that are not star-shaped around the point of estimation. This enables us to handle domains with sharp concavities, holes, and local pinches, such as polynomial sectors. Additionally, we introduce a data-driven selection rule based on the general ideas of Goldenshluger and Lepski. Our results demonstrate that the local polynomial estimators are minimax under a $L^2$ risk across a wide range of Hölder-type functional classes. In the adaptive case, we provide oracle inequalities and explicitly determine the convergence rate of our statistical procedure. Simulations on polynomial sectors show that our oracle estimates outperform those of the most popular alternative method, found in the sparr package for the R software. Our statistical procedure is implemented in an online R package which is readily accessible.
- [12] arXiv:2311.07511 (replaced) [pdf, other]
-
Title: Uncertainty estimation of machine learning spatial precipitation predictions from satellite dataJournal-ref: Machine Learning: Science and Technology 5 (2024) 035044Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Atmospheric and Oceanic Physics (physics.ao-ph); Applications (stat.AP); Methodology (stat.ME)
Merging satellite and gauge data with machine learning produces high-resolution precipitation datasets, but uncertainty estimates are often missing. We addressed the gap of how to optimally provide such estimates by benchmarking six algorithms, mostly novel even for the more general task of quantifying predictive uncertainty in spatial prediction settings. On 15 years of monthly data from over the contiguous United States (CONUS), we compared quantile regression (QR), quantile regression forests (QRF), generalized random forests (GRF), gradient boosting machines (GBM), light gradient boosting machine (LightGBM), and quantile regression neural networks (QRNN). Their ability to issue predictive precipitation quantiles at nine quantile levels (0.025, 0.050, 0.100, 0.250, 0.500, 0.750, 0.900, 0.950, 0.975), approximating the full probability distribution, was evaluated using quantile scoring functions and the quantile scoring rule. Predictors at a site were nearby values from two satellite precipitation retrievals, namely PERSIANN (Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks) and IMERG (Integrated Multi-satellitE Retrievals), and the site's elevation. The dependent variable was the monthly mean gauge precipitation. With respect to QR, LightGBM showed improved performance in terms of the quantile scoring rule by 11.10%, also surpassing QRF (7.96%), GRF (7.44%), GBM (4.64%) and QRNN (1.73%). Notably, LightGBM outperformed all random forest variants, the current standard in spatial prediction with machine learning. To conclude, we propose a suite of machine learning algorithms for estimating uncertainty in spatial data prediction, supported with a formal evaluation framework based on scoring functions and scoring rules.
- [13] arXiv:2405.08730 (replaced) [pdf, html, other]
-
Title: A Generalized Difference-in-Differences Estimator for Randomized Stepped-Wedge and Observational Staggered Adoption SettingsComments: 38 pages main text, 23 pages additional material, 3 figures, 9 tables Update includes additional discussion and an added data example for the observational settingSubjects: Methodology (stat.ME); Applications (stat.AP)
Staggered treatment adoption arises in the evaluation of policy impact and implementation in a variety of settings. This occurs in both randomized stepped-wedge trials and non-randomized quasi-experimental panel data settings using causal inference methods based on difference-in-differences analysis. In both settings, it is crucial to carefully consider the target estimand and possible treatment effect heterogeneities in order to estimate the effect without bias and in an interpretable fashion. This paper proposes a novel non-parametric approach to this estimation for either setting. By constructing an estimator using two-by-two difference-in-difference comparisons as building blocks with arbitrary weights, the investigator can select weights to target the desired estimand in an unbiased manner under assumed treatment effect homogeneity, and minimize the variance under an assumed working covariance structure. This provides desirable bias properties while using the comparisons efficiently to mitigate the loss of precision. Weightings are shown for simple settings and two data examples are examined. The methods are used to re-analyze data from both a randomized stepped-wedge trial on the impact of novel tuberculosis diagnostic tools and an observational staggered adoption study on the effects of COVID-19 vaccine financial incentive lotteries in U.S. states; these are compared to analyses using previous methods. A full algorithm with R code is provided to implement this method and to compare to existing methods. The proposed method allows for high flexibility and clear targeting of desired effects, providing one solution to the bias-variance-generalizability tradeoff.
- [14] arXiv:2408.11792 (replaced) [pdf, other]
-
Title: Optical ISAC: Fundamental Performance Limits and Transceiver DesignComments: 8 pages, 3 figuresSubjects: Information Theory (cs.IT); Machine Learning (cs.LG); Applications (stat.AP); Computation (stat.CO)
This paper characterizes the optimal capacity-distortion (C-D) tradeoff in an optical point-to-point (P2P) system with single-input single-output for communication and single-input multiple-output for sensing (SISO-COM and SIMO-SEN) within an integrated sensing and communication (ISAC) framework. We consider the optimal rate-distortion (R-D) region and explore several inner (IB) and outer (OB) bounds. We introduce practical, asymptotically optimal maximum a posteriori (MAP) and maximum likelihood estimators (MLE) for target distance, addressing nonlinear measurement-to-state relationships and non-conjugate priors. As the number of sensing antennas increases, these estimators converge to the Bayesian Cramér-Rao bound (BCRB). We also establish that the achievable rate-CRB (AR-CRB) serves as an OB for the optimal C-D region, valid for both unbiased estimators and asymptotically large numbers of receive antennas. To clarify that the input distribution determines the tradeoff across the Pareto boundary of the C-D region, we propose two algorithms: \textit{i}) an iterative Blahut-Arimoto algorithm (BAA)-type method, and \textit{ii}) a memory-efficient closed-form (CF) approach. The CF approach includes a CF optimal distribution for high optical signal-to-noise ratio (O-SNR) conditions. Additionally, we adapt and refine the Deterministic-Random Tradeoff (DRT) to this optical ISAC context.