Title:DEDPUL: Difference-of-Estimated-Densities-based Positive-Unlabeled Learning

Abstract:Positive-Unlabeled Learning is an analog to supervised binary classification for the case when the negative (N) sample in the training set is noisy, i.e. is contaminated with latent instances of the positive (P) class and hence is unlabeled (U). The objective is to classify U, which requires to identify the mixing proportions of P and N in U first. Recently, unbiased Risk Estimation has been successfully applied to train classifiers on PU data, achieving state-of-the-art results (Kiryo et al., 2017). This approach, however, exhibits two major bottlenecks. First, the mixing proportions are assumed to be known in the domain or estimated with additional methods. Second, the approach relies on the classifier being a neural network. In this paper, we propose DEDPUL, a method that solves PU Learning without the aforementioned issues. The mechanism behind DEDPUL is to apply a computationally cheap post-processing procedure to predictions of any classifier trained to distinguish P from U. Instead of assuming the proportions to be identified, DEDPUL estimates them alongside with classifying U. Experiments show that DEDPUL outperforms the Risk Estimation approach when neural networks are used as classifiers. On some data sets, the performance can be improved even further if ensembles of trees are used as classifiers instead. At the same time, DEDPUL also outperforms the current state-of-the-art in Mixture Proportion Estimation, especially in the cases when P and U distributions are similar.