Title:Brief Communication: A Re-Examination of the Eye Movement Data used by Hooge et al (2018): "Is human classification by experienced untrained observers a gold standard in fixation detection?"

Abstract:Hooge et al. 2018 asked the question: "Is human classification by experienced untrained observers a gold standard in fixation detection?" They conclude the answer is no. I have had a close look at their data and their report, and I find that both the data itself, the method of presentation of the data to the human raters, and the analysis of agreement statistics are problematic. I think that data used to address this important question should be very high quality, and the method of presentation of the data should be optimized. Alternatively, the title should be changed to: "Is human classification by experienced untrained observers a gold standard in fixation detection when data quality is very poor?" Numerous examples of various types of low quality data are presented herein. In addition, it appears that, by including missing data in the data submitted for assessment of agreement statistics, Hooge et al (2018) have made a serious mistake. Also, the treatment of some inter-fixation intervals as saccades that might have been identified by human experts is completely unjustified. The results regarding these saccades are without merit or meaning.