A study on a mixed stopping strategy for total recall tasks

How do we calculate how many relevant documents are in a collection? In this abstract, we discuss our line of research about total recall systems such as interactive system for systematic reviews based on an active learning framework [4–6]. In particular, we will present 1) the problem in mathematical terms, and 2) the experiments of an interactive system that continuously monitors the costs of reviewing additional documents and suggests the user whether to continue or not in the search based on the available remaining resources. We will discuss the results of this system on the ongoing CLEF 2019 eHealth task.