Classification threshold and training data affect the quality and utility of focal species data processed with automated audio recognition software

Automated recognition is increasingly used to extract information about species vocalizations from audio recordings. During processing, recognizers calculate the probability of correct classification (“score”) for each acoustic signal assessed. Our goal was to investigate the implications of recognizer score for ecological research and monitoring. We trained four recognizers with clips of Common Nighthawk (Chordeiles minor) calls recorded at different distances: near, midrange, far, and mixed distances. We found distance explained 49% and 41% of the variation in score for the near and mixed-distance recognizers, but only 3% and 6% of the variation for the midrange and far recognizers. We calculated detection functions for each of the recognizers at various score thresholds and found that the detection function for the near and mixed-distance recognizers satisfied the assumptions of density estimation for most score thresholds, while the detection function for the midrange and far recognizers did not. The detection functions also showed that score threshold choice is a decision about sampling area, not just about the balance between recall and precision. Overall, we showed that training recognizers with ‘high-quality’ clips that were recorded at close range will improve the utility of the data without affecting how many true positives the recognizer detects.