Examining pollinator attraction with object detection of insects using YOLO

Understanding the exact mechanisms of pollination performed by insects and which insect taxa are involved in pollination, are very important tasks of contemporary ecology. This is especially important regarding the current loss of biodiversity. This project explored how the deep learningaided tool YOLO (You Only Look Once) can help to identify and track insect pollinators in videos. For this, videos of flowers being visited by different insects were used to train a YOLO-model that can detect eight classes of insect taxa and track them throughout videos. Furthermore, this project explored a method of pre-processing long videos in a way that filters out video extracts with a low amount of pixel difference between consecutive frames. The goal of this was to find video extracts with a higher likelihood of containing insects, to later use them as training data. Both the pre-processing step and the model training were carried out using Marvin. As a result of the preprocessing, the selected video extracts did contain insects, but some were also falsely filtered out by this method. Thus, the attempt of pre-processing videos using the difference between frames was only in part successful and needs to be refined in further studies. The trained model, however, performed well at detecting insects, but the performance varied between the different classes. While bees and a wide variety of insects summarisingly labeled as “insect”were detected reliably, other classes like “hoverfly”were not reliably detected due to a lack of training data. Therefore, the model can already, for some of the classes, be used to study which insect pollinators visit flowers. This is, next to answering ecological questions, also interesting for applications in agriculture. There, detecting insects in camera images can be used to monitor both insect pollination and insect pests.