Application of a novel deep learning–based 3D videography workflow to bat flight

Abstract

AbstractStudying the detailed biomechanics of flying animals requires accurate three‐dimensional coordinates for key anatomical landmarks. Traditionally, this relies on manually digitizing animal videos, a labor‐intensive task that scales poorly with increasing framerates and numbers of cameras. Here, we present a workflow that combines deep learning–powered automatic digitization with filtering and correction of mislabeled points using quality metrics from deep learning and 3D reconstruction. We tested our workflow using a particularly challenging scenario: bat flight. First, we documented four bats flying steadily in a 2 m3 wind tunnel test section. Wing kinematic parameters resulting from manually digitizing bats with markers applied to anatomical landmarks were not significantly different from those resulting from applying our workflow to the same bats without markers for five out of six parameters. Second, we compared coordinates from manual digitization against those yielded via our workflow for bats flying freely in a 344 m3 enclosure. Average distance between coordinates from our workflow and those from manual digitization was less than a millimeter larger than the average human‐to‐human coordinate distance. The improved efficiency of our workflow has the potential to increase the scalability of studies on animal flight biomechanics.