SPIDER: Lightweight Speaker Identification on Resource-Constrained Embedded Devices

Voice-based Speaker Identification (SI) can be framed as the problem of Closed-Set Speaker Identification (CSSI), recognizing a speaker from a known set, or OSSI, additionally recognizing unknown speakers. Precise and accurate Open-Set Speaker Identification (SI) can enable a variety of applications, ranging from human presence detection to authentication. Existing SI solutions are typically driven by deep learning approaches, which involve computationally demanding models often running in cloud back-ends. Enabling local SI models running directly on resource-constrained embedded devices can enable new use cases while preserving speaker privacy. In this work, we fill this gap and present SPIDER, a lightweight, on-device CSSI and OSSI solution capable of running on the off-the-shelf Nordic nRF52840 and nRF5340 system-on-chip microcontrollers, which feature as little as 256 and 512 kB of RAM, respectively, and 1 MB of flash memory. SPIDER is 16x-67x smaller than currently-available SI models, and yet, the 16x smaller version achieves a comparable accuracy of 94.33 % for CSSI and 91.8% for OSSI. Our evaluation across multiple datasets confirms the viability of performing accurate SI directly on resource-constrained embedded devices using only low-cost microphones. To foster further research and development, we open source our implementation of SPIDER, empowering the community to explore new SI use cases where cloud connectivity or backhaul infrastructure is impractical or undesirable.