Vol. 14 (11) pp. 247-255

ASYNCHRONOUS, SELF-SUPERVISED AND UNCERTAINTY-AWARE AUDIO-VISUAL FUSION FOR VECTOR SURVEILLANCE IN RESOURCE-CONSTRAINED ENVIRONMENTS

  • UFR of Science and Technology, Iba Der Thiam University, Thies, Senegal.
  • Faculty of Science and Technology, Cheikh Anta Diop University of Dakar, Dakar, Senegal.
  • Department of Mathematics, Institut Superieur d Informatique (ISI), Dakar, Senegal.
  • Ecole Superieure Polytechnique, Cheikh Anta Diop University of Dakar, Dakar, Senegal.
3 Downloads 18 Views

Abstract

Background: Automated surveillance of disease vectors is a major public health need in low-resource settings, where annotated data are scarce, low-cost sensors produce noisy signals and acoustic and visual streams are rarely synchronised. Supervised multimodal approaches assume abundant labels and precise temporal alignment, two conditions seldom met in the field. Methods: We propose a self-supervised audio-visual fusion framework that tolerates temporal asynchrony and accounts for sensor reliability. A Res Net-18 audio encoder processes Mel spectrograms of mosquito wingbeats and a Mobile NetV2 visual encoder processes RGB images. Both are trained without labels using an Info NCE contrastive objective in which every visual segment falling within a temporal window of an audio segment is treated as a positive match. An uncertainty module attached to each branch drives an adaptive fusion that down-weights the less reliable modality. Representations were evaluated by linear probing against unimodal and supervised multimodal baselines, and under acoustic noise, low light and audio-visual time shifts.

Keywords

How to Cite This Article

Serigne Modou Kara Samb et,al (2026); ASYNCHRONOUS, SELF-SUPERVISED AND UNCERTAINTY-AWARE AUDIO-VISUAL FUSION FOR VECTOR SURVEILLANCE IN RESOURCE-CONSTRAINED ENVIRONMENTS, International Journal of Advanced Research (IJAR), 14 (11), 247-255, ISSN 2320-5407.

Corresponding Author

Serigne Modou Kara Samb
UFR of Science and Technology, Iba Der Thiam University, Thies, Senegal.
Senegal

Publication Achievement

Publication card

Share your publication card

Post this card with your article title and abstract to your professional network.

🎉 Proud to contribute to scientific research! Share your publication with your colleagues and professional network.

Article Analytics