Master’s Dissertation Defense, Samuel Sérvulo

We would like to congratulate Samuel Sérvulo for his new achievement, Master in Computer Science, at the UFMG.

Title: Active recognition of small objects using audiovisual fusion

Abstract

Robots routinely face the need to recognize common use objects, be it for domestic use, search and rescue tasks or surveillance systems. This ability fundamentally requires them to process sensory information and best represent it, in order to maximize its performance. This work presents an active perception approach to object recognition using both audio and visual stimuli, acquired by sensors mounted on a robot, which uses an articulated rod to  poke the object in order to actively generate audio signatures.

The object domain consists of a structured set of small objects, in which simple geometries and single-material compositions are adopted in order to make it easier to achieve a comprehension of the make-up of audio signatures. For each combination of geometry and material composition, an audiovisual signature is developed in a machine learning approach that implements sensor fusion.

Performance of classification is evaluated for the original signals and for decreasing signal-to-noise ratio of the audio signals, where two strategies for sensor fusion are comparatively evaluated: decision fusion  in a meta-learning manner, and feature fusion. Decision fusion is shown to perform best and improves over audio – or video-only classification, with accuracies of 98.6%, 96.2%, and 95.1%, respectively, enhancing recognition and providing stability over high interference scenarios. The audio descriptors introduced are ranked according to their discriminatory power.

Contributions of this work includes evaluation of techniques for representation of impulsive signals, a framework for audiovisual fusion and the release of the dataset used.

Keywords: Object recognition, sensor fusion, robot audition

 

Committee

Prof. Mario Fernando Montenegro Campos – Advisor (DCC – UFMG)
Profa. Izabela Lyon Freire – Co-Advisor (DCC – UFMG)
Prof. Hani Camille Yehia (DELT – UFMG)
Prof. Douglas Guimarães Macharet (DCC – UFMG)
Prof. Erickson Rangel do Nascimento (DCC – UFMG)