Semantic Description of Objects in Images Based on Prototype Theory

This research aims to build a model for the semantic description of objects based on features detected in images. We introduce a novel semantic description approach inspired by the Prototype Theory. Inspired by the human approach used for representing categories, we propose a novel model that encodes, stores and retrieves the central semantic meaning of the object’s category: the prototype. The proposed prototype-based description model create discriminative descriptor signatures and describe an object highlighting its most distinctive features within the category. To achieve this goal, a mathematical model is developed to stand for and construct the semantic prototypes of object’s categories using Convolutional Neural Networks (CNNs). Our global semantic descriptor builds low-dimensional and semantically interpretable signatures that encode the semantic information of the objects using the constructed semantic prototypes. In our experiments, using publicly available datasets, we show that: i) the mathematical model of the proposed semantic prototype is able to describe the category internal structure; ii) the proposed semantic distance metric can be understood as the object typicality score within a category; iii) the global semantic descriptor representation preserves the semantic information used by the CNN classification models, and objects typicality score; iv) our semantic descriptor encoding significantly outperforms state-of-the-art global descriptors in terms of cluster metrics.