Food places classification in egocentric images using Siamese neural networks

Identifier:  imarina:6390078

Handle: https://hdl.handle.net/20.500.11797/imarina6390078

Authors:  Sarker, MMK; Banu, SF; Rashwan, HA; Abdel-Nasser, M; Singh, VK; Chambon, S; Radeva, P; Puig, D

Abstract:
© 2019 The authors and IOS Press. All rights reserved. Wearable cameras are become more popular in recent years for capturing the unscripted moments of the first-person that help to analyze the users lifestyle. In this work, we aim to recognize the places related to food in egocentric images during a day to identify the daily food patterns of the first-person. Thus, this system can assist to improve their eating behavior to protect users against food-related diseases. In this paper, we use Siamese Neural Networks to learn the similarity between images from corresponding inputs for one-shot food places classification. We tested our proposed method with'MiniEgoFoodPlaces' with 15 food related places. The proposed Siamese Neural Networks model with MobileNet achieved an overall classification accuracy of 76.74% and 77.53% on the validation and test sets of the “MiniEgoFoodPlaces” dataset, respectively outperforming with the base models, such as ResNet50, InceptionV3, and InceptionResNetV2.