Image Retrieval with Missing Regions by Reconstructing and Concatenating Content and Semantic Features

In recent years, the performance of deep neural networks in improving the image retrieval process has been remarkable. Utilizing deep neural networks; however, leads to poor results in retrieving images with missing regions. The operators’ dysfunctions, who consider the relationship between the image pixels, statistically extract incomplete information from an image, which in turn reduces the number of image features and or leads to features' inaccurate identification. An attempt has been made to eliminate the problem of missing image information through image inpainting techniques; therefore, a content-based image retrieval method is proposed for images with missing regions. In this method, through image inpainting the crucial missing information is reconstructed. The image dataset is being queried to find similar samples. For this purpose, a two-stage inpainting framework based on encoder-decoder is used in the image retrieval system. Also, the features of each image are extracted from the integration and concatenating of content and semantic features. Through using handcraft features such as color and texture image content information is extracted from the Resnet-۵۰ deep neural network. Finally, similar images are retrieved based on the minimum Euclidean distance. The performance of the image retrieval model with missing regions is evaluated with the average precision criterion on the Paris ۶K datasets. The best retrieval results are ۶۰.۱۱%, ۵۰.۱۴%, and ۴۲.۴۳% for retrieving the top one, five, and ten samples after reconstructing the image with the most missing regions with a destruction frequency of ۶ Hz, respectively.

کلیدواژه ها:

image inpainting ، content features ، semantic features ، ResNet-۵۰ ، deep neural network