2020, Tahmasebzadeh, Golsa, Hakimov, Sherzod, Müller-Budack, Eric, Ewerth, Ralph

Content-based information retrieval is based on the information contained in documents rather than using metadata such as keywords. Most information retrieval methods are either based on text or image. In this paper, we investigate the usefulness of multimodal features for cross-lingual news search in various domains: politics, health, environment, sport, and finance. To this end, we consider five feature types for image and text and compare the performance of the retrieval system using different combinations. Experimental results show that retrieval results can be improved when considering both visual and textual information. In addition, it is observed that among textual features entity overlap outperforms word embeddings, while geolocation embeddings achieve better performance among visual features in the retrieval task.

2021, Ben Abbès, Sarra, Hantach, Rim, Calvez, Philippe, Buscaldi, Davide, Dessì, Danilo, Dragoni, Mauro, Reforgiato Recupero, Diego, Sack, Harald

[no abstract available]

2021, Cheema, Gullal S., Hakimov, Sherzod, Müller-Budack, Eric, Ewerth, Ralph

Opinion and sentiment analysis is a vital task to characterize subjective information in social media posts. In this paper, we present a comprehensive experimental evaluation and comparison with six state-of-the-art methods, from which we have re-implemented one of them. In addition, we investigate different textual and visual feature embeddings that cover different aspects of the content, as well as the recently introduced multimodal CLIP embeddings. Experimental results are presented for two different publicly available benchmark datasets of tweets and corresponding images. In contrast to the evaluation methodology of previous work, we introduce a reproducible and fair evaluation scheme to make results comparable. Finally, we conduct an error analysis to outline the limitations of the methods and possibilities for the future work.