Abstract—Social media, as a major platform for
communication and information exchange, is a rich repository
of the opinions and sentiments of 2.3 billion users about a vast
spectrum of topics. In this sense, user text interactions are
widely used to sense the whys of certain social user’s demands
and cultural- driven interests. However, the knowledge
embedded in the 1.8 billion pictures which are uploaded daily in
public profiles has just started to be exploited. Following this
trend on visual-based social analysis, we present a novel
methodology based on neural networks to build a combined
image-and-text based personality trait model, trained with
images posted together with words found highly correlated to
specific personality traits. So, the key contribution in this work
is to explore whether OCEAN personality trait modeling can be
addressed based on images, here called MindPics, appearing
with certain tags with psychological insights. We found that
there is a correlation between posted images and the personality
estimated from their accompanying texts. Thus, the
experimental results are consistent with previous
cyber-psychology results based on texts, suggesting that images
could also be used for personality estimation: classification
results on some personality traits show that specific and
characteristic visual patterns emerge, in essence representing
abstract concepts. These results open new avenues of research
for further refining the proposed personality model under the
supervision of psychology experts, and to further substitute
current textual personality questionnaires by image-based ones.
Index Terms—Personality trait analysis, deep learning, visual
classification, OCEAN model, social networks.
P. Rodriguez, F. Xavier Roca, and J. Gonzàlez are with the Computer
Vision Center, Edifici O, Campus Universitat Autònoma de Barcelona,
08193 Bellaterra (Cerdanyola del Vallès), Barcelona, Catalonia Spain
(e-mails: prodriguez, xavir, poal@cvc.uab.es).
J. M. Gonfaus is with Visual Tagging Services S.L., Parc de Recerca
UAB, Edifici Eureka, Campus Universitat Autònoma de Barcelona, 08193
Bellaterra (Cerdanyola del Vallès), Barcelona, Catalonia Spain (e-mail:
pep.gonfaus@visual-tagging.com).
[PDF]
Cite: Pau Rodriguez, Jordi Gonzàlez, Josep M. Gonfaus, and F. Xavier Roca, "Integrating Vision and Language in Social Networks for
Identifying Visual Patterns of Personality Traits," International Journal of Social Science and Humanity vol. 9, no. 1, pp. 6-12, 2019.