Effect of Image Captioning with Description on the Working Memory

Nithiya Shree Uppara, Troy McDaniel, Hemanth Venkateswara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Working memory plays an important role in human activities across academic, professional, and social settings. Working memory is defined as the memory extensively involved in goal-directed behaviors in which information must be retained and manipulated to ensure successful task execution. The aim of this research is to understand the effect of image captioning with image description on an individual’s working memory. A study was conducted with eight neutral images comprising situations relatable to daily life such that each image could have a positive or negative description associated with the outcome of the situation in the image. The study consisted of three rounds where the first and second round involved two parts and the third round consisted of one part. The image was captioned a total of five times across the entire study. The findings highlighted that only 25% of participants were able to recall the captions which they captioned for an image after a span of 9–15 days; when comparing the recall rate of the captions, 50% of participants were able to recall the image caption from the previous round in the present round; and out of the positive and negative description associated with the image, 65% of participants recalled the former description rather than the latter.

Original languageEnglish (US)
Title of host publicationHCI International 2021 - Late Breaking Papers
Subtitle of host publicationCognition, Inclusion, Learning, and Culture - 23rd HCI International Conference, HCII 2021, Proceedings
EditorsConstantine Stephanidis, Don Harris, Wen-Chin Li, Dylan D. Schmorrow, Cali M. Fidopiastis, Margherita Antona, Qin Gao, Jia Zhou, Panayiotis Zaphiris, Andri Ioannou, Andri Ioannou, Robert A. Sottilare, Jessica Schwarz, Matthias Rauterberg
PublisherSpringer Science and Business Media Deutschland GmbH
Pages107-120
Number of pages14
ISBN (Print)9783030903275
DOIs
StatePublished - 2021
Event23rd International Conference on Human-Computer Interaction, HCII 2021 - Virtual, Online
Duration: Jul 24 2021Jul 29 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13096 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference23rd International Conference on Human-Computer Interaction, HCII 2021
CityVirtual, Online
Period7/24/217/29/21

Keywords

  • Image captioning
  • Sentiment analysis
  • Working memory

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Effect of Image Captioning with Description on the Working Memory'. Together they form a unique fingerprint.

Cite this