Neurosymbolic Visual Commonsense: On Integrated Reasoning and Learning about Space and Motion in Embodied Multimodal Interaction
2024 (English)In: Proceedings of the 3rd International Workshop on Spatio-Temporal Reasoning and Learning (STRL 2024) co-located with the 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024), Jeju island, South Korea, August 5, 2024 / [ed] Parisa Kordjamshidi; Jae Hee Lee; Mehul Bhatt; Michael Sioutis; Zhiguo Long, Technical University of Aachen , 2024, Vol. 3827Conference paper, Published paper (Refereed)
Abstract [en]
We present recent and emerging advances in computational cognitive vision addressing artificial visual and spatial intelligence at the interface of (spatial) language, (spatial) logic and (spatial) cognition research. With a primary focus on explainable sensemaking of dynamic visuospatial imagery, we highlight the (systematic and modular) integration of methods from knowledge representation and reasoning, computer vision, spatial informatics, and computational cognitive modelling. A key emphasis here is on generalised (declarative) neurosymbolic reasoning & learning about space, motion, actions, and events relevant to embodied multimodal interaction under ecologically valid naturalistic settings in everyday life. Practically, this translates to general-purpose mechanisms for computational visual commonsense encompassing capabilities such as (neurosymbolic) semantic question-answering, relational spatio-temporal learning, visual abduction etc.
The presented work is motivated by and demonstrated in the applied backdrop of areas as diverse as autonomous driving, cognitive robotics, design of digital visuoauditory media, and behavioural visual perception research in cognitive psychology and neuroscience. More broadly, our emerging work is driven by an interdisciplinary research mindset addressing human-centred responsible AI through a methodological confluence of AI, Vision, Psychology, and (human-factors centred) Interaction Design.
Place, publisher, year, edition, pages
Technical University of Aachen , 2024. Vol. 3827
Series
CEUR Workshop Proceedings, E-ISSN 1613-0073 ; 3827
Keywords [en]
Cognitive vision, Knowlede representation and reasoning (KR), Machine Learning, Integration of reasoning & learning, Commonsense reasoning, Declarative spatial reasoning, Relational Learning, Computational cognitive modelling, Human-Centred AI, Responsible AI
National Category
Computer Sciences Human Computer Interaction Computer graphics and computer vision
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:oru:diva-117535Scopus ID: 2-s2.0-85210239908OAI: oai:DiVA.org:oru-117535DiVA, id: diva2:1917570
Conference
3rd International Workshop on Spatio-Temporal Reasoning and Learning (STRL 2024) co-located with the 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024), Jeju island, South Korea, August 5, 2024
Funder
Swedish Foundation for Strategic ResearchSwedish Research Council2024-12-032024-12-032025-02-01Bibliographically approved