To Örebro University

Cognitive media studies has developed as an area of research at the interface of disciplines as diverse as aesthetics, psychology, neuroscience, film theory, and cognitive science. In this context, the focus of this talk is on the foundational significance of artificial intelligence and visuo-spatial cognition and computation for the design of inte-grated analytical–empirical methods for the (multi-modal) analysis of human behaviour data vis-a-vis a range of digital visuo-auditory narrative media (e.g., narrative film). The presentation focusses on the methodological foundations and assistive technologies for systematic formalization and empirical analyses aimed at, for instance, the generation of evidence, establishing and characterizing correlates between principles for the synthesis of the moving image (e.g., from a cinematographic viewpoint), and its perceptual recipient effects and influence on observers.

In the backdrop a range of completed and ongoing experiments, we emphasize the core results on the semantic interpretation of human behaviour vis-a-vis narrative film and its visuo-auditory reception. We demonstrate the manner in which AI-based models for machine coding of narrative, and relational inference and learning serves as basis to externalize explicit and inferred knowledge about embodied visuo-auditory reception, e.g., using modalities such as diagrammatic representations, natural language, complex (dynamic) data visualizations.

Demonstration: The presentation will particularly showcase methods and tools developed to perform perceptual narrativisation or sensemaking with multi-modal, dynamic human-behaviour data (combining visuo-spatial imagery such as film/video, eye-tracking, head-tracking during a perception task) for a chosen set of experimental material based on existing films, as well as lab-developed experimental content.