To Örebro University

We develop a cognitive evaluation schema for analysing the diversity and nuances of visuospatial complexity and multimodal interactions encountered in naturalistic everyday driving conditions. The proposed schema is based on a finegrained empirical analysis of real-world everyday driving situations involving stakeholders such as drivers, pedestrians, cyclists. Our method involves a semantic analysis of egocentric POVs of stakeholders, focusing on the sequence and duration of events (e.g. velocity or direction change), the combination of modalities used (e.g., gestures, gaze, head-movements), audio, quantity and variety of moving and static objects in the scene e.g., (cars, signs), behavioural metrics from the stakeholders (e.g. gaze allocation, steering), etc. The proposed cognitive evaluation schema consists of three key aspects: (1) Scene characteristics consisting of a combination of quantitative (e.g., clutter, size), structural (e.g. symmetry), and dynamic attributes (e.g. motion), (2) Multimodal interactions consisting of the mode and method of interaction, as well as the level of joint attention achieved, (3) Recipient effects characterising subject’s behaviour and driving performance through physiological measurements (e.g. eye-tracking, head rotation) in a series of virtual reality (VR) environments replicating a number of naturalistic scenarios (and variations therefrom). Driven by behavioural methods in visual perception, we aim to open-up an interdisciplinary frontier for the human-centred design, evaluation / testing of artificial vision modules within AI-technologies for autonomous driving, cognitive robotics etc., where embodied, multimodal human-machine interaction is of the essence. We also demonstrate the practical application of basic visual perception research towards technology-centric settings of social significance.