To Örebro University

We position recent and emerging research in cognitive vision and perception addressing three key questions: (1) What kind of relational abstraction mechanisms are needed to perform (explainable) grounded inference --e.g., question-answering, qualitative generalisation, hypothetical reasoning-- relevant to embodied multimodal interaction? (2) How can such abstraction mechanisms be founded on behaviourally established cognitive human-factors emanating from naturalistic empirical observation? and (3) How to articulate behaviourally established abstraction mechanisms as formal declarative models suited for grounded knowledge representation and reasoning (KR) as part of large-scale hybrid AI and computational cognitive systems.

We contextualise (1--3) in the backdrop of recent results at the interface of AI/KR, and Spatial Cognition and Computation. Our main purpose is to emphasise the importance of behavioural research based foundations for next-generation, human-centred AI, e.g., as relevant to applications in Autonomous Vehicles, Social and Industrial Robots, and Visuo-Auditory Media.

We develop a human-centred, cognitive model of visuospatial complexity in everyday, naturalistic driving conditions. With a focus on visual perception, the model incorporates quantitative, structural, and dynamic attributes identifiable in the chosen context; the human-centred basis of the model lies in its behavioural evaluation with human subjects with respect to psychophysical measures pertaining to embodied visuoauditory attention. We report preliminary steps to apply the developed cognitive model of visuospatial complexity for human-factors guided dataset creation and benchmarking, and for its use as a semantic template for the (explainable) computational analysis of visuospatial complexity.

Parametric design is an established method in engineering and architecture facilitating the rapid generation and evaluation of a large number of configurations and shapes of complex physical structures according to constraints specified by the designer. However, the emphasis of parametric design systems, particularly in the context of architectural design of large-scale spaces, is on numerical aspects (e.g., maximising areas, specifying dimensions of walls) and does not address human-centred design criteria, for example, as developed from behavioural evidence-based studies. This paper aims at providing an evidence-based human-centred approach for defining design constraints for parametric modelling systems. We determine design rules that address wayfinding issues through behavioural multi-modal data analysis of a wayfinding case study in two healthcare environments of the Parkland hospital (Dallas). Our rules are related to the environmental factors of visibility and positioning of manifest cues along the navigation route. We implement our rules in FreeCAD, an open-source parametric system.

Parametric design systems serve as powerful assistive tools in the design process by providing a flexible approach for the generation of a vast number of design alternatives. However, contemporary parametric design systems focus primarily on low-level engineering and structural forms, without an explicit means to also take into account high-level, cognitively motivated people-centred design goals.

We present a precedent-based parametric design method that integrates people-centred design “precedents” rooted in empirical evidence directly within state of the art parametric design systems. As a use-case, we illustrate the general method in the context of an empirical study focusing on the multi-modal analysis of wayfinding behaviour in two large-scale healthcare environments. With this use-case, we demonstrate the manner in which: (1). a range of empirically established design precedents —e.g., pertaining to visibility and navigation— may be articulated as design constraints to be embedded directly within state of the art parametric design tools (e.g., Grasshopper); and (2). embedded design precedents lead to the (parametric) generation of a number of morphologies that satisfy people-centred design criteria (in this case, pertaining to wayfinding).

Our research presents an exemplar for the integration of cognitively motivated design goals with parametric design-space exploration methods. We posit that this opens-up a range of technological challenges for the engineering and development of next-generation computer aided architecture design systems.

A people-centred approach for designing large-scale built-up spaces necessitates systematic anticipation of user’s embodied visuo-locomotive experience from the viewpoint of human-environment interaction factors pertaining to aspects such as navigation, wayfinding, usability. In this context, we develop a behaviour-based visuo-locomotive complexity model that functions as a key correlate of cognitive performance vis-a-vis internal navigation in built-up spaces. We also demonstrate the model’s implementation and application as a parametric tool for the identification and manipulation of the architectural morphology along a navigation path as per the parameters of the proposed visuospatial complexity model. We present examples based on an empirical study in two healthcare buildings and showcase the manner in which a dynamic and interactive parametric (complexity) model can promote behaviour-based decision-making throughout the design process to maintain desired levels of visuospatial complexity as part of a navigation or wayfinding experience. 

Navigation performance in urban and large-scale built-up spaces (e.g. airports, train-stations, hospitals) depends on gradual environmental perception during locomotion, and spatial knowledge acquisition, update/integration at different times along a path. Rotational locomotion is regularly involved in everyday navigation; this, combined with the fact that people cannot perceive the whole of a large-scale setting at once often leads to incidents of cognitive loading and disorientation. Our research explores the mechanisms involved in rotational locomotion for human navigators, and the role of familiarity as well as the cost of cognitive load on orientation accuracy and spatial memory. We examine the impact of structural and featural cues on spatial knowledge updating in relation to egorotations from the viewpoint of behaviour-based design practice and evidencebased design interventions. The results are based on a case study in a train station, experimenting on rotational problems in navigation. Here we present preliminary results emphasizing the role of environmental cues in rotational location, outline possibilities for further study, and discuss implications for evidence-based design practice and cognitive design assistance technology development.

We study active human visuo-locomotive experience in everyday navigation from the viewpoints of environmental familiarity, embodied reorientation, and (sensorimotor) spatial update. Following a naturalistic, in situ, embodied multimodal behaviour analysis method, we conclude that familiar users rely on environmental cues as a navigation-aid and exhibit proactive decision-making, whereas unfamiliar users rely on manifest cues, are late in decision-making, and show no sign of sensorimotor spatial update. Qualitative analysis reveals that both groups are able to sketch-map their route and consider path integration: i.e., conscious spatial representation updating was possible but not preferred during active navigation. Overall, the experimental task did not trigger automatic or reflexlike spatial updating, as subjects preferred strategies involving memory of perceptual cues and available manifest cues instead of relying on motor simulation and continuous spatial update. Rooted in the behavioural outcomes, we also position applications in computational modelling of navigation within cognitive technologies for architectural design synthesis.

We propose an evidence based methodology for the systematic analysis and cognitive characterisation of multimodal interactions in naturalistic roadside situations such as driving, crossing a street etc. Founded on basic human modalities of embodied interaction, the proposed methodology utilises three key characteristics crucial to roadside situations, namely: explicit and implicit mode of interaction, formal and informal means of signalling, and levels of context-specific (visual) attention. Driven by the fine-grained interpretation and modelling of human behaviour in naturalistic settings, we present an application of the proposed model with examples from a work-in-progress dataset consisting of baseline multimodal interaction scenarios and variations built therefrom with a particular emphasis on joint attention and diversity of modalities employed. Our research aims to open up an interdisciplinary frontier for the human-centred design and evaluation of artificial cognitive technologies (e.g., autonomous vehicles, robotics) where embodied (multimodal) human interaction and normative compliance are of central significance.

How do the limits of high-level visual processing affect human performance in naturalistic, dynamic settings of (multimodal) interaction where observers can draw on experience to strategically adapt attention to familiar forms of complexity? In this backdrop, we investigate change detection in a driving context to study attentional allocation aimed at overcoming environmental complexity and temporal load. Results indicate that visuospatial complexity substantially increases change blindness but also that participants effectively respond to this load by increasing their focus on safety-relevant events, by adjusting their driving, and by avoiding non-productive forms of attentional elaboration, thereby also controlling “looked-but-failed-to-see” errors. Furthermore, analyses of gaze patterns reveal that drivers occasionally, but effectively, limit attentional monitoring and lingering for irrelevant changes. Overall, the experimental outcomes reveal how drivers exhibit effective attentional compensation in highly complex situations. Our findings uncover implications for driving education and development of driving skill-testing methods, as well as for human-factors guided development of AI-based driving assistance systems.