Till Örebro universitet

The long-standing goal of creating a comprehensive, multi-purpose knowledge resource, reminiscent of the 1984 Cyc project, still persists in AI. Despite the success of knowledge resources like WordNet, ConceptNet, Wolfram|Alpha and other commercial knowledge graphs, verifiable, general-purpose, widely available sources of knowledge remain a critical deficiency in AI infrastructure. Large language models struggle due to knowledge gaps; robotic planning lacks necessary world knowledge; and the detection of factually false information relies heavily on human expertise. What kind of knowledge resource is most needed in AI today? How can modern technology shape its development and evaluation? A recent AAAI workshop gathered over 50 researchers to explore these questions. This paper synthesizes our findings and outlines a community-driven vision for a new knowledge infrastructure. In addition to leveraging contemporary advances in knowledge representation and reasoning, one promising idea is to build an open engineering framework to exploit knowledge modules effectively within the context of practical applications. Such a framework should include sets of conventions and social structures that are adopted by contributors.

Situation awareness in driving involves detection of events and environmental changes. Failure in detection can be attributed to the density of these events in time, amongst other factors. In this research, we explore the effect of temporal proximity, and event duration in a change detection task during driving in VR. We replicate real-world interaction events in the streetscape and systematically manipulate temporal proximity among them. The results demonstrate that events occurring simultaneously deteriorate detection performance, while performance improves as the temporal gap increases. Moreover, attentional engagement to an event of 5-10 sec leads to compromised perception for the following event. We discuss the importance of naturalistic embodied perception studies for evaluating driving assistance and driver’s education.

We present recent and emerging advances in computational cognitive vision addressing artificial visual and spatial intelligence at the interface of (spatial) language, (spatial) logic and (spatial) cognition research. With a primary focus on explainable sensemaking of dynamic visuospatial imagery, we highlight the (systematic and modular) integration of methods from knowledge representation and reasoning, computer vision, spatial informatics, and computational cognitive modelling. A key emphasis here is on generalised (declarative) neurosymbolic reasoning & learning about space, motion, actions, and events relevant to embodied multimodal interaction under ecologically valid naturalistic settings in everyday life. Practically, this translates to general-purpose mechanisms for computational visual commonsense encompassing capabilities such as (neurosymbolic) semantic question-answering, relational spatio-temporal learning, visual abduction etc.

The presented work is motivated by and demonstrated in the applied backdrop of areas as diverse as autonomous driving, cognitive robotics, design of digital visuoauditory media, and behavioural visual perception research in cognitive psychology and neuroscience. More broadly, our emerging work is driven by an interdisciplinary research mindset addressing human-centred responsible AI through a methodological confluence of AI, Vision, Psychology, and (human-factors centred) Interaction Design.

Failure in change detection in the surrounding environment while driving is attributed among other things to the number of incidents and the density of them occurring along the task as this is directly related to an increase in cognitive load. Here, we investigate the role of time proximity between events on the detection performance during a naturalistic driving task in a virtual simulation. Participants performed a change detection task while driving, in which we systematically manipulated the time difference between changes and we analysed the effect on detection performance by the driver. Our research demonstrates that events occurring simultaneously deteriorate detection performance (in terms of detection rate, and detection time), while performance improves as the temporal gap increases. Moreover, the outcomes suggest that the duration of an event affects the detection of the following one, with better performance recorded for very short or very long duration events and worse for medium duration events between (5-10 sec). These outcomes are crucial for driving assistance and training, considering the detection of safety-critical events or efficient attentional disengagement on time from irrelevant targets.

We address computational cognitive vision and perception at the interface of language, logic, cognition, and artificial intelligence. The chapter presents general methods for the processing and semantic interpretation of dynamic visuospatial imagery with a particular emphasis on the ability to abstract, learn, and reason with cognitively rooted structured characterisations of commonsense knowledge pertaining to space and motion. The presented work constitutes a systematic model and methodology integrating diverse, multi-faceted AI methods pertaining Knowledge Representation and Reasoning, Computer Vision, and Machine Learning towards realising practical, human-centred artificial visual intelligence.

In everyday activities where continuous visual awareness is critical such as driving, several cognitive processes pertaining to visual attention are of the essence, for instance, change detection, anticipation, monitoring, etc. Research suggests that environmental load and task difficulty contribute to failures in visual perception that can be essential for detecting and reacting to safety-critical incidents. However, it is unclear how gaze patterns and attentional strategies are compromised because of environmental complexity in naturalistic driving. In a change detection task during everyday simulated driving, we investigate inattention blindness in relation to environmental complexity and the kind of interaction incidents drivers address. We systematically analyse and evaluate safety-critical situations from real-world driving videos and replicate a number of them in a virtual driving experience. Participants (N= 80) aged 23-45 years old, drove along three levels of environmental complexity (low-medium-high) and various incidents of interaction with roadside users (e.g., pedestrians, cyclists, pedestrians in a wheelchair), categorized as safety critical or not. Participants detected changes in the behaviour of road users and in object properties. We collect multimodal data including eye-tracking, egocentric view videos, movement trace, head movements, driving behaviour, and detection button presses. Results suggest that gaze behaviour (number and duration of fixations, 1st fixation on AOI) is affected negatively by an increase in environmental complexity, but the effect is moderate for safety-critical incidents. Moreover, anticipatory and monitoring attention was crucial for detecting critical changes in behaviour and reacting on time. However, in highly complex environments participants effectively limit attentional monitoring and lingering for non-critical changes and they also controlled “look-but-fail-to-see errors", especially while addressing a safety-related event. We conclude that drivers change attentional strategies, avoiding non-productive forms of attentional elaboration (anticipatory and monitoring) and efficiently disengaging from targets when the task difficulty is high. We discuss the implications for driving education and research driven development of autonomous driving. 

How do the limits of high-level visual processing affect human performance in naturalistic, dynamic settings of (multimodal) interaction where observers can draw on experience to strategically adapt attention to familiar forms of complexity? In this backdrop, we investigate change detection in a driving context to study attentional allocation aimed at overcoming environmental complexity and temporal load. Results indicate that visuospatial complexity substantially increases change blindness but also that participants effectively respond to this load by increasing their focus on safety-relevant events, by adjusting their driving, and by avoiding non-productive forms of attentional elaboration, thereby also controlling “looked-but-failed-to-see” errors. Furthermore, analyses of gaze patterns reveal that drivers occasionally, but effectively, limit attentional monitoring and lingering for irrelevant changes. Overall, the experimental outcomes reveal how drivers exhibit effective attentional compensation in highly complex situations. Our findings uncover implications for driving education and development of driving skill-testing methods, as well as for human-factors guided development of AI-based driving assistance systems.

This article investigates the role that kinematic features play in human action similarity judgments. The results of three experiments with human participants are compared with the computational model that solves the same task. The chosen model has its roots in developmental robotics and performs action classification based on learned kinematic primitives. The comparative experimental results show that both model and human participants can reliably identify whether two actions are the same or not. Specifically, most of the given actions could be similarity judged based on very limited information from a single feature domain (velocity or spatial). Both velocity and spatial features were however necessary to reach a level of human performance on evaluated actions. The experimental results also show that human performance on an action identification task indicated that they clearly relied on kinematic information rather than on action semantics. The results show that both the model and human performance are highly accurate in an action similarity task based on kinematic-level features, which can provide an essential basis for classifying human actions.

The purpose of this article is to highlight the critical importance of language generation today. In particular, language generation is explored from the following three aspects: multi-modality, multilinguality, and multitask, which all of them play crucial role for Natural Language Generation (NLG) community. We present the activities conducted within the Multi3Generation COST Action (CA18231), as well as current trends and future perspectives for multitask, multilingual and multimodal language generation.