To Örebro University

This paper presents the Multitask, Multilingual, Multimodal Language Generation COST Action – Multi3Generatio(CA18231), an interdisciplinary networof research groups working on different aspects of language generation. This "meta-paper" will serve as reference for citationof the Action in future publications. It presents the objectives, challenges and a the links for the achieved outcomes.

We present computational visuo-spatial representation and reasoning from the viewpoint of the research areas of artificial intelligence, spatial cognition and computation, and human-computer interaction. The particular focus is on demonstrating recent advances in the theory and practice of spatial reasoning, and its significance and potential as a foundational AI method for (intelligent) computational cinematography & editing systems.

High-level semantic interpretation of (dynamic) visual imagery calls for general and systematic methods integrating techniques in knowledge representation and computer vision. Towards this, we position "deep semantics", denoting the existence of declarative models –e.g., pertaining "space and motion"– and corresponding formalisation and methods supporting (domain-independent) explainability capabilities such as semantic question-answering, relational (and relationally-driven) visuospatial learning, and (non-monotonic) visuospatial abduction. Rooted in recent work, we summarise and report the status quo on deep visuospatial semantics —and our approach to neurosymbolic integration and explainable visuo-spatial computing in that context— with developed methods and tools in diverse settings such as behavioural research in psychology, art & social sciences, and autonomous driving.

One of the well-known digraph substitution cipher is the Playfair Cipher. It secures information mathematically by mangling message with key. The privacy of intended sender and receiver information is protected from eavesdropper. However the original 5 x 5 Playfair Cipher can support only 25 uppercase alphabets. Here we have implemented a new technique which includes a rectangular matrix having 10 columns and 9 rows and six iteration steps for encryption as well as decryption purpose. This 10 x 9 rectangular matrix includes all alphanumeric characters and some special characters. Cryptanalysis is done to show that the modified cipher is a strong one. Finally we have implemented this concept with the help of MATLAB.

This chapter explores and discusses how models of debriefing can support interprofessional learning in simulation-based education activities. The role of video as a tool for feedback and reflection is described and the ways video can be integrated in the learning environment to optimize opportunities for learning.

This paper presents a brief overview of work-in-progress for a study on correlations between visual and haptic spatial attention in a multimodal single-user application comparing different modalities. The aim is to gain insight into how auditory and haptic versus visual representations of temporal events may affect task performance and spatial attention. For this purpose, a 3D application involving one haptic model and two different sound models for interactive sonification are developed.

Advancements in telemedicine have been helpful for frequent monitoring of patients with Parkinson’s disease (PD) from remote locations and assessment of their individual symptoms and treatment-related complications. These data can be useful for helping clinicians to interpret symptom states and individually tailor the treatments by visualizing the physiological information collected by sensor-based systems. In this paper we present a visualization metaphor that represents symptom information of PD patients during tapping tests performed with a smartphone. The metaphor has been developed and evaluated with a clinician. It enabled the clinician to observe fine motor impairments and identify motor fluctuations regarding several movement aspects of patients that perform the tests from their homes.

Detecting activities of daily living (ADL) allows for rich inference about user behavior, which can be of use in the care of for example, elderly people, chronic diseases, and psychological conditions. This paper proposes a domain knowledge-based solution for detecting 24 different ADLs in the UJA dataset. The solution is inspired by a Finite State Machine and performs activity recognition unobtrusively using only binary sensors. Each day in the dataset is segmented into: morning, day, evening in order to facilitate the inference from the sensors. The model performs the ADL recognition in two steps. The first step is to detect the sequence of activities in a given event stream of binary sensors, and the second step is to assign a starting and ending times for each of detected activities. Our proposed model achieved an accuracy of 81.3% using only a very small amount of operations, making it an interesting approach for resource-constrained devices that are common in smart environments. It should be noted, however, that the model can end up in faulty states which could cause a series of mis-classifications before the model is returned to the true state.

Field-Programmable Gate Arrays (FPGA) are hardware components that hold several desirable properties for wearable and Internet of Things (IoT) devices. They offer hardware implementations of algorithms using parallel computing, which can be used to increase battery life or achieve short response-times. Further, they are re-programmable and can be made small, power-efficient and inexpensive. In this paper we propose a classifier targeted specifically for implementation on FPGAs by using principles from hyperdimensional computing and cellular automata. The proposed algorithm is shown to perform on par with Naive Bayes for two benchmark datasets while also being robust to noise. It is also synthesized to a commercially available off-the-shelf FPGA reaching over 57.1 million classifications per second for a 3-class problem using 40 input features of 8 bits each. The results in this paper show that the proposed classifier could be a viable option for applications demanding low power-consumption, fast real-time responses, or a robustness against post-training noise.

During recent years, many schools have started to implement information and communication technologies (ICTs)-based learning devices (such as laptops, tablets, mobile phones, and active boards) in the classroom settings in order to increase learning outcomes. The aim of this study is to find which activities and outcomes are evident in the usages of computing devices and how those devices aid elementary-level students in their learning activities. The study includes five overt participant observations at some schools in a Swedish municipality including unstructured interviews and explains the findings through activity theory and the Alberta Education Framework for Student Learning . The major activities found were dealing with the (1) educational application assignments, (2) storytelling, (3) report writing and (4) practical interaction assignments. We concluded that ICTs aid students in becoming more concentrated, focus driven, engaged and amused, thus learning becomes more interesting.

Evolutionary knowledge is important to understand and address contemporary challenges such as loss of biodiversity, climate change and antibiotic resistance. An important aspect that is considered to be a threshold concept in teaching and learning about evolution is the time it involves. The history of evolution comprises several scales of magnitude, some of which are far from direct human experience and therefore difficult to understand. One way of addressing this issue is to use dynamic visualizations that represent time, for example, to facilitate teaching and learning about evolution.

This thesis investigates how students’ comprehension of evolution and evolutionary time can be facilitated by visualizations in educational settings. Two different dynamic visualizations were investigated. In paper I different temporal versions of a spatio-temporal animation depicting hominin evolution were explored. The temporal information was expressed as one or several timelines along which an animated cursor moved, indicating the rate of time. Two variables, the number of timelines with different scales, and the mode of the default animated time rate (either constant throughout the animation or decreasing as the animation progressed), were combined to give four different time representations. The temporal aspects investigated were undergraduate students' ability to find events at specific times, comprehend order, comprehend concurrent events, comprehend the length of time intervals, and their ability to compare the lengths of time intervals.

In paper II, perceptions and comprehension of temporal aspects in an interactive, multi-touch tabletop application, DeepTree, were investigated. This application depicts the tree of life. The focus was on the interactive aspects, especially how the zooming feature was perceived, but also on any misinterpretations associated with the interaction. The same temporal aspects listed for paper I were also implicitly investigated.

The findings indicate that handling the problem of large differences in scale by altering the rate of time in the visualization can facilitate perception of certain temporal aspects while, at the same time, can hinder a correct comprehension of other temporal aspects. Findings concerning DeepTree indicate that the level of interactions varies among users, and that the zooming feature is perceived in two ways, either as a movement in time or as a movement in the metaphorical tree. Several misinterpretations were observed, for example the assumption that the zooming time in the tree corresponds to real time, that there is an implicit coherent timeline along the y-axis of the tree, and that more nodes along a branch corresponds to a longer time.

The research reported in this thesis supports the claim that careful choice, and informed use of visualizations matters, and that different visualizations are best suited for different educational purposes