To Örebro University

oru.seÖrebro University Publications
Change search
Link to record
Permanent link

Direct link
Alirezaie, Marjan
Publications (10 of 40) Show all publications
Abraham, S. S., Alirezaie, M. & De Raedt, L. (2024). CLEVR-POC: Reasoning-Intensive Visual Question Answering in Partially Observable Environments. In: 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings: . Paper presented at Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024, Torino, Italy, May 20-25, 2024 (pp. 3297-3313). European Language Resources Association (ELRA)
Open this publication in new window or tab >>CLEVR-POC: Reasoning-Intensive Visual Question Answering in Partially Observable Environments
2024 (English)In: 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, European Language Resources Association (ELRA) , 2024, p. 3297-3313Conference paper, Published paper (Refereed)
Abstract [en]

The integration of learning and reasoning is high on the research agenda in AI. Nevertheless, there is only a little attention to use existing background knowledge for reasoning about partially observed scenes to answer questions about the scene. Yet, we as humans use such knowledge frequently to infer plausible answers to visual questions (by eliminating all inconsistent ones). Such knowledge often comes in the form of constraints about objects and it tends to be highly domain or environment-specific. We contribute a novel benchmark called CLEVR-POC for reasoning-intensive visual question answering (VQA) in partially observable environments under constraints. In CLEVR-POC, knowledge in the form of logical constraints needs to be leveraged to generate plausible answers to questions about a hidden object in a given partial scene. For instance, if one has the knowledge that all cups are colored either red, green or blue and that there is only one green cup, it becomes possible to deduce the color of an occluded cup as either red or blue, provided that all other cups, including the green one, are observed. Through experiments, we observe that the low performance of pre-trained vision language models like CLIP (≈ 22%) and a large language model (LLM) like GPT-4 (≈ 46%) on CLEVR-POC ascertains the necessity for frameworks that can handle reasoning-intensive tasks where environment-specific background knowledge is available and crucial. Furthermore, our demonstration illustrates that a neuro-symbolic model, which integrates an LLM like GPT-4 with a visual perception network and a formal logical reasoner, exhibits exceptional performance on CLEVR-POC.

Place, publisher, year, edition, pages
European Language Resources Association (ELRA), 2024
Keywords
LLM and Reasoning, logical constraints, partial observability, visual question answering, Computational linguistics, Visual languages, Background knowledge, Language model, Large language model and reasoning, Partially observable environments, Performance, Question Answering, Research agenda, Knowledge management
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-118582 (URN)2-s2.0-85195916891 (Scopus ID)9782493814104 (ISBN)
Conference
Joint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024, Torino, Italy, May 20-25, 2024
Available from: 2025-01-16 Created: 2025-01-16 Last updated: 2025-01-16Bibliographically approved
Lee, J., Sioutis, M., Ahrens, K., Alirezaie, M., Kerzel, M. & Wermter, S. (2023). Neuro-Symbolic Spatio-Temporal Reasoning. In: Pascal Hitzler; Md Kamruzzaman Sarker; Aaron Eberhart (Ed.), Compendium of Neurosymbolic Artificial Intelligence: (pp. 410-429). IOS Press, 369
Open this publication in new window or tab >>Neuro-Symbolic Spatio-Temporal Reasoning
Show others...
2023 (English)In: Compendium of Neurosymbolic Artificial Intelligence / [ed] Pascal Hitzler; Md Kamruzzaman Sarker; Aaron Eberhart, IOS Press, 2023, Vol. 369, p. 410-429Chapter in book (Other academic)
Abstract [en]

Knowledge about space and time is necessary to solve problems in the physical world. Spatio-Temporal knowledge, however, is required beyond interacting with the physical world, and is also often transferred to the abstract world of concepts through analogies and metaphors. As spatial and temporal reasoning is ubiquitous, different attempts have been made to integrate this into AI systems. In the area of knowledge representation, spatial and temporal reasoning has been largely limited to modeling objects and relations and developing reasoning methods to verify statements about objects and relations. On the other hand, neural network researchers have tried to teach models to learn spatial relations from data with limited reasoning capabilities. Bridging the gap between these two approaches in a mutually beneficial way could allow us to tackle many complex real-world problems. In this chapter, we view this integration problem from the perspective of Neuro-Symbolic AI. Specifically, we propose a synergy between logical reasoning and machine learning that will be grounded on spatial and temporal knowledge. A (symbolic) spatio-Temporal knowledge base and a base of possibly grounded examples could provide a dependable causal seed upon which machine learning models could generalize. Describing some successful applications, remaining challenges, and evaluation datasets pertaining to this direction is the main topic of this contribution.

Place, publisher, year, edition, pages
IOS Press, 2023
Series
Frontiers in Artificial Intelligence and Applications, ISSN 0922-6389, E-ISSN 1879-8314 ; 369
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-110447 (URN)10.3233/FAIA230151 (DOI)001106215800020 ()2-s2.0-85171791809 (Scopus ID)9781643684062 (ISBN)9781643684079 (ISBN)
Available from: 2023-12-20 Created: 2023-12-20 Last updated: 2025-01-20Bibliographically approved
Sioutis, M., Long, Z., Lee, J., Bhatt, M., Toumpa, A., Finzel, B., . . . Bouraoui, Z. (2023). Preface. Paper presented at 2nd International Workshop on Spatio-Temporal Reasoning and Learning (STRL 2023), Macao, China, August 21, 2023. CEUR Workshop Proceedings, 3475
Open this publication in new window or tab >>Preface
Show others...
2023 (English)In: CEUR Workshop Proceedings, E-ISSN 1613-0073, Vol. 3475Article in journal, Editorial material (Other academic) Published
Place, publisher, year, edition, pages
Technical University of Aachen, 2023
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-110217 (URN)2-s2.0-85173489517 (Scopus ID)
Conference
2nd International Workshop on Spatio-Temporal Reasoning and Learning (STRL 2023), Macao, China, August 21, 2023
Available from: 2023-12-14 Created: 2023-12-14 Last updated: 2023-12-14Bibliographically approved
Strannegård, C., Hammer, P. & Alirezaie, M. (2023). Preface. In: Patrick Hammer; Marjan Alirezaie; Claes Strannegård (Ed.), Artificial General Intelligence: 16th International Conference, AGI 2023, Stockholm, Sweden, June 16–19, 2023, Proceedings (pp. v-vi). Springer, 13921 LNCS
Open this publication in new window or tab >>Preface
2023 (English)In: Artificial General Intelligence: 16th International Conference, AGI 2023, Stockholm, Sweden, June 16–19, 2023, Proceedings / [ed] Patrick Hammer; Marjan Alirezaie; Claes Strannegård, Springer, 2023, Vol. 13921 LNCS, p. v-viChapter in book (Other academic)
Place, publisher, year, edition, pages
Springer, 2023
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 13921
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-110221 (URN)2-s2.0-85163345602 (Scopus ID)9783031334689 (ISBN)9783031334696 (ISBN)
Available from: 2023-12-14 Created: 2023-12-14 Last updated: 2023-12-14Bibliographically approved
Chimamiwa, G., Giaretta, A., Alirezaie, M., Pecora, F. & Loutfi, A. (2022). Are Smart Homes Adequate for Older Adults with Dementia?. Sensors, 22(11), Article ID 4254.
Open this publication in new window or tab >>Are Smart Homes Adequate for Older Adults with Dementia?
Show others...
2022 (English)In: Sensors, E-ISSN 1424-8220, Vol. 22, no 11, article id 4254Article, review/survey (Refereed) Published
Abstract [en]

Smart home technologies can enable older adults, including those with dementia, to live more independently in their homes for a longer time. Activity recognition, in combination with anomaly detection, has shown the potential to recognise users' daily activities and detect deviations. However, activity recognition and anomaly detection are not sufficient, as they lack the capacity to capture the progression of patients' habits across the different stages of dementia. To achieve this, smart homes should be enabled to recognise patients' habits and changes in habits, including the loss of some habits. In this study, we first present an overview of the stages that characterise dementia, alongside real-world personas that depict users' behaviours at each stage. Then, we survey the state of the art on activity recognition in smart homes for older adults with dementia, including the literature that combines activity recognition and anomaly detection. We categorise the literature based on goals, stages of dementia, and targeted users. Finally, we justify the necessity for habit recognition in smart homes for older adults with dementia, and we discuss the research challenges related to its implementation.

Place, publisher, year, edition, pages
MDPI, 2022
Keywords
Activity recognition, ageing, dementia, habit recognition, smart homes
National Category
Gerontology, specialising in Medical and Health Sciences Occupational Therapy
Identifiers
urn:nbn:se:oru:diva-99532 (URN)10.3390/s22114254 (DOI)000809104700001 ()35684874 (PubMedID)2-s2.0-85131268514 (Scopus ID)
Funder
EU, Horizon 2020, 754285
Available from: 2022-06-15 Created: 2022-06-15 Last updated: 2024-03-27Bibliographically approved
Fouladgar, N., Alirezaie, M. & Främling, K. (2022). CN-waterfall: a deep convolutional neural network for multimodal physiological affect detection. Neural Computing & Applications, 34(3), 2157-2176
Open this publication in new window or tab >>CN-waterfall: a deep convolutional neural network for multimodal physiological affect detection
2022 (English)In: Neural Computing & Applications, ISSN 0941-0643, E-ISSN 1433-3058, Vol. 34, no 3, p. 2157-2176Article in journal (Refereed) Published
Abstract [en]

Affective computing solutions, in the literature, mainly rely on machine learning methods designed to accurately detect human affective states. Nevertheless, many of the proposed methods are based on handcrafted features, requiring sufficient expert knowledge in the realm of signal processing. With the advent of deep learning methods, attention has turned toward reduced feature engineering and more end-to-end machine learning. However, most of the proposed models rely on late fusion in a multimodal context. Meanwhile, addressing interrelations between modalities for intermediate-level data representation has been largely neglected. In this paper, we propose a novel deep convolutional neural network, called CN-Waterfall, consisting of two modules: Base and General. While the Base module focuses on the low-level representation of data from each single modality, the General module provides further information, indicating relations between modalities in the intermediate- and high-level data representations. The latter module has been designed based on theoretically grounded concepts in the Explainable AI (XAI) domain, consisting of four different fusions. These fusions are mainly tailored to correlation- and non-correlation-based modalities. To validate our model, we conduct an exhaustive experiment on WESAD and MAHNOB-HCI, two publicly and academically available datasets in the context of multimodal affective computing. We demonstrate that our proposed model significantly improves the performance of physiological-based multimodal affect detection.

Place, publisher, year, edition, pages
Springer, 2022
Keywords
Multimodal affect detection, Deep convolutional neural network, Physiological-based sensors, Data fusion
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-94919 (URN)10.1007/s00521-021-06516-3 (DOI)000698886400003 ()2-s2.0-85115620535 (Scopus ID)
Funder
Knut and Alice Wallenberg Foundation
Note

Funding agency:

Umeå University

Available from: 2021-10-13 Created: 2021-10-13 Last updated: 2022-09-12Bibliographically approved
Sam Abraham, S. & Alirezaie, M. (2022). Compositional Generalization and Neuro-Symbolic Architectures. In: AAAI - Combining Learning and Reasoning: Programming Languages, Formalisms, and Representations, CLeaR 2022. Paper presented at 36th AAAI conference on artificial intelligence (AAAI-2022), Vancouver, BC, Canada, February 22 - March 1, 2022.
Open this publication in new window or tab >>Compositional Generalization and Neuro-Symbolic Architectures
2022 (English)In: AAAI - Combining Learning and Reasoning: Programming Languages, Formalisms, and Representations, CLeaR 2022, 2022Conference paper, Published paper (Refereed)
Abstract [en]

Compositional generalization is the ability to understand novel combinations of known concepts. Although it is considered as an innate skill for humans, recent studies have shown that neural networks lack this characteristic. In this paper, we focus on compositional generalization with respect to the two specific tasks of word problem solving and visual relation recognition and propose a neuro-symbolic solution, using DeepProbLog, that addresses the problem of compositionality in state-of-the-art neural systems for these tasks.

Keywords
Compositional generalization, Neuro-symbolic AI, DeepProbLog, Word Problem Solving, Visual Relation Recognition
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:oru:diva-96878 (URN)
Conference
36th AAAI conference on artificial intelligence (AAAI-2022), Vancouver, BC, Canada, February 22 - March 1, 2022
Available from: 2022-01-28 Created: 2022-01-28 Last updated: 2022-03-31Bibliographically approved
Fouladgar, N., Alirezaie, M. & Främling, K. (2022). Metrics and Evaluations of Time Series Explanations: An Application in Affect Computing. IEEE Access, 10, 23995-24009
Open this publication in new window or tab >>Metrics and Evaluations of Time Series Explanations: An Application in Affect Computing
2022 (English)In: IEEE Access, E-ISSN 2169-3536, Vol. 10, p. 23995-24009Article in journal (Refereed) Published
Abstract [en]

Explainable artificial intelligence (XAI) has shed light on enormous applications by clarifying why neural models make specific decisions. However, it remains challenging to measure how sensitive XAI solutions are to the explanations of neural models. Although different evaluation metrics have been proposed to measure sensitivity, the main focus has been on the visual and textual data. There is insufficient attention devoted to the sensitivity metrics tailored for time series data. In this paper, we formulate several metrics, including max short-term sensitivity (MSS), max long-term sensitivity (MLS), average short-term sensitivity (ASS) and average long-term sensitivity (ALS), that target the sensitivity of XAI models with respect to the generated and real time series. Our hypothesis is that for close series with the same labels, we obtain similar explanations. We evaluate three XAI models, LIME, integrated gradient (IG), and SmoothGrad (SG), on CN-Waterfall, a deep convolutional network. This network is a highly accurate time series classifier in affect computing. Our experiments rely on data-, metric- and XAI hyperparameter- related settings on the WESAD and MAHNOB-HCI datasets. The results reveal that (i) IG and LIME provide a lower sensitivity scale than SG in all the metrics and settings, potentially due to the lower scale of important scores generated by IG and LIME, (ii) the XAI models show higher sensitivities for a smaller window of data, (iii) the sensitivities of XAI models fluctuate when the network parameters and data properties change, and (iv) the XAI models provide unstable sensitivities under different settings of hyperparameters.

Place, publisher, year, edition, pages
IEEE, 2022
Keywords
Measurement, Sensitivity, Data models, Time series analysis, Predictive models, Perturbation methods, Computational modeling, Explainable AI, metrics, time series data, deep convolutional neural network
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-98271 (URN)10.1109/ACCESS.2022.3155115 (DOI)000766548000001 ()2-s2.0-85125751693 (Scopus ID)
Note

Funding agency:

Umeå University

Available from: 2022-03-28 Created: 2022-03-28 Last updated: 2022-03-28Bibliographically approved
Fouladgar, N., Alirezaie, M. & Främling, K. (2021). Exploring Contextual Importance and Utility in Explaining Affect Detection. In: Matteo Baldoni; Stefania Bandini (Ed.), AIxIA 2020 – Advances in Artificial Intelligence: . Paper presented at 19th International Conference of the Italian-Association-for-Artificial-Intelligence (AIxIA 2020), (Virtual conference), November 25-27, 2020 (pp. 3-18). Springer, 12414
Open this publication in new window or tab >>Exploring Contextual Importance and Utility in Explaining Affect Detection
2021 (English)In: AIxIA 2020 – Advances in Artificial Intelligence / [ed] Matteo Baldoni; Stefania Bandini, Springer, 2021, Vol. 12414, p. 3-18Conference paper, Published paper (Refereed)
Abstract [en]

By the ubiquitous usage of machine learning models with their inherent black-box nature, the necessity of explaining the decisions made by these models has become crucial. Although outcome explanation has been recently taken into account as a solution to the transparency issue in many areas, affect computing is one of the domains with the least dedicated effort on the practice of explainable AI, particularly over different machine learning models. The aim of this work is to evaluate the outcome explanations of two black-box models, namely neural network (NN) and linear discriminant analysis (LDA), to understand individuals affective states measured by wearable sensors. Emphasizing on context-aware decision explanations of these models, the two concepts of Contextual Importance (CI) and Contextual Utility (CU) are employed as a model-agnostic outcome explanation approach. We conduct our experiments on the two multimodal affect computing datasets, namely WESAD and MAHNOB-HCI. The results of applying a neural-based model on the first dataset reveal that the electrodermal activity, respiration as well as accelorometer sensors contribute significantly in the detection of "meditation" state for a particular participant. However, the respiration sensor does not intervene in the LDA decision of the same state. On the course of second dataset and the neural network model, the importance and utility of electrocardiogram and respiration sensors are shown as the dominant features in the detection of an individual "surprised" state, while the LDA model does not rely on the respiration sensor to detect this mental state.

Place, publisher, year, edition, pages
Springer, 2021
Series
Lecture Notes in Computer Science (LNCS), ISSN 0302-9743, E-ISSN 1611-3349 ; 12414
Keywords
Explainable AI, Affect detection, Black-Box decision, Contextual importance and utility
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-102684 (URN)10.1007/978-3-030-77091-4_1 (DOI)000886994000001 ()2-s2.0-85111382491 (Scopus ID)9783030770914 (ISBN)9783030770907 (ISBN)
Conference
19th International Conference of the Italian-Association-for-Artificial-Intelligence (AIxIA 2020), (Virtual conference), November 25-27, 2020
Available from: 2022-12-13 Created: 2022-12-13 Last updated: 2022-12-13Bibliographically approved
Chimamiwa, G., Alirezaie, M., Pecora, F. & Loutfi, A. (2021). Multi-sensor dataset of human activities in a smart home environment. Data in Brief, 34, Article ID 106632.
Open this publication in new window or tab >>Multi-sensor dataset of human activities in a smart home environment
2021 (English)In: Data in Brief, E-ISSN 2352-3409, Vol. 34, article id 106632Article in journal (Refereed) Published
Abstract [en]

Time series data acquired from sensors deployed in smart homes present valuable information for intelligent systems to learn activity patterns of occupants. With the increasing need to enable people to age in place independently, the availability of such data is key to the development of home monitoring solutions. In this article we describe an unlabelled dataset of measurements collected from multiple environmental sensors placed in a smart home to capture human activities of daily living. Various sensors were used including passive infrared, force sensing resistors, reed switches, mini photocell light sensors, temperature and humidity, and smart plugs. The sensors record data from the user's interactions with the environment, such as indoor movements, pressure applied on the bed, or current consumption when using electrical appliances. Millions of raw sensor data samples were collected continuously at a frequency of 1 Hz over a period of six months between 26 February 2020 and 26 August 2020. The dataset can be useful in the analysis of different methods, including data-driven algorithms for activity or habit recognition. In particular, the research community might be interested in investigating the performance of algorithms when applied on unlabelled datasets and not necessarily on annotated datasets. Furthermore, by applying artificial intelligence (AI) algorithms on such data collected over long periods, it is possible to extract patterns that reveal the user's habits as well as detect changes in the habits. This can benefit in detecting deviations in order to provide timely interventions for patients, e.g., people with dementia.

Place, publisher, year, edition, pages
Elsevier, 2021
Keywords
Activities of daily living, Activity recognition, Habit recognition, Smart homes, Time series dataset
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-88421 (URN)10.1016/j.dib.2020.106632 (DOI)000617525400022 ()33376761 (PubMedID)2-s2.0-85097861784 (Scopus ID)
Funder
Knowledge Foundation
Note

Funding Agency:

European Commission 754285

Available from: 2021-01-12 Created: 2021-01-12 Last updated: 2024-03-27Bibliographically approved
Organisations

Search in DiVA

Show all publications