To Örebro University

As district heating networks evolve to meet climate-neutral objectives, optimizing their control under heterogeneous building characteristics and dynamic environmental conditions remains a significant challenge. Traditional control strategies often lack the adaptability necessary to account for building-specific dynamics and to ensure real-time adherence to operational safety constraints. In this work, we present an integrated machine learning-based framework that combines an adaptive context-aware transformer model with deep reinforcement learning to address these limitations. The proposed approach introduces an adaptive context-aware transformer as a predictive environment within a Deep Q-Network (DQN) framework, enabling data-driven, building-specific control of district heating systems. Utilizing real-world data from 148 residential buildings across Sweden and Finland, the model incorporates contextual embeddings and temporal features to predict indoor temperature trajectories with high accuracy, achieving root mean square error values between 0.18-0.24 degrees C for Swedish buildings and 0.26-0.32 degrees C for Finnish buildings. The DQN agent leverages these predictions to optimize heating control while ensuring compliance with operational safety limits and preserving occupant comfort. Experimental results demonstrate significant energy savings, with mid-rise buildings achieving up to 14.85% reduction in energy consumption, and peak seasonal savings exceeding 20% during spring months. This integrated approach illustrates the potential for substantial energy optimization and reliable indoor climate management in future district heating networks.

High-resolution imaging is essential in remote sensing and geoscience for precise environmental and geological analysis. DR-SCAN (Dual-Branch Residual Spatial and Channel Attention Networks), a neural network architecture for image super-resolution across these domains, is introduced. Evaluated on the UCMerced Land Use and DeepRock-SR datasets, DR-SCAN demonstrates adaptability to diverse remote sensing landscapes and effectiveness in resolving pore-scale geological features. Feature map visualizations highlight the model’s ability to prioritize crit-ical spatial features, enhancing interpretability for domain-specific applications.

This study investigates the efficiency and safety outcomes of implementing different adaptive coordination models for automated vehicle (AV) fleets, managed by a centralized coordinator that dynamically responds to human-controlled vehicle behavior. The simulated scenarios replicate an underground mining environment characterized by narrow tunnels with limited connectivity. To address the unique challenges of such settings, we propose a novel metric — Path Overlap Density (POD) — to predict efficiency and potentially the safety performance of AV fleets. The study also explores the impact of map features on AV fleets performance. The results demonstrate that both AV fleet coordination strategies and underground tunnel network characteristics significantly influence overall system performance. While map features are critical for optimizing efficiency, adaptive coordination strategies are essential for ensuring safe operations. 

Integrating human expertise with machine learning is crucial for applications demanding high accuracy and safety, such as autonomous driving. This study introduces Interactive Double Deep Q-network (iDDQN), a Human-in-the-Loop (HITL) approach that enhances Reinforcement Learning (RL) by merging human insights directly into the RL training process, improving model performance. Our proposed iDDQN method modifies the Q-value update equation to integrate human and agent actions, establishing a collaborative approach for policy development. Additionally, we present an offline evaluative framework that simulates the agent's trajectory as if no human intervention to assess the effectiveness of human interventions. Empirical results in simulated autonomous driving scenarios demonstrate that iDDQN outperforms established approaches, including Behavioral Cloning (BC), HG-DAgger, Deep Q-Learning from Demonstrations (DQfD), and vanilla DRL in leveraging human expertise for improving performance and adaptability.

Designing effective reward functions is crucial to training reinforcement learning (RL) algorithms. However, this design is non-trivial, even for domain experts, due to the subjective nature of certain tasks that are hard to quantify explicitly. In recent works, large language models (LLMs) have been used for reward generation from natural language task descriptions, leveraging their extensive instruction tuning and commonsense understanding of human behavior. In this work, we hypothesize that LLMs, guided by human feedback, can be used to formulate reward functions that reflect human implicit knowledge. We study this in three challenging settings - autonomous driving, humanoid locomotion, and dexterous manipulation - wherein notions of “good” behavior are tacit and hard to quantify. To this end, we introduce REvolve, a truly evolutionary framework that uses LLMs for reward design in RL. REvolve generates and refines reward functions by utilizing human feedback to guide the evolution process, effectively translating implicit human knowledge into explicit reward functions for training (deep) RL agents. Experimentally, we demonstrate that agents trained on REvolve-designed rewards outperform other state-of-the-art baselines. 

Transformer-based models have gained increasing popularity for time-series prediction; however, in specific applications such as residential heating systems, static contextual data of buildings is crucial to effectively capture and learn complex environmental dynamics. This paper presents a novel transformer-based model that adapts the contextual meta-data of residential buildings, generalizing across diverse environments. The model integrates temporal data with adaptive embedding of building-specific contextual meta-data such as geographic locations and building characteristics to dynamically learn and adapt to the variations. These adaptive context embeddings allow the model to comprehensively understand how different buildings respond to environmental changes over time. Initial results show improved accuracy and reliability in indoor temperature predictions of residential buildings, demonstrating the model’s potential to optimize district heating systems across a diverse array of residential buildings. This proposed model provides a basis for developing proactive heat management systems in buildings.

Goal imagination in robotics is an emerging concept and involves the capability to automatically generate realistic goals, which, in turn, requires the assessment of the feasibility of transitioning from the current conditions of an initial scene to thedesired goal state. Existing research has explored the utilization of diverse image-generative models to create images depicting potential goal states based on the current state and instructions. In this paper, we illustrate the limitations of current state-of-the-art image generative models in accurately assessing the feasibility of specific actions in particular situations. Consequently, we present how integrating large language models, which possess profound knowledge of real-world objects and affordances, can enhance the performance of image-generative models in discerning plausible from implausible actions and simulating the outcomes of actions in a given context. This will be a step towards achieving the pragmatic goal of imagination in robotics.

There is an increased interest in using social robots to assist older adults during their daily life activities. As social robots are designed to interact with older users, it becomes relevant to study these interactions under the lens of social cognition. Gaze following, the social ability to infer where other people are looking at, deteriorates with older age. Therefore, the referential gaze from robots might not be an effective social cue to indicate spatial locations to older users. In this study, we explored the performance of older adults, middle-aged adults, and younger controls in a task assisted by the referential gaze of a Pepper robot. We examined age-related differences in task performance, and in self-reported social perception of the robot. Our main findings show that referential gaze from a robot benefited task performance, although the magnitude of this facilitation was lower for older participants. Moreover, perceived anthropomorphism of the robot varied less as a result of its referential gaze in older adults. This research supports that social robots, even if limited in their gazing capabilities, can be effectively perceived as social entities. Additionally, this research suggests that robotic social cues, usually validated with young participants, might be less optimal signs for older adults.

Supplementary Information: The online version contains supplementary material available at 10.1007/s12369-022-00926-6.

The green area factor model is a crucial tool for conserving and creating urban greenery and ecosystem services within neighborhood land. This model serves as a valuable index, streamlining the planning, assessment, and comparison of local-scale green infrastructures. However, conventional on-site measurements of the green area factor are resource intensive. In response, this study pioneers a computational approach that integrates ecological and social dimensions to estimate the green area factor. Employing satellite remote sensing and advanced deep learning techniques, the methodology utilizes satellite orthophotos of urban areas subjected to semantic segmentation, identifying and categorizing diverse green elements. Ground truths are established through on-site measurements of green area factors and satellite orthophotos from benchmarking sites in <spacing diaeresis>Orebro, Sweden. Results reveal an 82.0% average F1-score for semantic segmentations, signifying a favourable correlation between computationally estimated and measured green area factors. The proposed methodology is potential for adapting to various urban settings. In essence, this research introduces a promising, cost-effective solution for assessing urban greenness, particularly beneficial for urban administrators and planners aiming for insightful and comprehensive green strategies in city planning.

This study investigates the complexities of Mixed Traffic with Fleets of Automated Vehicles (MTF-AVs) in underground mining environments characterized by confined spaces, limited visibility, and strict navigation requirements. The research focuses on integrating human-controlled vehicles into coordinated AV fleets, addressing the unpredictable interactions that arise from human behaviour. The ORU coordination framework, originally designed for a fully autonomous system, is adapted for mixed traffic scenarios to evaluate the impact of human behaviour on system efficiency and safety. Through a series of simulations, the study explores how fleet coordination algorithms adapt to human driver behaviour. These simulations demonstrate that human error and rule violations significantly reduce performance, increasing safety risks and decreasing efficiency. Findings emphasize the need for advanced coordination algorithms that dynamically adapt to unpredictable human behaviour in MTF-AVs. Such algorithms would optimize interactions between automated and human-controlled vehicles, enhancing both safety and efficiency in these complex and dynamic environments. Future research will further explore the influence of human behaviour on the coordination system and develop advanced coordination algorithms with methods to evaluate these interactions effectively.