To Örebro University

We propose a flexible hierarchical navigation stack for a mobile robot in complex dynamic environments. Addressing the growing need for reliable navigation in real-world scenarios, where dynamic agents and environmental uncertainties pose significant challenges, our solution decomposes this complexity into task planning, navigation, control, and safe velocity components. In contrast to the prior art, our system at every level incorporates diverse contextual information about the environment, anticipates navigation risks and proactively avoids collisions with dynamic agents.

To achieve natural and intuitive interaction with people, HRI frameworks combine a wide array of methods for human perception, intention communication, human-aware navigation and collaborative action. In practice, when encountering unpredictable behavior of people or unexpected states of the environment, these frameworks may lack the ability to dynamically recognize such states, adapt and recover to resume the interaction. Large Language Models (LLMs), owing to their advanced reasoning capabilities and context retention, present a promising solution for enhancing robot adaptability. This potential, however, may not directly translate to improved interaction metrics. This paper considers a representative interaction with an industrial robot involving approach, instruction, and object manipulation, implemented in two conditions: (1) fully scripted and (2) including LLM-enhanced responses. We use gaze tracking and questionnaires to measure the participants' task efficiency, engagement, and robot perception. The results indicate higher SUbjective ratings for the LLM condition, but objective metrics show that the scripted condition performs comparably, particularly in efficiency and focus during simple tasks. We also note that the scripted condition may have an edge over LLM-enhanced responses in terms of response latency and energy consumption, especially for trivial and repetitive interactions.

In this paper we present a hierarchical motion and intent prediction system prototype, designed to efficiently operate in complex environments while safely handling risks arising from diverse and uncertain human motion and activities. Our system uses an array of advanced cues to describe human motion and activities, including generalized motion patterns, full-body poses, heterogeneous agent types and causal contextual factors that influence human behavior.

Soft Actor-Critic (SAC) has achieved notable success in continuous control tasks but struggles in sparse reward settings, where infrequent rewards make efficient exploration challenging. While novelty-based exploration methods address this issue by encouraging the agent to explore novel states, they are not trivial to apply to SAC. In particular, managing the interaction between novelty-based exploration and SAC’s stochastic policy can lead to inefficient exploration and redundant sample collection. In this paper, we propose KEA (Keeping Exploration Alive) which tackles the inefficiencies in balancing exploration strategies when combining SAC with novelty-based exploration. KEA integrates a novelty-augmented SAC with a standard SAC agent, proactively coordinated via a switching mechanism. This coordination allows the agent to maintain stochasticity in high-novelty regions, enhancing exploration efficiency and reducing repeated sample collection. We first analyze this potential issue in a 2D navigation task, and then evaluate KEA on the DeepSea hard-exploration benchmark as well as sparse reward control tasks from the DeepMind Control Suite. Compared to state-of-the-art novelty-based exploration baselines, our experiments show that KEA significantly improves learning efficiency and robustness in sparse reward setups.

Long-term human motion prediction (LHMP) is important for the safe and efficient operation of autonomous robots and vehicles in environments shared with humans. Accurate predictions are important for applications including motion planning, tracking, human-robot interaction, and safety monitoring. In this letter, we exploit Maps of Dynamics (MoDs), which encode spatial or spatio-temporal motion patterns as environment features, to achieve LHMP for horizons of up to 60 seconds. We propose an MoD-informed LHMP framework that supports various types of MoDs and includes a ranking method to output the most likely predicted trajectory, improving practical utility in robotics. Further, a time-conditioned MoD is introduced to capture motion patterns that vary across different times of day. We evaluate MoD-LHMP instantiated with three types of MoDs. Experiments on two real-world datasets show that MoD-informed method outperforms learning-based ones, with up to 50% improvement in average displacement error, and the time-conditioned variant achieves the highest accuracy overall.

Successful adoption of industrial robots will strongly depend on their ability to safely and efficiently operate in human environments, engage in natural communication, understand their users, and express intentions intuitively while avoiding unnecessary distractions. To achieve this advanced level of Human-Robot Interaction (HRI), robots need to acquire and incorporate knowledge of their users’ tasks and environment and adopt multimodal communication approaches with expressive cues that combine speech, movement, gazes, and other modalities. This paper presents several methods to design, enhance, and evaluate expressive HRI systems for non-humanoid industrial robots. We present the concept of a small anthropomorphic robot communicating as a proxy for its non-humanoid host, such as a forklift. We developed a multimodal and LLM-enhanced communication framework for this robot and evaluated it in several lab experiments, using gaze tracking and motion capture to quantify how users perceive the robot and measure the task progress

RadaRays allows for the accurate modeling and simulation of rotating FMCW radar sensors in complex environments, including the simulation of reflection, refraction, and scattering of radar waves. Our software is able to handle large numbers of objects and materials in real-time, making it suitable for use in a variety of mobile robotics applications. We demonstrate the effectiveness of RadaRays through a series of experiments and show that it can more accurately reproduce the behavior of FMCW radar sensors in a variety of environments, compared to the ray casting-based lidar-like simulations that are commonly used in simulators for autonomous driving such as CARLA. Our experiments additionally serve as a valuable reference point for researchers to evaluate their own radar simulations. By using RadaRays, developers can significantly reduce the time and cost associated with prototyping and testing FMCW radar-based algorithms. We also provide a Gazebo plugin that makes our work accessible to the mobile robotics community.

Motion planning for mobile robots in dynamic environments shared with people is challenging. In a typical hierarchical planning framework, robots use a global planner for overall path-finding and a local planner for immediate adjustments to the path. Without considering the typical patterns of motion of dynamic entities, the global planner might generate paths that lead the robot into highly congested areas, where it is forced to wait, replan or manoeuvre around dynamic obstacles. Maps of Dynamics (MoDs) are a way to mitigate these issues. MoDs represent patterns of motion exhibited by moving entities in the environment, using probabilistic models. The use of MoDs in cost functions for motion planning enables a robot to plan motions that consider the motion patterns encoded in the MoDs. In previous work, it has been shown that the use of MoDs in the cost function helps generate more efficient paths, i.e., paths that lead to the robot and pedestrians spending less time waiting for each other. It has also been shown that using MoDs in the sampling step of sampling-based motion planning is beneficial to a mobile robot since it can result in reduced computation time by explicitly guiding the sampling process using information encoded in MoDs. However, existing work on the use of MoDs in the sampling process is limited. Correspondingly, an analysis of the performance of sampling heuristics for MoDs is also largely lacking. Since such an analysis is crucial to understand the effectiveness of MoDs in a practical setting, we ask the research question: can we obtain reasonably low-cost solutions using sampling-based motion planners that consider the flow of dynamic entities, in a reasonable amount of time. In this paper, we propose substantial improvements to two existing sampling heuristics: the Dijkstra-graph sampling (DGS), previously restricted to a specific type of MoD is extended to use any MoD; and the intensity-map (normalized number of observations of dynamic entities in each grid cell) is utilized more effectively by using importance sampling instead of rejection sampling. We show that an ellipsoidal heuristic can also be used with MoDs. We experimentally validate several sampling heuristics on two different sampling-based motion planners and present a comprehensive evaluation (52800 runs) of their performance on real-world data from densely populated environments. We conclude that reasonably cost solutions can be quickly obtained using a combination of the sampling heuristics within practically feasible time limits. Using the RRT* planner with our proposed MoD-aware, Dijkstra-graph-based heuristic yields approximate to 5%, %10 and 12% higher success rates after 2, 4 and 8 s of planning respectively, compared to uniform sampling, the baseline.

Neural implicit surface representations are currently receiving a lot of interest as a means to achieve high-fidelity surface reconstruction at a low memory cost, compared to traditional explicit representations. However, state-of-the-art methods still struggle with excessive memory usage and non-smooth surfaces. This is particularly problematic in large-scale applications with sparse inputs, as is common in robotics use cases. To address these issues, we first introduce a sparse structure, tri-quadtrees, which represents the environment using learnable features stored in three planar quadtree projections. Secondly, we concatenate the learnable features with a Fourier feature positional encoding. The combined features are then decoded into signed distance values through a small multi-layer perceptron. We demonstrate that this approach facilitates smoother reconstruction with a higher completion ratio with fewer holes. Compared to two recent baselines, one implicit and one explicit, our approach requires only 10%–50% as much memory, while achieving competitive quality. The code is released on https://github.com/ljjTYJR/3QFP.

Coordinating a fleet of robots in unstructured, human-shared environments is challenging. Human behavior is hard to predict, and its uncertainty impacts the performance of the robotic fleet. Various multi-robot planning and coordination algorithms have been proposed, including Multi-Agent Path Finding (MAPF) methods to precedence-based algorithms. However, it is still unclear how human presence impacts different coordination strategies in both simulated environments and the real world. With the goal of studying and further improving multi-robot planning capabilities in those settings, we propose a method to develop and benchmark different multi-robot coordination algorithms in realistic, unstructured and human-shared environments. To this end, we introduce a multi-robot benchmark framework that is based on state-of-the-art open-source navigation and simulation frameworks and can use different types of robots, environments and human motion models. We show a possible application of the benchmark framework with two different environments and three centralized coordination methods (two MAPF algorithms and a loosely-coupled coordination method based on precedence constraints). We evaluate each environment for different human densities to investigate its impact on each coordination method. We also present preliminary results that show how informing each coordination method about human presence can help the coordination method to find faster paths for the robots.