To Örebro University

Sim-to-real transfer remains a major challenge in reinforcement learning (RL) for robotics, as policies trained in simulation often fail to generalize to the real world due to discrepancies in environment dynamics. Domain Randomization (DR) mitigates this issue by exposing the policy to a wide range of randomized dynamics during training, yet leading to a reduction in performance. While standard approaches typically train policies agnostic to these variations, we investigate whether sim-to-real transfer can be improved by conditioning the policy on an estimate of the dynamics parameters - referred to as context. To this end, we integrate a context estimation module into a DR-based RL framework and systematically compare SOTA supervision strategies. We evaluate the resulting context-aware policies in both a canonical control benchmark and a real-world pushing task using a Franka Emika Panda robot. Results show that context-aware policies outperform the context-agnostic baseline across all settings, although the best supervision strategy depends on the task.

The increasing complexity and unpredictability of manipulation tasks in modern industrial and service robotics have highlighted the limitations of pre-programmed robot solutions. Robots operating under changing object positions, variable obstacles, and unforeseen perturbations must adjust their actions online to reliably satisfy the high-performance requirements of real-world deployment scenarios.

A promising direction for enabling such flexible and reliable manipulation lies in the use of Behavior Trees (BTs), a formalism for transparent decision-making that structures robot behavior hierarchically through modular, reusable components. BTs are a well-suited solution because their inherent reactivity allows the system to respond effectively to high-level disturbances, such as perception or grasping failures. At the same time, their modular design facilitates the reuse of sub-behaviors across different scenarios, enabling automation systems to be easily reconfigured to meet varying operational demands. However, existing BT-based approaches fall short in scenarios in which more advanced forms of robustness to local perturbations and task variations are required. This thesis contributes novel solutions to address these limitations and enhance the applicability of BTs as control policies in real-world manipulation settings.

To rigorously assess the solutions proposed in this thesis, we first need to formalize the terminology and evaluation criteria associated with BT-based robot control. We begin by identifying a subset of properties that are most relevant to our scope, such as reactivity, modularity, and robustness, and clarifying their definitions by resolving ambiguities found in prior work. For each of these properties, we examine how they have been evaluated in the literature and propose additional metrics to address identified gaps in existing evaluation practices.

The first technical contribution addresses the reactivity of BT policies and their way of handling simultaneous control objectives. While BTs effectivel ymanage global, high-level disturbances, flexible manipulation also requires rapid response to local, low-level perturbations that do not warrant changes to the high-level plan. Furthermore, when BTs are coupled with convention allow-level controllers for redundant manipulators, they often struggle to satisfy multiple, potentially competing objectives in a coherent and reliable manner. To address these limitations, we integrate BTs with a prioritized control strategy that decomposes each manipulation skill, such as grasping, into multiple control objectives with defined priorities, distributed across the BT nodes and executed concurrently. This integration introduces an additional layer of low-level reactivity, ensures the reliable satisfaction of multiple objectives, and reinforces the modularity of the BT policy by assigning distinct goals to separate leaf nodes.

Although the proposed framework provides robustness to both high- and low-level disturbances during execution, it still relies on manually specified parameters, which often need adjustment to specific task variations, such as minor changes of object positions or obstacle configurations. The second technical contribution is a data-driven approach based on Reinforcement Learning that augments the BT with a context-based adaptation policy. This module observes task-relevant features, referred to as a context, and selects appropriate BT parameters at execution time. The result is a policy that adapts its behavior on the fly to previously unseen variations, without manual intervention.

Despite its benefits, the proposed framework remains limited to adapting only to directly observable task variations and requires training procedures that, when performed on a physical robot, are often unsafe and timeconsuming. The last technical contribution addresses both limitations by introducing a context estimator that infers latent dynamics parameters — such as friction coefficients or object mass — from recent interaction data. Conditioning the context-based adaptation policy on this latent estimate enables the BT-based policy to operate robustly even under partial observability. Moreover, because these latent parameters often underlie the discrepancies between simulation and reality, the very same mechanism also provides a principled way to bridge the sim-to-real gap: policies are trained in simulation with domain randomization, while the estimated context compensates for the mismatched dynamics, improving robustness at deployment.

The first WARA Robotics Mobile Manipulation Challenge, held in December 2024 at ABB Corporate Research in Västerås, Sweden, addressed the automation of task-intensive and repetitive manual labor in laboratory environments -- specifically the transport and cleaning of glassware. Designed in collaboration with AstraZeneca, the challenge invited academic teams to develop autonomous robotic systems capable of navigating human-populated lab spaces and performing complex manipulation tasks, such as loading items into industrial dishwashers. This paper presents an overview of the challenge setup, its industrial motivation, and the four distinct approaches proposed by the participating teams. We summarize lessons learned from this edition and propose improvements in design to enable a more effective second iteration to take place in 2025. The initiative bridges an important gap in effective academia-industry collaboration within the domain of autonomous mobile manipulation systems by promoting the development and deployment of

Behavior trees (BTs) are increasingly popular in the robotics community. Yet in the growing body of published work on this topic, there is a lack of consensus on what to measure and how to quantify BTs when reporting results. This is not only due to the lack of standardized measures, but due to the sometimes ambiguous use of definitions to describe BT properties. This work provides a comprehensive overview of BT properties the community is interested in, how they relate to each other, the metrics currently used to measure BTs, and whether the metrics appropriately quantify those properties of interest. Finally, we provide the practitioner with a set of metrics to measure, as well as insights into the properties that can be derived from those metrics. By providing this holistic view of properties and their corresponding evaluation metrics, we hope to improve clarity when using BTs in robotics. This more systematic approach will make reported results more consistent and comparable when evaluating BTs.

Stack-of-Tasks (SoT) control allows a robot to simultaneously fulfill a number of prioritized goals formulated in terms of (in)equality constraints in error space. Since this approach solves a sequence of Quadratic Programs (QP) at each time-step, without taking into account any temporal state evolution, it is suitable for dealing with local disturbances. However, its limitation lies in the handling of situations that require non-quadratic objectives to achieve a specific goal, as well as situations where countering the control disturbance would require a locally suboptimal action. Recent works address this shortcoming by exploiting Finite State Machines (FSMs) to compose the tasks in such a way that the robot does not get stuck in local minima. Nevertheless, the intrinsic trade-off between reactivity and modularity that characterizes FSMs makes them impractical for defining reactive behaviors in dynamic environments. In this letter, we combine the SoT control strategy with Behavior Trees (BTs), a task switching structure that addresses some of the limitations of the FSMs in terms of reactivity, modularity and re-usability. Experimental results on a Franka Emika Panda 7-DOF manipulator show the robustness of our framework, that allows the robot to benefit from the reactivity of both SoT and BTs.

Integrating the heterogeneous controllers of a complex mechanical system, such as a mobile manipulator, within the same structure and in a modular way is still challenging. In this work we extend our framework based on Behavior Trees for the control of a redundant mechanical system to the problem of commanding more complex systems that involve multiple low-level controllers. This allows the integrated systems to achieve non-trivial goals that require coordination among the sub-systems.