To Örebro University

The vision of robots assisting with our daily life tasks hinges on a fundamental challenge: most robot behaviors require programming by experts, creating a barrier for the many. Learning from Demonstration (LfD) is a family of methods with the potential to democratize robot programming by enabling laypeople to teach robots new skills simply by showing them. However, most LfD methods operate as black-box systems, making it difficult for humans to interpret, adapt, or reuse — key factors for human-robot collaboration and understanding.

In this thesis, we instead build upon Behavior Trees (BTs) — a transparent and manageable robot programming framework, whose reactive, modular design and potential for interoperability make them well-suited for constructing sophisticated robot behaviors. To bridge the gap between high-level planning and low-level control, engineers often embed entire behaviors within the BT’s leaf nodes. While functional, this approach encapsulates complete skills, obscuring intermediate subgoals and undermining the transparency and modularity that BTs are meant to provide — ultimately limiting skill reuse and adaptability.

The structure of a BT plays a crucial role in effective behavior design. While several approaches have aimed to learn BT structures from demonstrations, they often rely on predefined action sets and state spaces. This necessity for expert-curated inputs constrains the robot’s learning flexibility and reintroduces a degree of expert dependency. Compounding these challenges is a more foundational issue: the BT community lacks universally accepted definitions and rigorous evaluation methods for key properties such as interpretability and modularity. This absence leads to inconsistent claims and makes meaningful comparisons across different studies a significant challenge.

To address these gaps, this dissertation proposes a path toward more intuitive and effective robot learning, articulated through three key contributions:

First, this thesis formalizes core BT properties for robotics and introduces metrics for the systematic evaluation and comparison of learned policies. Building on this foundation, it investigates how different BT structures impact the interpretability of the control policy, identifying design patterns that best align with human intuition and understanding.

Second, this thesis presents a unified control framework that integrates BTs with a high-frequency Stack-of-Tasks (SoT) control strategy, enhancing transparency of BT policies by explicitly revealing the underlying subgoals. This approach allows BT nodes to function as hierarchical, high-frequency control objectives. Moreover, the resulting system achieves rapid reactivity in dynamic environments while supporting the coexistence of heterogeneous controllers and preserving a clean, modular decomposition of complex tasks.

Finally, this thesis introduces an end-to-end, label-free LfD pipeline that simultaneously learns the global BT structure and the underlying actions —modeled as Dynamic Movement Primitives—directly from raw demonstration data. By leveraging vision-language models to automatically extract and annotate state representations, this method eliminates the need for handcrafted action sets, predefined state spaces, and time-consuming manual labeling.

In summary, this thesis provides a comprehensive framework for learning interpretable, modular, and adaptable robot control policies from demonstration, bridging the gap between transparent policy representation and practical, high-frequency robot control, which marks a significant step toward making robot programming more accessible, robust, and understandable for both experts and non-experts.

The first WARA Robotics Mobile Manipulation Challenge, held in December 2024 at ABB Corporate Research in Västerås, Sweden, addressed the automation of task-intensive and repetitive manual labor in laboratory environments -- specifically the transport and cleaning of glassware. Designed in collaboration with AstraZeneca, the challenge invited academic teams to develop autonomous robotic systems capable of navigating human-populated lab spaces and performing complex manipulation tasks, such as loading items into industrial dishwashers. This paper presents an overview of the challenge setup, its industrial motivation, and the four distinct approaches proposed by the participating teams. We summarize lessons learned from this edition and propose improvements in design to enable a more effective second iteration to take place in 2025. The initiative bridges an important gap in effective academia-industry collaboration within the domain of autonomous mobile manipulation systems by promoting the development and deployment of

Behavior trees (BTs) are increasingly popular in the robotics community. Yet in the growing body of published work on this topic, there is a lack of consensus on what to measure and how to quantify BTs when reporting results. This is not only due to the lack of standardized measures, but due to the sometimes ambiguous use of definitions to describe BT properties. This work provides a comprehensive overview of BT properties the community is interested in, how they relate to each other, the metrics currently used to measure BTs, and whether the metrics appropriately quantify those properties of interest. Finally, we provide the practitioner with a set of metrics to measure, as well as insights into the properties that can be derived from those metrics. By providing this holistic view of properties and their corresponding evaluation metrics, we hope to improve clarity when using BTs in robotics. This more systematic approach will make reported results more consistent and comparable when evaluating BTs.

Stack-of-Tasks (SoT) control allows a robot to simultaneously fulfill a number of prioritized goals formulated in terms of (in)equality constraints in error space. Since this approach solves a sequence of Quadratic Programs (QP) at each time-step, without taking into account any temporal state evolution, it is suitable for dealing with local disturbances. However, its limitation lies in the handling of situations that require non-quadratic objectives to achieve a specific goal, as well as situations where countering the control disturbance would require a locally suboptimal action. Recent works address this shortcoming by exploiting Finite State Machines (FSMs) to compose the tasks in such a way that the robot does not get stuck in local minima. Nevertheless, the intrinsic trade-off between reactivity and modularity that characterizes FSMs makes them impractical for defining reactive behaviors in dynamic environments. In this letter, we combine the SoT control strategy with Behavior Trees (BTs), a task switching structure that addresses some of the limitations of the FSMs in terms of reactivity, modularity and re-usability. Experimental results on a Franka Emika Panda 7-DOF manipulator show the robustness of our framework, that allows the robot to benefit from the reactivity of both SoT and BTs.

Integrating the heterogeneous controllers of a complex mechanical system, such as a mobile manipulator, within the same structure and in a modular way is still challenging. In this work we extend our framework based on Behavior Trees for the control of a redundant mechanical system to the problem of commanding more complex systems that involve multiple low-level controllers. This allows the integrated systems to achieve non-trivial goals that require coordination among the sub-systems.