To Örebro University

In this article, we address the problem of realizing a complete efficient system for automated management of fleets of autonomous ground vehicles in industrial sites. We elicit from current industrial practice and the scientific state of the art the key challenges related to autonomous transport vehicles in industrial environments and relate them to enabling techniques in perception, task allocation, motion planning, coordination, collision prediction, and control. We propose a modular approach based on least commitment, which integrates all modules through a uniform constraint-based paradigm. We describe an instantiation of this system and present a summary of the results, showing evidence of increased flexibility at the control level to adapt to contingencies.

The main focus of this paper is to present a case study of a SLAM solution for Automated Guided Vehicles (AGVs) operating in real-world industrial environments. The studied solution, called Gold-fish SLAM, was implemented to provide localization estimates in dynamic industrial environments, where there are static landmarks that are only rarely perceived by the AGVs. The main idea of Gold-fish SLAM is to consider the goods that enter and leave the environment as temporary landmarks that can be used in combination with the rarely seen static landmarks to compute online estimates of AGV poses. The solution is tested and verified in a factory of paper using an eight ton diesel-truck retrofitted with an AGV control system running at speeds up to 3 m/s. The paper includes also a general discussion on how SLAM can be used in industrial applications with AGVs

The main focus of this paper is to present a case study of a SLAM solution for Automated Guided Vehicles (AGVs) operating in real-world industrial environ- ments. The studied solution, called Gold-fish SLAM, was implemented to provide localization estimates in dynamic industrial environments, where there are static landmarks that are only rarely perceived by the AGVs. The main idea of Gold-fish SLAM is to consider the goods that enter and leave the environment as temporary landmarks that can be used in combination with the rarely seen static landmarks to compute online estimates of AGV poses. The solution is tested and verified in a factory of paper using an eight ton diesel-truck retrofitted with an AGV control sys- tem running at speeds up to 3 meters per second. The paper includes also a general discussion on how SLAM can be used in industrial applications with AGVs.

In this paper we report on the extension of the classical HTN planner SHOP to plan in partially observable domains with uncertainty. Our algorithm PC-SHOP uses belief states to handle situations involving incomplete and uncertain information about the state of the world. Sensing and acting are integrated in the primitive actions through the use of a stochastic model. PC-SHOP is showed to scale up well compared to some of the state-of-the-art planners. We outline the main characteristics of the algorithm, and present performance results on some problems found in the literature.

In this paper we propose an approach to handle requests of finding objects in real world environments by mobile robots. The proposed approach checks candidate objects based on the likelihood they constitute an answer to the requests in a reactive way. As a result, run-time perceived objects are handled “on the fly” without extra cost. We present the theoretical concepts of the proposed approach, and describe the experiments we run to validate it.

Autonomous mobile robots are being developed with the aim of accomplishing complex tasks in different environments, including human habitats as well as less friendly places, such as distant planets and underwater regions. A major challenge faced by such robots is to make sure that their actions are executed correctly and reliably, despite the dynamics and the uncertainty inherent in their working space. This thesis is concerned with the ability of a mobile robot to reliably monitor the execution of its plans and detect failures.

Existing approaches for monitoring the execution of plans rely mainly on checking the explicit effects of plan actions, i.e., effects encoded in the action model. This supposedly means that the effects to monitor are directly observable, but that is not always the case in a real-world environment. In this thesis, we propose to use semantic domain-knowledge to derive and monitor implicit expectations about the effects of actions. For instance, a robot entering a room asserted to be an office should expect to see at least a desk, a chair, and possibly a PC. These expectations are derived from knowledge about the type of the room the robot is entering. If the robot enters a kitchen instead, then it should expect to see an oven, a sink, etc.

The major contributions of this thesis are as follows.

• We define the notion of Semantic Knowledge-based Execution Monitoring SKEMon, and we propose a general algorithm for it based on the use of description logics for representing knowledge.

• We develop a probabilistic approach of semantic Knowledge-based execution monitoring to take into account uncertainty in both acting and sensing. Specifically, we allow for sensing to be unreliable and for action models to have more than one possible outcome. We also take into consideration uncertainty about the state of the world. This development is essential to the applicability of our technique, since uncertainty is a pervasive feature in robotics.

• We present a general schema to deal with situations where perceptual information relevant to SKEMon is missing. The schema includes steps for modeling and generating a course of action to actively collect such information. We describe approaches based on planning and greedy action selection to generate the information-gathering solutions. The thesis also shows how such a schema can be applied to respond to failures occurring before or while an action is executed. The failures we address are ambiguous situations that arise when the robot attempts to anchor symbolic descriptions (relevant to a plan action) in perceptual information. The work reported in this thesis has been tested and verified using a mobile robot navigating in an indoor environment. In addition, simulation experiments were conducted to evaluate the performance of SKEMon using known metrics. The results show that using semantic knowledge can lead to high performance in monitoring the execution of robot plans.

This paper presents an overview of an autonomous robotic material handling system. The goal of the system is to extend the functionalities of traditional AGVs to operate in highly dynamic environments. Traditionally, the reliable functioning of AGVs relies on the availability of adequate infrastructure to support navigation. In the target environments of our system, such infrastructure is difficult to setup in an efficient way. Additionally, the location of objects to handle are unknown, which requires that the system be able to detect and track object positions at runtime. Another requirement of the system is to be able to generate trajectories dynamically, which is uncommon in industrial AGV systems.

This paper presents an overview of an autonomousrobotic material handling system. The goal of the system is toextend the functionalities of traditional AGVs to operate in highlydynamic environments. Traditionally, the reliable functioning ofAGVs relies on the availability of adequate infrastructure tosupport navigation. In the target environments of our system,such infrastructure is difficult to setup in an efficient way.Additionally, the location of objects to handle are unknown,which requires that the system be able to detect and track objectpositions at runtime. Another requirement of the system is to beable to generate trajectories dynamically, which is uncommon inindustrial AGV systems.

In this paper we present an algorithm for planning in non-deterministic domains. Our algorithm C-SHOP extends the successful classical HTN planner SHOP, by introducing new mechanisms to handle situations where there is incomplete and uncertain information about the state of the environment. Being an HTN planner, C-SHOP supports coding domain-dependent knowledge in a powerful way that describes how to solve the planning problem.

To handle uncertainty, belief states are used to represent incomplete information about the state of the world, and actions are allowed to have stochastic outcomes. This allows our algorithm to solve problems involving partial observability through feedback at execution time. We outline the main characteristics of the algorithm, and present performance results on some problems found in literature.

In this paper we report on the integration of a high-level plan executor with a behavior-based architecture. The executor is designed to execute plans that solve problems in partially observable domains. We discuss the different modules of the overall architecture and how we made the different modules interact using a shared representation. We also give a detailed description of the hierarchical architecture of the executor and how execution-time failures are handled.

Intelligent agents acting in real world environments need to synthesize their course of action based on multiple sources of knowledge. They also need to generate plans that smoothly integrate actions from different domains. In this paper we present a generic approach to synthesize plans for solving planning problems involving multiple domains. The proposed approach performs search hierarchically by starting planning in one domain and considering subgoals related to the other domains as abstract tasks to be planned for later when their respective domains are considered. To plan in each domain, a domain-dependent planner can be used, making it possible to integrate different planners, possibly with different specializations. We outline the algorithm, and the assumptions underlying its functionality. We also demonstrate through a detailed example, how the proposed framework compares to planning in one global domain.

To cope with the dynamics and uncertainty inherent in real world environments, autonomous mobile robots need to perform execution monitoring for verifying that their plans are executed as expected. Domain semantic knowledge has lately been proposed as a source of information to derive and monitor implicit expectations of executing actions. For instance, when the robot moves into an office, it would expect to see a desk and a chair. Such expectations are checked using the immediately available perceptual information. We propose to extend the semantic knowledge-based execution monitoring to handle situations where some of the required information is missing. To this end, we use AI sensor-based planning to actively search for such information. We show how verifying execution expectations can be formulated and solved as a planning problem involving sensing actions. Our approach is illustrated by showing test scenarios run in an indoor environment using a mobile robot.

Executing plans by mobile robots, in real world environments, faces the challenging issues of uncertainty and environment dynamics. Thus, execution monitoring is needed to verify that plan actions are executed as expected. Semantic domain-knowledge has lately been proposed as a source of information to derive and monitor implicit expectations of executing actions. For instance, when a robot moves into a room asserted to be an office, it would expect to see a desk and a chair. We propose to extend the semantic knowledge-based execution monitoring to take uncertainty in actions and sensing into account when verifying the expectations derived from semantic knowledge. We consider symbolic probabilistic action models, and show how semantic knowledge is used together with a probabilistic sensing model in the monitoring process of such actions. Our approach is illustrated by showing test scenarios ran in an indoor environment using a mobile robot

Executing plans by mobile robots, in real world environments, faces the challenging issues of uncertainty and environment dynamics. Thus, execution monitoring is needed to verify that plan actions are executed as expected. Semantic domain-knowledge has lately been proposed as a source of information to derive and monitor implicit expectations of executing actions. For instance, when a robot moves into a room asserted to be an office, it would expect to see a desk and a chair. We propose to extend the semantic knowledge-based execution monitoring to take uncertainty in actions and sensing into account when verifying the expectations derived from semantic knowledge. We consider symbolic probabilistic action models, and show how semantic knowledge is used together with a probabilistic sensing model in the monitoring process of such actions. Our approach is illustrated by showing test scenarios run in an indoor environment using a mobile robot.

Even the best laid plans can fail, and robot plans executed in real world domains tend to do so often. The ability of a robot to reliably monitor the execution of plans and detect failures is essential to its performance and its autonomy. In this paper, we propose a technique to increase the reliability of monitoring symbolic robot plans. We use semantic domain knowledge to derive implicit expectations of the execution of actions in the plan, and then match these expectations against observations. We present two realizations of this approach: a crisp one, which assumes deterministic actions and reliable sensing, and uses a standard knowledge representation system (LOOM); and a probabilistic one, which takes into account uncertainty in action effects, in sensing, and in world states. We perform an extensive validation of these realizations through experiments performed both in simulation and on real robots.

We describe a novel intelligent execution monitoring approach for mobile robots acting in indoor environments such as offices and houses. Traditionally, monitoring execution in mobile robotics amounted to looking for discrepancies between the model-based predicted state of executing an action and the real world state as computed from sensing data. We propose to employ semantic knowledge as a source of information to monitor execution. The key idea is to compute implicit expectations, from semantic domain information, that can be observed at run time by the robot to make sure actions are executed correctly. We present the semantic knowledge representation formalism, and how semantic knowledge is used in monitoring. We also describe experiments run in an indoor environment using a real mobile robot

We present an approach for recovery from perceptual failures, or more precisely anchoring failures. Anchoring is the problem of connecting symbols representing objects to sensor data corresponding to the same objects. The approach is based on using planning, but our focus is not on the plan generation per se. We focus on the very important aspect of situation assessment and how it is carried out for recovering from anchoring failures. The proposed approach uses background knowledge to create hypotheses about world states and handles uncertainty in terms of probabilistic belief states. This work is relevant both from the perspective of developing the anchoring framework, and as a study in plan-based recovery from epistemic failures in mobile robots. Experiments on a mobile robot are shown to validate the applicability of the proposed approach.

An autonomous robot using symbolic reasoning, sensing and acting in a real environment needs the ability to create and maintain the connection between symbols representing objects in the world and the corresponding perceptual representations given by its sensors. This connection has been named perceptual anchoring. In complex environments, anchoring is not always easy to establish: the situation may often be ambiguous as to which percept actually corresponds to a given symbol.

In this paper, we extend perceptual anchoring to deal robustly with ambiguous situations by providing general methods for detecting them and recovering from them. We consider different kinds of ambiguous situations. We also present methods to recover from these situations based onautomatically formulating them as conditional planning problems that then are solved by a planner.

We illustrate our approach by showing experiments involving a mobile robot equipped with a color camera and an electronic nose.

In this paper, we propose a new approach forautomatically building symbolic relational descriptions of staticconfigurations of objects to be manipulated by a robotic system.The main goal of our work is to provide advanced cognitiveabilities for such robotic systems to make them more aware ofthe outcome of their actions. We describe how such symbolicrelations are automatically extracted for configurations ofbox-shaped objects using notions from geometry and staticequilibrium in classical mechanics. We also present extensivesimulation results as well as some real-world experiments aimedat verifying the output of the proposed approach.

In this paper, we propose an approach for robotic manipulation systems to autonomously reason about their environments under incomplete information. The target application is to automate the task of unloading the content of shipping containers. Our goal is to capture possible support relations between objects in partially known static configurations. We employ support vector machines (SVM) to estimate the probability of a support relation between pairs of detected objects using features extracted from their geometrical properties and 3D sampled points of the scene. The set of probabilistic support relations is then used for reasoning about optimally selecting an object to be unloaded first. The proposed approach has been extensively tested and verified on data sets generated in simulation and from real world configurations.

In this article, we describe an approach to address the issue of automatically building and using high-level symbolic representations that capture physical interactions between objects in static configurations. Our work targets robotic manipulation systems where objects need to be safely removed from piles that come in random configurations. We assume that a 3D visual perception module exists so that objects in the piles can be completely or partially detected. Depending on the outcome of the perception, we divide the issue into two sub-issues: 1) all objects in the configuration are detected; 2) only a subset of objects are correctly detected. For the first case, we use notions from geometry and static equilibrium in classical mechanics to automatically analyze and extract act and support relations between pairs of objects. For the second case, we use machine learning techniques to estimate the probability of objects supporting each other. Having the support relations extracted, a decision making process is used to identify which object to remove from the configuration so that an expected minimum cost is optimized. The proposed methods have been extensively tested and validated on data sets generated in simulation and from real world configurations for the scenario of unloading goods from shipping containers.

An important aspect of the navigation of mobile robots is the avoidance of static and dynamic obstacles. This paper deals with obstacle avoidance using artificial potential fields and selected traffic rules. The potential field method is optimized by a mixture of fuzzy methods and market-based optimization (MBO) between competing potential fields of mobile robots. Here, depending on the local situation, some potential fields are strengthened and some are weakened. The optimization takes place especially when several mobile robots act in a small area. In addition, to avoid an undesired behavior of the mobile platform in the vicinity of obstacles, central symmetrical potential fields are `deformed' by using fuzzy rules.

Mobile robots play an increasing role in everyday life, be it for industrial purposes, military missions, or for health care and for the support of handicapped people. A prominent aspect is the multi-robot planning, and autonomous navigation of a team of mobile robots, especially the avoidance of static and dynamic obstacles. The present paper deals with obstacle avoidance using artificial potential fields and selected traffic rules. As a novelty, the potential field method is enhanced by a decentralized market-based optimization (MBO) between competing potential fields of mobile robots. Some potential fields are strengthened and others are weakened depending on the local situation. In addition to that, circular potential fields are ’deformed’ by using fuzzy rules to avoid an undesired behavior of a robot in the vicinity of obstacles.

A comparison of particle swarm optimisation (PSO) and market-based optimisation (MBO) is presented when applied to obstacle avoidance by mobile robots using artificial potential fields and special traffic rules. Most notably, PSO and MBO are applied to optimise the motion of mobile robots when acting in a common confined workspace. Simulation results show that both methods perform equally well with slight advantage for PSO.

This paper addresses the safe navigation of multiple nonholonomic mobile robots in shared areas. Obstacle avoidance for mobile robots is performed by artificial potential fields and special traffic rules. In addition, the behavior of mobile robots is optimized by particle swarm optimization (PSO). The control of non-holonomic vehicles is performed using the virtual leader principle together with a local linear controller.

Human-robot interaction and robot-robot interaction and cooperation in shared spatial areas is a challenging field of research regarding safety, stability and performance. In this paper the collision avoidance between human and robot by extrapolation of human intentions and a suitable optimization of tracking velocities is discussed. Furthermore for robot-robot interactions in a shared area traffic rules and artificial force potential fields and their optimization by market-based approach are applied for obstacle avoidance. For testing and verification, the navigation strategy is implemented and tested in simulation of more realistic vehicles. Extensive simulation experiments are performed to examine the improvement of the traditional potential field (PF) method by the MBO strategy.