To Örebro University

If a non-expert wants to program a robot manipulator he needs a natural interface that does not require rigorous robot programming skills. Programming-by-demonstration (PbD) is an approach which enables the user to program a robot by simply showing the robot how to perform a desired task. In this approach, the robot recognizes what task it should perform and learn how to perform it by imitating the teacher. One fundamental problem in imitation learning arises from the fact that embodied agents often have different morphologies. Thus, a direct skill transfer from human to a robot is not possible in the general case. Therefore, we need a systematic approach to PbD that takes the capabilities of the robot into account–regarding both perception and body structure. In addition, the robot should be able to learn from experience and improve over time. This raises the question of how to determine the demonstrator’s goal or intentions. We show that this is possible–to some degree–to infer from multiple demonstrations. We address the problem of generation of a reach-to-grasp motion that produces the same results as a human demonstration. It is also of interest to learn what parts of a demonstration provide important information about the task. The major contribution is the investigation of a next-state-planner using a fuzzy time-modeling approach to reproduce a human demonstration on a robot. We show that the proposed planner can generate executable robot trajectories based on a generalization of multiple human demonstrations. We use the notion of hand-states as a common motion language between the human and the robot. It allows the robot to interpret the human motions as its own, and it also synchronizes reaching with grasping. Other contributions include the model-free learning of human to robot mapping, and how an imitation metric ca be used for reinforcement learning of new robot skills. The experimental part of this thesis presents the implementation of PbD of pick-and-place-tasks on different robotic hands/grippers. The different platforms consist of manipulators and motion capturing devices.

This paper presents a novel approach to skill modeling acquired from human demonstration. The approach is based on fuzzy modeling and is using a planner for generating corresponding robot trajectories. One of the main challenges stems from the morphological differences between human and robot hand/arm structure, which makes direct copying of human motions impossible in the general case. Thus, the planner works in hand state space, which is defined such that it is perception-invariant and valid for both human and robot hand. We show that this representation simplifies task reconstruction and preserves the essential parts of the task as well as the coordination between reaching and grasping motion. We also show how our approach can generalize observed trajectories based on multiple demonstrations and that the robot can match a demonstrated behavoir, despite morphological differences. To validate our approach we use a general-purpose robot manipulator equipped with an anthropomorphic three-fingered robot hand.

We present a system which includes an under-actuated anthropomorphic hand and control algorithms for autonomous grasping of everyday objects. The system comprised a control framework for hybrid force/position control in simulation and reality, a grasp simulator, and an under-actuated robot hand equipped with tactile sensors.We start by presenting the robot hand, the simulation environment and the control framework that enable dynamic simulation of an under-actuated robot hand. We continue by presenting simulation results and also discuss and exemplify the use of simulation in relation to autonomous grasping. Finally, we use the very same controller in real world grasping experiments to validate the simulations and to exemplify system capabilities and limitations.

In this paper we present an approach to reproduce human demonstrations in a reach-to-grasp context. The demonstration is represented in hand state space. By using the distance to the target object as a scheduling variable, the way in which the robot approaches the object is controlled. The controller that we deploy to execute the motion is formulated as a nextstateplanner. The planner produces an action from the current state instead of planning the whole trajectory in advance which can be error prone in non-static environments. The results have a direct application in Programming-by-Demonstration. It also contributes to cognitive systems since the ability to reach-tograsp supports the development of cognitive abilities.

This article presents an approach to Programming by Demonstration (PbD) to simplify programming of industrial manipulators. By using a set of task primitives for a known task type, the demonstration is interpreted and a manipulator program is automatically generated. A pick-and-place task is analyzed, based on the velocity profile, and decomposed in task primitives. Task primitives are basic actions of the robot/gripper, which can be executed in a sequence to form a complete a task. For modeling and generation of the demonstrated trajectory, fuzzy time clustering is used, resulting in smooth and accurate motions. To illustrate our approach, we carried out our experiments on a real industrial manipulator.

In this paper we propose a system consisting of a manipulator equipped with range sensors, that is instructed to follow a trajectory demonstrated by a human teacher wearing a motion capturing device. During the demonstration a three dimensional occupancy grid of the environment is built using the range sensor information and the trajectory. The demonstration is followed by an exploration phase, where the robot undergoes self-improvement of the task, during which the occupancy grid is used to avoid collisions. In parallel a reinforcement learning (RL) agent, biased by the demonstration, learns a point-to-point task policy. When changes occur in the workspace, both the occupancy grid and the learned policy will be updated online by the system.

This thesis address how robotic arms, called manipulators, can learn a task demonstrated by a teacher. The concept of showing a robot a task, instead of manually programming it, is appealing since it makes it easier to instruct robots. This thesis will introduce the basics of manipulators and techniques suitable for robot learning including an introduction to reinforcement learning. Also a number of other researchers' work are reviewed from the viewpoint of how they apply robot learning from a teacher, and how this knowledge can be reused when a similar problem is faced. One key part of this thesis is an overview of the field Robot Learning from Demonstration, focusing on robotic manipulators, but work including humanoids and mobile robots are also covered. Challenges, such as how to learn from the demonstration, and what to learn are presented together with related work. Initial experiments on learning from a teacher's demonstration, have been carried out using a manipulator and a motion capturing device as a platform. The experiments investigated are

position teaching of a robotic arm using neural networks and a minimum distance classifier,

reinforcement learning algorithm for a reaching task, where a demonstrated trajectory was used as bias.

Based on the presented work we suggest a future work direction and that provide the robot with some basic behaviours needed to learn other higher level tasks.

Having a robot that can learn from and improve upon a human demonstration is a challenge for robotic scientists, and useful for non-engineers who want a robotic assistant to perform a particular task. In this paper we address some of the difficulties one will have to overcome when developing such a system for an articulated manipulator with more degrees-offreedom (d.o.f.) than most mobile robots on wheels. Making a good data capture of what is shown to the robot is one such problem. Another key scientific challenge is the curse of dimensionality that arises from the high dimensional state and action spaces in this application, which we propose to address by combination of supervised and reinforcement learning to gain benefits from both paradigms. We also point out that one has to be careful when trying to obtain an agent that learns a task in as few trials as possible, since it might require much more computational time.