To Örebro University

This paper presents the design and implementation of the software for a run-time assurance infrastructure in the E-care home system. An experimental evaluation is conducted to verify that the run-time assurance infrastructure is functioning correctly, and to enable detecting performance degradation in experimental IoT network deployments within the context of E-care home.

Smart home environments have a significant potential to provide for long-term monitoring of users with special needs in order to promote the possibility to age at home. Such environments are typically equipped with a number of heterogeneous sensors that monitor both health and environmental parameters. This paper presents a framework called E-care@home, consisting of an IoT infrastructure, which provides information with an unambiguous, shared meaning across IoT devices, end-users, relatives, health and care professionals and organizations. We focus on integrating measurements gathered from heterogeneous sources by using ontologies in order to enable semantic interpretation of events and context awareness. Activities are deduced using an incremental answer set solver for stream reasoning. The paper demonstrates the proposed framework using an instantiation of a smart environment that is able to perform context recognition based on the activities and the events occurring in the home.

The demand for smart home technologies that enable ageingin place is rising. Through activity recognition, users’ activities can be monitored. However, for dementia patients, activity recognition alone cannot address the challenges associated with changes in the user’s habits along the disease’s stage transitions. Extending activity recognition to habit recognition enables the capturing of patients’ habits and change sin habits in order to detect anomalies. This paper aims to introduce relevant features for habit recognition solutions, extracted from data, in order to enrich the representation of the user’s habits. This solution is personalisable to meet the specific needs of the patients and generalizable for use in different scenarios. In this way caregivers are better informed on the expected changes of the patient’s habits, which can help to mitigate further deterioration through early treatment and intervention.

The way in which humans behave, speak andinteract is deeply influenced by their culture. For example,greeting is done differently in France, in Sweden or in Japan;and the average interpersonal distance changes from onecultural group to the other. In order to successfully coexistwith humans, robots should also adapt their behavior to theculture, customs and manners of the persons they interact with.In this paper, we deal with an important ingredient of culturaladaptation: how to generate robot plans that respect givencultural preferences, and how to execute them in a way thatis sensitive to those preferences. We present initial results inthis direction in the context of the CARESSES project, a jointEU-Japan effort to build culturally competent assistive robots.

As more robots and sensors are deployed in work and home environments, there is a growing need for these devices to act with some degree of autonomy to fulfill their purpose. Automated planning can be used to synthesize plans of action that achieve this. The main challenge addressed in this thesis is to consider how the automated planning problem changes when considered in the context of environments that are populated by humans. Humans have their own plans, and automatically generated plans should not interfere with these. We refer to this as social acceptability. Opportunities for proactive behavior often arise during execution. The planner should be able to identify these opportunities and proactively plan accordingly. Both social acceptability and proactivity require the planner to identify relevant situations from available information. We refer to this capability as context-awareness, and it may require complex inferences based on observed human activities. Finally, planning may have to consider cooperation with humans to reach common goals or to enable robots and humans to support one another.

This thesis analyzes the requirements that emerge from human-aware planning — what it takes to make automated planning socially acceptable, proactive, context aware, and to make it support cooperation with humans. We formally state the human-aware planning problem, and propose a planning and execution framework for human-aware planning that is based on constraint reasoning and flaw-resolution techniques, and which fulfills the identified requirements. This approach is modular and extendable: new types of constraints can be added and solvers can be exchanged and re-arranged. This allows us to address the identified requirements for humanaware planning. In particular, we introduce Interaction Constraints (ICs) for this purpose, and propose patterns of Ics for social acceptability, proactivity, and contextawareness. We also consider cooperative plans in which certain actions are assigned to humans and the implications that this has. We evaluate the proposed methods and patterns on a series of use cases, as well as a variety of domains including a real-world robotic system. We evaluate the proposed methods and patterns on a series of use cases, as well as a variety of domains including a real-world robotic system. introduce Interaction Constraints (ICs) for this purpose, and propose patterns of ICs for social acceptability, proactivity, and context-awareness. We also consider cooperative plans in which certain actions are assigned to humans and the implications that this has. We evaluate the proposed methods and patterns on a series of use cases, as well as a variety of domains including a real-world robotic system.

We describe an approach for integrating ontologies with a constraint-based planner to compile configuration planning domains based on the current context. We consider two alternative approaches: The first one integrates SPARQL queries directly with the planner while the second one generates SPARQL queries dynamically from provided triples. The first approach offers the full freedom of the SPARQL query language, while the second offers a more dynamic way for the planner to influence queries based on what is currently relevant for the planner. We evaluate the approach based on how much redundancy is removed by “outsourcing” knowledge into the ontology compared to modeling it directly into the domain of the planner.

As research in smart homes and activity recognition is increasing, it is of ever increasing importance to have benchmarks systems and data upon which researchers can compare methods. While synthetic data can be useful for certain method developments, real data sets that are open and shared are equally as important. This paper presents the E-care@home system, its installation in a real home setting, and a series of data sets that were collected using the E-care@home system. Our first contribution, the E-care@home system, is a collection of software modules for data collection, labeling, and various reasoning tasks such as activity recognition, person counting, and configuration planning. It supports a heterogeneous set of sensors that can be extended easily and connects collected sensor data to higher-level Artificial Intelligence (AI) reasoning modules. Our second contribution is a series of open data sets which can be used to recognize activities of daily living. In addition to these data sets, we describe the technical infrastructure that we have developed to collect the data and the physical environment. Each data set is annotated with ground-truth information, making it relevant for researchers interested in benchmarking different algorithms for activity recognition.

Behavior Trees (BTs) are a formalism increasingly used to control the execution of robotic systems. The strength of BTs resides in their compact, hierarchical and transparent representation. However, when used in practical applications transparency is often hindered by the introduction of implicit run-time relations between nodes, e.g., because of data dependencies or hardware-related ordering constraints. Manually verifying the correctness of a BT with respect to these hidden relations is a tedious and error-prone task. This paper presents a modular planning-based approach for automatically testing BTs offline at design time, to identify possible executions that may violate given data and ordering constraints and to exhibit traces of these executions to help debugging. Our approach supports both basic and advanced BT node types, e.g., supporting parallel behaviors, and can be extended with other node types as needed. We evaluate our approach on BTs used in a commercially deployed robotics system and on a large set of randomly generated trees showing that our approach scales to realistic sizes of more than 3000 nodes. 

In this position paper we argue for moving towards generalpurpose domains to promote the usage of task planning forreal-world robot systems. Planning approaches should extractconcrete domains based on their current context in order tosolve problems. Towards this aim, we define the problem ofdomain reasoning, by which a planning domain is obtainedfrom a more general, multi-purpose domain definition, giventhe current deployment and context of the robot system. Weprovide examples motivating the need for domain reasoningin robot task planning, as well as a discussion of potentialsolutions to the domain reasoning problem.

The real-world applicability of automated planners depends on the  expressiveness of the problem modeling language.  Contemporary  planners can deal with causal features of the problem, but only  limited forms of temporal, resource and relational constraints.  These constraints should be fully supported for dealing with  real-world applications.  We propose a highly-expressive, action-based planning language which  includes causal, relational, temporal and resource constraints.  This paper also contributes an approach for solving such rich  planning problems by decomposition and constraint reasoning.  The approach is general with respect to the types of constraints  used in the problem definition language, in that additional solvers  need only satisfy certain formal properties. The approach is  evaluated on a domain which utilizes many features offered by the  introduced language.

Consider a family whose home is equipped with several service robots. The actions planned for the robots must adhere to {\em Interaction Constraints (ICs)} relating them to human activities and preferences. These constraints must be sufficiently expressive to model both temporal and logical dependencies among robot actions and human behavior, and must accommodate incomplete information regarding human activities. In this paper we introduce an approach for automatically generating plans that are conformant wrt.~given ICs and partially specified human activities. The approach allows to separate causal reasoning about actions from reasoning about \ICs, and we illustrate the computational advantage this brings with experiments on a large-scale (semi-)realistic household domain with hundreds of human activities and several robots.

Planning for robots in environments co-inhabited by humans entails handling exogenous events during plan execution. Such events require plans to be continuously adapted to ensure that they remain "human-aware", i.e., adherent to human preferences and needs. We use an approach whereby human-awareness is enforced through so-called interaction constraints. Interaction constraints are used to infer context and appropriate goals online. The current plan is modified at run time so as to achieve courses of action that are continuously human-aware. The approach is evaluated in a research facility environment in which we simulate multiple days of planning and execution.

The main contribution of this paper is a planning language that can handle temporal constraints, resources and background knowledge. We provide a solver for this language based on problem decomposition that uses constraint satisfaction problems (CSPs) as a common ground. We argue that the usage of more expressive languages not only allows a more direct modeling of planning domains, but can speed up the planning process as well. We also present an experiment in support of that argument.

This paper describes the design and implementationof a network scheduler prototype for IoT networks within the e-healthcare domain. The network scheduler combines a constraint-based task planner with the Lightweight Machine-to-Machine (LwM2M) protocol to be able to reconfigure IoT networks at run-time based on recognized activities and changes in the environment. To support such network scheduling, we implement a LwM2M application layer for the IoT devices that provides sensor data, network stack information, and a set of controllable parameters that affect the communication performance and the energy consumption.

In many cases, complex multidisciplinary research projects may show a lack of coordinated development and integration, and a big effort is often required in the final phase of the projects in order to merge software developed by heterogeneous research groups. This is particularly true in advanced robotic projects: the objective here is to deliver a system that integrates all the hardware and software components, is capable of autonomous behaviour, and needs to be deployed in real-world scenarios toward providing an impact on future research and, ultimately, on society. On the other hand, in recent years there has been a growing interest for techniques related to software integration, but these have been mostly applied to the IT commercial domain.

This paper presents the work performed in the context of the project CARESSES, a multidisciplinary research project focusing on socially assistive robotics that involves 9 partners from the EU and Japan. Given the complexity of the project, a huge importance has been placed on software integration, task planning and architecture definition since the first stages of the work: to this aim, some of the practices commonly used in the commercial domain for software integration, such as merging software from the early stage, have been applied. As a case study, the document describes the steps which have been followed in the first year of the project discussing strengths and weaknesses of this approach.

Context-recognition and activity recognition systems in multi-user environments such as smart homes, usually assume to know the number of occupants in the environment. However, being able to count the number of users in the environment is important in order to accurately recognize the activities of (groups of) agents. For smart environments without cameras, the problem of counting the number of agents is non-trivial. This is in part due to the difficulty of using a single non-vision based sensors to discriminate between one or several persons, and thus information from several sensors must be combined in order to reason about the presence of several agents. In this paper we address the problem of counting the number of agents in a topologically known environment using simple sensors that can indicate anonymous human presence. To do so, we connect an ontology to a probabilistic model (a Hidden Markov Model) in order to estimate the number of agents in each section of the environment. We evaluate our methods on a smart home setup where a number of motion and pressure sensors are distributed in various rooms of the home.

Smart home environments equipped with distributed sensor networks are capable of helping people by providing services related to health, emergency detection or daily routine management. A backbone to these systems relies often on the system’s ability to track and detect activities performed by the users in their home. Despite the continuous progress in the area of activity recognition in smart homes, many systems make a strong underlying assumption that the number of occupants in the home at any given moment of time is always known. Estimating the number of persons in a Smart Home at each time step remains a challenge nowadays. Indeed, unlike most (crowd) counting solution which are based on computer vision techniques, the sensors considered in a Smart Home are often very simple and do not offer individually a good overview of the situation. The data gathered needs therefore to be fused in order to infer useful information. This paper aims at addressing this challenge and presents a probabilistic approach able to estimate the number of persons in the environment at each time step. This approach works in two steps: first, an estimate of the number of persons present in the environment is done using a Constraint Satisfaction Problem solver, based on the topology of the sensor network and the sensor activation pattern at this time point. Then, a Hidden Markov Model refines this estimate by considering the uncertainty related to the sensors. Using both simulated and real data, our method has been tested and validated on two smart homes of different sizes and configuration and demonstrates the ability to accurately estimate the number of inhabitants.

Safety applications require fast, precise and highly reliable sensors at low costs. This paperpresents signal processing methods for an active multispectral optical point sensorinstrumentation for which a first technical implementation exists. Due to the very demandingrequirements for safeguarding equipment, these processing methods are targeted to run on asmall embedded system with a guaranteed reaction time T < 2 ms and a sufficiently lowfailure rate according to applicable safety standards, e.g., ISO-13849. The proposed dataprocessing concept includes a novel technique for distance-aided fusion of multispectral datain order to compensate for displacement-related alteration of the measured signal. Thedistance measuring is based on triangulation with precise results even for low-resolutiondetectors, thus strengthening the practical applicability. Furthermore, standard components,such as support vector machines (SVMs), are used for reliable material classification. Allmethods have been evaluated for variants of the underlying sensor principle. Therefore, theresults of the evaluation are independent of any specific hardware.