Robust perception is an essential component to enable long-term operation of mobile robots. It depends on failure resilience through reliable sensor data and pre-processing, as well as failure awareness through introspection, for example the ability to self-assess localization performance. This paper presents CorAl: a principled, intuitive, and generalizable method to measure the quality of alignment between pairs of point clouds, which learns to detect alignment errors in a self-supervised manner. CorAl compares the differential entropy in the point clouds separately with the entropy in their union to account for entropy inherent to the scene. By making use of dual entropy measurements, we obtain a quality metric that is highly sensitive to small alignment errors and still generalizes well to unseen environments. In this work, we extend our previous work on lidar-only CorAl to radar data by proposing a two-step filtering technique that produces high-quality point clouds from noisy radar scans. Thus, we target robust perception in two ways: by introducing a method that introspectively assesses alignment quality, and by applying it to an inherently robust sensor modality. We show that our filtering technique combined with CorAl can be applied to the problem of alignment classification, and that it detects small alignment errors in urban settings with up to 98% accuracy, and with up to 96% if trained only in a different environment. Our lidar and radar experiments demonstrate that CorAl outperforms previous methods both on the ETH lidar benchmark, which includes several indoor and outdoor environments, and the large-scale Oxford and MulRan radar data sets for urban traffic scenarios. The results also demonstrate that CorAl generalizes very well across substantially different environments without the need of retraining.
This paper targets high-precision robot localization. We address a general problem for voxel-based map representations that the expressiveness of the map is fundamentally limited by the resolution since integration of measurements taken from different perspectives introduces imprecisions, and thus reduces localization accuracy.We propose SuPer maps that contain one Submap per Perspective representing a particular view of the environment. For localization, a robot then selects the submap that best explains the environment from its perspective. We propose SuPer mapping as an offline refinement step between initial SLAM and deploying autonomous robots for navigation. We evaluate the proposed method on simulated and real-world data that represent an important use case of an industrial scenario with high accuracy requirements in an repetitive environment. Our results demonstrate a significantly improved localization accuracy, up to 46% better compared to localization in global maps, and up to 25% better compared to alternative submapping approaches.
This paper presents the accurate, highly efficient, and learning-free method CFEAR Radarodometry for large-scale radar odometry estimation. By using a filtering technique that keeps the k strongest returns per azimuth and by additionally filtering the radar data in Cartesian space, we are able to compute a sparse set of oriented surface points for efficient and accurate scan matching. Registration is carried out by minimizing a point-to-line metric and robustness to outliers is achieved using a Huber loss. We were able to additionally reduce drift by jointly registering the latest scan to a history of keyframes and found that our odometry method generalizes to different sensor models and datasets without changing a single parameter. We evaluate our method in three widely different environments and demonstrate an improvement over spatially cross-validated state-of-the-art with an overall translation error of 1.76% in a public urban radar odometry benchmark, running at 55Hz merely on a single laptop CPU thread.
This article presents an accurate, highly efficient, and learning-free method for large-scale odometry estimation using spinning radar, empirically found to generalize well across very diverse environments—outdoors, from urban to woodland, and indoors in warehouses and mines—without changing parameters. Our method integrates motion compensation within a sweep with one-to-many scan registration that minimizes distances between nearby oriented surface points and mitigates outliers with a robust loss function. Extending our previous approach conservative filtering for efficient and accurate radar odometry (CFEAR), we present an in-depth investigation on a wider range of datasets, quantifying the importance of filtering, resolution, registration cost and loss functions, keyframe history, and motion compensation. We present a new solving strategy and configuration that overcomes previous issues with sparsity and bias, and improves our state-of-the-art by 38%, thus, surprisingly, outperforming radar simultaneous localization and mapping (SLAM) and approaching lidar SLAM. The most accurate configuration achieves 1.09% error at 5 Hz on the Oxford benchmark, and the fastest achieves 1.79% error at 160 Hz.
This paper presents an efficient and accurate radar odometry pipeline for large-scale localization. We propose a radar filter that keeps only the strongest reflections per-azimuth that exceeds the expected noise level. The filtered radar data is used to incrementally estimate odometry by registering the current scan with a nearby keyframe. By modeling local surfaces, we were able to register scans by minimizing a point-to-line metric and accurately estimate odometry from sparse point sets, hence improving efficiency. Specifically, we found that a point-to-line metric yields significant improvements compared to a point-to-point metric when matching sparse sets of surface points. Preliminary results from an urban odometry benchmark show that our odometry pipeline is accurate and efficient compared to existing methods with an overall translation error of 2.05%, down from 2.78% from the previously best published method, running at 12.5ms per frame without need of environmental specific training.
In robotics perception, numerous tasks rely on point cloud registration. However, currently there is no method that can automatically detect misaligned point clouds reliably and without environment-specific parameters. We propose "CorAl", an alignment quality measure and alignment classifier for point cloud pairs, which facilitates the ability to introspectively assess the performance of registration. CorAl compares the joint and the separate entropy of the two point clouds. The separate entropy provides a measure of the entropy that can be expected to be inherent to the environment. The joint entropy should therefore not be substantially higher if the point clouds are properly aligned. Computing the expected entropy makes the method sensitive also to small alignment errors, which are particularly hard to detect, and applicable in a range of different environments. We found that CorAl is able to detect small alignment errors in previously unseen environments with an accuracy of 95% and achieve a substantial improvement to previous methods.
This paper presents a new detector for filtering noise from true detections in radar data, which improves the state of the art in radar odometry. Scanning Frequency-Modulated Continuous Wave (FMCW) radars can be useful for localisation and mapping in low visibility, but return a lot of noise compared to (more commonly used) lidar, which makes the detection task more challenging. Our Bounded False-Alarm Rate (BFAR) detector is different from the classical Constant False-Alarm Rate (CFAR) detector in that it applies an affine transformation on the estimated noise level after which the parameters that minimize the estimation error can be learned. BFAR is an optimized combination between CFAR and fixed-level thresholding. Only a single parameter needs to be learned from a training dataset. We apply BFAR tothe use case of radar odometry, and adapt a state-of-the-art odometry pipeline (CFEAR), replacing its original conservative filtering with BFAR. In this way we reduce the state-of-the-art translation/rotation odometry errors from 1.76%/0.5◦/100 m to 1.55%/0.46◦/100 m; an improvement of 12.5%.
Autonomous systems, that need to operate in human environments and interact with the users, rely on understanding and anticipating human activity and motion. Among the many factors which influence human motion, semantic attributes, such as the roles and ongoing activities of the detected people, provide a powerful cue on their future motion, actions, and intentions. In this work we adapt several popular deep learning models for trajectory prediction with labels corresponding to the roles of the people. To this end we use the novel THOR-Magni dataset, which captures human activity in industrial settings and includes the relevant semantic labels for people who navigate complex environments, interact with objects and robots, work alone and in groups. In qualitative and quantitative experiments we show that the role-conditioned LSTM, Transformer, GAN and VAE methods can effectively incorporate the semantic categories, better capture the underlying input distribution and therefore produce more accurate motion predictions in terms of Top-K ADE/FDE and log-likelihood metrics.
Matching and merging overlapping point clouds is a common procedure in many applications, including mobile robotics, three-dimensional mapping, and object visualization. However, fully automatic point-cloud matching, without manual verification, is still not possible because no matching algorithms exist today that can provide any certain methods for detecting misaligned point clouds. In this article, we make a comparative evaluation of geometric consistency methods for classifying aligned and nonaligned point-cloud pairs. We also propose a method that combines the results of the evaluated methods to further improve the classification of the point clouds. We compare a range of methods on two data sets from different environments related to mobile robotics and mapping. The results show that methods based on a Normal Distributions Transform representation of the point clouds perform best under the circumstances presented herein.
We present a perception system for enabling automated loading with waist-articulated wheel loaders. To enable autonomous loading of piled materials, using either above-ground wheel loaders or underground load-haul-dump vehicles, 3D data of the pile shape is needed. However, using common 3D scanners, the scan data is distorted while the wheel loader is moving towards the pile. Existing methods that make use of 3D scan data (for autonomous loading as well as tasks such as mapping, localisation, and object detection) typically assume that each 3D scan is accurate. For autonomous robots moving over rough terrain, it is often the case that the vehicle moves a substantial amount during the acquisition of one 3D scan, in which case the scan data will be distorted. We present a study of auto-loading methods, and how to locate piles in real-world scenarios with nontrivial ground geometry. We have compared how consistently each method performs for live scans acquired in motion, and also how the methods perform with different view points and scan configurations. The system described in this paper uses a novel method for improving the quality of distorted 3D scans made from a vehicle moving over uneven terrain. The proposed method for improving scan quality is capable of increasing the accuracy of point clouds without assuming any specific features of the environment (such as planar walls), without resorting to a “stop-scan-go” approach, and without relying on specialised and expensive hardware. Each new 3D scan is registered to the preceding using the normal-distributions transform (NDT). After each registration, a mini-loop closure is performed with a local, per-scan, graph-based SLAM method. To verify the impact of the quality improvement, we present data that shows how auto-loading methods benefit from the corrected scans. The presented methods are validated on data from an autonomous wheel loader, as well as with simulated data. The proposed scan-correction method increases the accuracy of both the vehicle trajectory and the point cloud. We also show that it increases the reliability of pile-shape measures used to plan an efficient attack pose when performing autonomous loading.
This paper presents a method for improving the quality of distorted 3D point clouds made from a vehicle equipped with a laser scanner moving over uneven terrain. Existing methods that use 3D point-cloud data (for tasks such as mapping, localisation, and object detection) typically assume that each point cloud is accurate. For autonomous robots moving in rough terrain, it is often the case that the vehicle moves a substantial amount during the acquisition of one point cloud, in which case the data will be distorted. The method proposed in this paper is capable of increasing the accuracy of 3D point clouds, without assuming any specific features of the environment (such as planar walls), without resorting to a "stop-scan-go" approach, and without relying on specialised and expensive hardware. Each new point cloud is matched to the previous using normal-distribution-transform (NDT) registration, after which a mini-loop closure is performed with a local, per-scan, graph-based SLAM method. The proposed method increases the accuracy of both the measured platform trajectory and the point cloud. The method is validated on both real-world and simulated data.
Local scan registration approaches commonlyonly utilize ego-motion estimates (e.g. odometry) as aninitial pose guess in an iterative alignment procedure. Thispaper describes a new method to incorporate ego-motionestimates, including uncertainty, into the objective function of aregistration algorithm. The proposed approach is particularlysuited for feature-poor and self-similar environments,which typically present challenges to current state of theart registration algorithms. Experimental evaluation showssignificant improvements in accuracy when using data acquiredby Automatic Guided Vehicles (AGVs) in industrial productionand warehouse environments.
In this article, we address the problem of realizing a complete efficient system for automated management of fleets of autonomous ground vehicles in industrial sites. We elicit from current industrial practice and the scientific state of the art the key challenges related to autonomous transport vehicles in industrial environments and relate them to enabling techniques in perception, task allocation, motion planning, coordination, collision prediction, and control. We propose a modular approach based on least commitment, which integrates all modules through a uniform constraint-based paradigm. We describe an instantiation of this system and present a summary of the results, showing evidence of increased flexibility at the control level to adapt to contingencies.
This paper presents a vision-based approach to SLAM in indoor / outdoor environments with minimalistic sensing and computational requirements. The approach is based on a graph representation of robot poses, using a relaxation algorithm to obtain a globally consistent map. Each link corresponds to a relative measurement of the spatial relation between the two nodes it connects. The links describe the likelihood distribution of the relative pose as a Gaussian distribution. To estimate the covariance matrix for links obtained from an omni-directional vision sensor, a novel method is introduced based on the relative similarity of neighbouring images. This new method does not require determining distances to image features using multiple view geometry, for example. Combined indoor and outdoor experiments demonstrate that the approach can handle qualitatively different environments (without modification of the parameters), that it can cope with violations of the “flat floor assumption” to some degree, and that it scales well with increasing size of the environment, producing topologically correct and geometrically accurate maps at low computational cost. Further experiments demonstrate that the approach is also suitable for combining multiple overlapping maps, e.g. for solving the multi-robot SLAM problem with unknown initial poses.
This paper presents a vision-based approach to SLAM in large-scale environments with minimal sensing and computational requirements. The approach is based on a graphical representation of robot poses and links between the poses. Links between the robot poses are established based on odometry and image similarity, then a relaxation algorithm is used to generate a globally consistent map. To estimate the covariance matrix for links obtained from the vision sensor, a novel method is introduced based on the relative similarity of neighbouring images, without requiring distances to image features or multiple view geometry. Indoor and outdoor experiments demonstrate that the approach scales well to large-scale environments, producing topologically correct and geometrically accurate maps at minimal computational cost. Mini-SLAM was found to produce consistent maps in an unstructured, large-scale environment (the total path length was 1.4 km) containing indoor and outdoor passages.
This paper describes a vision and 3D laser based registration approach which utilizes visual features to identify correspondences. Visual features are obtained from the images of a standard color camera and the depth of these features is determined by interpolating between the scanning points of a 3D laser range scanner, taking into consideration the visual information in the neighbourhood of the respective visual feature. The 3D laser scanner is also used to determine a position covariance estimate of the visual feature. To exploit these covariance estimates, an ICP algorithm based on the Mahalanobis distance is applied. Initial experimental results are presented in a real world indoor laboratory environment
This paper describes a novel registration approach that is based on a combination of visual and 3D range information.To identify correspondences, local visual features obtained from images of a standard color camera are compared and the depth of matching features (and their position covariance) is determined from the range measurements of a 3D laserscanner. The matched depth-interpolated image features allows to apply registration with known correspondences.We compare several ICP variants in this paper and suggest an extension that considers the spatial distance betweenmatching features to eliminate false correspondences. Experimental results are presented in both outdoor and indoor environments. In addition to pair-wise registration, we also propose a global registration method that registers allscan poses simultaneously.
3D range sensors, particularly 3D laser range scanners, enjoy a rising popularity and are used nowadays for many different applications. The resolution 3D range sensors provide in the image plane is typically much lower than the resolution of a modern color camera. In this paper we focus on methods to derive a high-resolution depth image from a low-resolution 3D range sensor and a color image. The main idea is to use color similarity as an indication of depth similarity, based on the observation that depth discontinuities in the scene often correspond to color or brightness changes in the camera image. We present five interpolation methods and compare them with an independently proposed method based on Markov Random Fields. The algorithms proposed in this paper are non-iterative and include a parameter-free vision-based interpolation method. In contrast to previous work, we present ground truth evaluation with real world data and analyse both indoor and outdoor data. Further, we suggest and evaluate four methods to determine a confidence measure for the accuracy of interpolated range values.
This paper presents a system for autonomous change detection with a security patrol robot. In an initial step a reference model of the environment is created and changes are then detected with respect to the reference model as differences in coloured 3D point clouds, which are obtained from a 3D laser range scanner and a CCD camera. The suggested approach introduces several novel aspects, including a registration method that utilizes local visual features to determine point correspondences (thus essentially working without an initial pose estimate) and the 3D-NDT representation with adaptive cell size to efficiently represent both the spatial and colour aspects of the reference model. Apart from a detailed description of the individual parts of the difference detection system, a qualitative experimental evaluation in an indoor lab environment is presented, which demonstrates that the suggested system is able register and detect changes in spatial 3D data and also to detect changes that occur in colour space and are not observable using range values only.
Autonomous navigation in real-world industrial environments is a challenging task in many respects. One of the key open challenges is fast planning and execution of trajectories to reach arbitrary target positions and orientations with high accuracy and precision, while taking into account non-holonomic vehicle constraints. In recent years, lattice-based motion planners have been successfully used to generate kinematically and kinodynamically feasible motions for non-holonomic vehicles. However, the discretized nature of these algorithms induces discontinuities in both state and control space of the obtained trajectories, resulting in a mismatch between the achieved and the target end pose of the vehicle. As endpose accuracy is critical for the successful loading and unloading of cargo in typical industrial applications, automatically planned paths have not been widely adopted in commercial AGV systems. The main contribution of this paper is a path smoothing approach, which builds on the output of a lattice-based motion planner to generate smooth drivable trajectories for non-holonomic industrial vehicles. The proposed approach is evaluated in several industrially relevant scenarios and found to be both fast (less than 2 s per vehicle trajectory) and accurate (end-point pose errors below 0.01 m in translation and 0.005 radians in orientation).
Autonomous navigation in real-world industrial environments is a challenging task in many respects. One of the key open challenges is fast planning and execution of trajectories to reach arbitrary target positions and orientations with high accuracy and precision, while taking into account non-holonomic vehicle constraints. In recent years, lattice-based motion planners have been successfully used to generate kinematically and kinodynamically feasible motions for non-holonomic vehicles. However, the discretized nature of these algorithms induces discontinuities in both state and control space of the obtained trajectories, resulting in a mismatch between the achieved and the target end pose of the vehicle. As endpose accuracy is critical for the successful loading and unloading of cargo in typical industrial applications, automatically planned paths have not be widely adopted in commercial AGV systems. The main contribution of this paper addresses this shortcoming by introducing a path smoothing approach, which builds on the output of a lattice-based motion planner to generate smooth drivable trajectories for non-holonomic industrial vehicles. In real world tests presented in this paper we demonstrate that the proposed approach is fast enough for online use (it computes trajectories faster than they can be driven) and highly accurate. In 100 repetitions we achieve mean end-point pose errors below 0.01 meters in translation and 0.002 radians in orientation. Even the maximum errors are very small: only 0.02 meters in translation and 0.008 radians in orientation.
3D range sensors, particularly 3D laser range scanners, enjoy a rising popularity and are used nowadays for many different applications. The resolution 3D range sensors provide in the image plane is typically much lower than the resolution of a modern colour camera. In this chapter we focus on methods to derive a highresolution depth image from a low-resolution 3D range sensor and a colour image. The main idea is to use colour similarity as an indication of depth similarity, based on the observation that depth discontinuities in the scene often correspond to colour or brightness changes in the camera image. We present five interpolation methods and compare them with an independently proposed method based on Markov random fields. The proposed algorithms are non-iterative and include a parameter-free vision-based interpolation method. In contrast to previous work, we present ground truth evaluation with real world data and analyse both indoor and outdoor data.
The problem of gas detection is relevant to manyreal-world applications, such as leak detection in industrialsettings and surveillance. In this paper we address the problemof gas detection in large areas with a mobile robotic platformequipped with a remote gas sensor. We propose a novelmethod based on convex relaxation for quickly finding anexploration plan that guarantees a complete coverage of theenvironment. Our method proves to be highly efficient in termsof computational requirements and to provide nearly-optimalsolutions. We validate our approach both in simulation andin real environments, thus demonstrating its applicability toreal-world problems.
An accurate model of gas emissions is of high importance in several real-world applications related to monitoring and surveillance. Gas tomography is a non-intrusive optical method to estimate the spatial distribution of gas concentrations using remote sensors. The choice of sensing geometry, which is the arrangement of sensing positions to perform gas tomography, directly affects the reconstruction quality of the obtained gas distribution maps. In this paper, we present an investigation of criteria that allow to determine suitable sensing geometries for gas tomography. We consider an actuated remote gas sensor installed on a mobile robot, and evaluated a large number of sensing configurations. Experiments in complex settings were conducted using a state-of-the-art CFD-based filament gas dispersal simulator. Our quantitative comparison yields preferred sensing geometries for sensor planning, which allows to better reconstruct gas distributions.
Air pollution causes millions of premature deaths every year, and fugitive emissions of, e.g., methane are major causes of global warming. Correspondingly, air pollution monitoring systems are urgently needed. Mobile, autonomous monitoring can provide adaptive and higher spatial resolution compared with traditional monitoring stations and allows fast deployment and operation in adverse environments. We present a mobile robot solution for autonomous gas detection and gas distribution mapping using remote gas sensing. Our ‘‘Autonomous Remote Methane Explorer’’ (ARMEx) is equipped with an actuated spectroscopy-based remote gas sensor, which collects integral gas measurements along up to 30 m long optical beams. State-of-the-art 3D mapping and robot localization allow the precise location of the optical beams to be determined, which then facilitates gas tomography (tomographic reconstruction of local gas distributions from sets of integral gas measurements). To autonomously obtain informative sampling strategies for gas tomography, we reduce the search space for gas inspection missions by defining a sweep of the remote gas sensor over a selectable field of view as a sensing configuration. We describe two different ways to find sequences of sensing configurations that optimize the criteria for gas detection and gas distribution mapping while minimizing the number of measurements and distance traveled. We evaluated anARMExprototype deployed in a large, challenging indoor environment with eight gas sources. In comparison with human experts teleoperating the platform from a distant building, the autonomous strategy produced better gas maps with a lower number of sensing configurations and a slightly longer route.
Creating an accurate model of gas emissions is an important task in monitoring and surveillance applications. A promising solution for a range of real-world applications are gas-sensitive mobile robots with spectroscopy-based remote sensors that are used to create a tomographic reconstruction of the gas distribution. The quality of these reconstructions depends crucially on the chosen sensing geometry. In this paper we address the problem of sensor planning by investigating sensing geometries that minimize reconstruction errors, and then formulate an optimization algorithm that chooses sensing configurations accordingly. The algorithm decouples sensor planning for single high concentration regions (hotspots) and subsequently fuses the individual solutions to a global solution consisting of sensing poses and the shortest path between them. The proposed algorithm compares favorably to a template matching technique in a simple simulation and in a real-world experiment. In the latter, we also compare the proposed sensor planning strategy to the sensing strategy of a human expert and find indications that the quality of the reconstructed map is higher with the proposed algorithm.
The problem of gas detection is relevant to many real-world applications, such as leak detection in industrial settings and landfill monitoring. In this paper, we address the problem of gas detection in large areas with a mobile robotic platform equipped with a remote gas sensor. We propose an algorithm that leverages a novel method based on convex relaxation for quickly solving sensor placement problems, and for generating an efficient exploration plan for the robot. To demonstrate the applicability of our method to real-world environments, we performed a large number of experimental trials, both on randomly generated maps and on the map of a real environment. Our approach proves to be highly efficient in terms of computational requirements and to provide nearly-optimal solutions.
This paper presents a framework of ICT solutions developed in the EU research project DIADEM that supports environmental management with an enhanced capacity to assess population exposure and health risks, to alert relevant groups and to organize efficient response. The emphasis is on advanced solutions which are economically feasible and maximally exploit the existing communication, computing and sensing resources. This approach enables efficient situation assessment in complex environmental management problems by exploiting relevant information obtained from citizens via the standard communication infrastructure as well as heterogeneous data acquired through dedicated sensing systems. This is achieved through a combination of (i) advanced approaches to gas detection and gas distribution modelling, (ii) a novel service-oriented approach supporting seamless integration of human-based and automated reasoning processes in large-scale collaborative sense making processes and (iii) solutions combining Multi-Criteria Decision Analysis, Scenario-Based Reasoning and advanced human-machine interfaces. This paper presents the basic principles of the DIADEM solutions, explains how different techniques are combined to a coherent decision support system and briefly discusses evaluation principles and activities in the DIADEM project.
Artificial olfaction can help to address pressing environmental problems due to unwanted gas emissions. Sensor networks and mobile robots equipped with gas sensors can be used for e.g. air pollution monitoring. Key in this context is the ability to derive truthful models of gas distribution from a set of sparse measurements. Most statistical gas distribution modelling methods assume that gas dispersion is a time constant random process. While this assumption approximately holds in some situations, it is necessary to model variations over time in order to enable applications of gas distribution modelling in a wider range of realistic scenarios. Time-invariant approaches cannot model well evolving gas plumes, for example, or major changes in gas dispersion due to a sudden change of the environmental conditions. This paper presents two approaches to gas distribution modelling, which introduce a time-dependency and a relation to a time-scale in generating the gas distribution model either by sub-sampling or by introducing a recency weight that relates measurement and prediction time. We evaluated these approaches in experiments performed in two real environments as well as on several simulated experiments. As expected, the comparison of different sub-sampling strategies revealed that more recent measurements are more informative to derive an estimate of the current gas distribution as long as a sufficient spatial coverage is given. Next, we compared a time-dependent gas distribution modelling approach (TD Kernel DM+V), which includes a recency weight, to the state-of-the-art gas distribution modelling approach (Kernel DM+V), which does not consider sampling times. The results indicate a consistent improvement in the prediction of unseen measurements, particularly in dynamic scenarios. Furthermore, this paper discusses the impact of meta-parameters in model selection and compares the performance of time-dependent GDM in different plume conditions. Finally, we investigated how to set the target time for which the model is created. The results indicate that TD Kernel DM+V performs best when the target time is set to the maximum sampling time in the test set.
Mobile robot olfaction solutions for gas distribution modelling offer a number of advantages, among them autonomous monitoring in different environments, mobility to select sampling locations, and ability to cooperate with other systems. However, most data-driven, statistical gas distribution modelling approaches assume that the gas distribution is generated by a time-invariant random process. Such time-invariant approaches cannot model well developing plumes or fundamental changes in the gas distribution. In this paper, we discuss approaches that explicitly consider the measurement time, either by sub-sampling according to a given time-scale or by introducing a recency weight that relates measurement and prediction time. We evaluate the performance of these time-dependent approaches in simulation and in real-world experiments using mobile robots. The results demonstrate that in dynamic scenarios improved gas distribution models can be obtained with time-dependent approaches.
To study gas dispersion, several statistical gas distribution modelling approaches have been proposed recently. A crucial assumption in these approaches is that gas distribution models are learned from measurements that are generated by a time-invariant random process. While a time-independent random process can capture certain fluctuations in the gas distribution, more accurate models can be obtained by modelling changes in the random process over time. In this work we propose a time-scale parameter that relates the age of measurements to their validity for building the gas distribution model in a recency function. The parameters of the recency function define a time-scale and can be learned. The time-scale represents a compromise between two conflicting requirements for obtaining accurate gas distribution models: using as many measurements as possible and using only very recent measurements. We have studied several recency functions in a time-dependent extension of the Kernel DM+V algorithm (TD Kernel DM+V). Based on real-world experiments and simulations of gas dispersal (presented in this paper) we demonstrate that TD Kernel DM+V improves the obtained gas distribution models in dynamic situations. This represents an important step towards statistical modelling of evolving gas distributions.
Gas distribution models can provide comprehensive information about a large number of gas concentration measurements, highlighting, for example, areas of unusual gas accumulation. They can also help to locate gas sources and to plan where future measurements should be carried out. Current physical modeling methods, however, are computationally expensive and not applicable for real world scenarios with real-time and high resolution demands. This chapter reviews kernel methodsthat statistically model gas distribution. Gas measurements are treated as randomvariables, and the gas distribution is predicted at unseen locations either using akernel density estimation or a kernel regression approach. The resulting statistical
apmodelsdo not make strong assumptions about the functional form of the gas distribution,such as the number or locations of gas sources, for example. The majorfocus of this chapter is on two-dimensional models that provide estimates for themeans and predictive variances of the distribution. Furthermore, three extensionsto the presented kernel density estimation algorithm are described, which allow toinclude wind information, to extend the model to three dimensions, and to reflecttime-dependent changes of the random process that generates the gas distributionmeasurements. All methods are discussed based on experimental validation usingreal sensor data.
Recognizing patterns is an essential skill in early mathematics education. However, first graders often have difficulties with tasks such as extending patterns of the form ABCABC. Studies show that this pattern-recognition ability is a good predictor of later pre-algebraic skills and mathematical achievement in general, or the development of mathematical difficulties on the other hand. To be able to foster children's pattern-recognition ability, it is crucial to investigate and understand their pattern-recognition processes early on. However, only a few studies have investigated the processes used to recognize patterns and how these processes are adapted to different patterns. These studies used external observations or relied on children's self-reports, yet young students often lack the ability to properly report their strategies. This paper presents the results of an empirical study using eye-tracking technology to investigate the pattern-recognition processes of 22 first-grade students. In particular, we investigated students with and without the risk of developing mathematical difficulties. The analyses of the students' eye movements reveal that the students used four different processes to recognize patterns-a finding that refines knowledge about pattern-recognition processes from previous research. In addition, we found that for patterns with different units of repeat (i.e. ABABAB versus ABCABCABC), the pattern-recognition processes used differed significantly for students at risk of developing mathematical difficulties but not for students without such risk. Our study contributes to a better understanding of the pattern-recognition processes of first-grade students, laying the foundation for enhanced, targeted support, especially for students at risk of developing mathematical difficulties.
In this paper we present an inspection robot to produce gas distribution maps and localize gas sources in large outdoor environments. The robot is equipped with a 3D laser range finder and a remote gas sensor that returns integral concentration measurements. We apply principles of tomography to create a spatial gas distribution model from integral gas concentration measurements. The gas distribution algorithm is framed as a convex optimization problem and it models the mean distribution and the fluctuations of gases. This is important since gas dispersion is not an static phenomenon and furthermore, areas of high fluctuation can be correlated with the location of an emitting source. We use a compact surface representation created from the measurements of the 3D laser range finder with a state of the art mapping algorithm to get a very accurate localization and estimation of the path of the laser beams. In addition, a conic model for the beam of the remote gas sensor is introduced. We observe a substantial improvement in the gas source localization capabilities over previous state-of-the-art in our evaluation carried out in an open field environment.
Building a model of gas concentrations has important indus-trial and environmental applications, and mobile robots ontheir own or in cooperation with stationary sensors play animportant role in this task. Since an exact analytical de-scription of turbulent flow remains an intractable problem,we propose an approximate approach which not only esti-mates the concentrations but also their variances for eachlocation. Our point of view is that of sequential Bayesianestimation given a lattice of 2D cells treated as hidden vari-ables. We first discuss how a simple Kalman filter pro-vides a solution to the estimation problem. To overcomethe quadratic computational complexity with the mappedarea exhibited by a straighforward application of Kalmanfiltering, we introduce a sparse implementation which runsin constant time. Experimental results for a real robot vali-date the proposed method.
This paper presents an overview of an autonomous robotic material handling system. The goal of the system is to extend the functionalities of traditional AGVs to operate in highly dynamic environments. Traditionally, the reliable functioning of AGVs relies on the availability of adequate infrastructure to support navigation. In the target environments of our system, such infrastructure is difficult to setup in an efficient way. Additionally, the location of objects to handle are unknown, which requires that the system be able to detect and track object positions at runtime. Another requirement of the system is to be able to generate trajectories dynamically, which is uncommon in industrial AGV systems.
This paper presents an overview of an autonomousrobotic material handling system. The goal of the system is toextend the functionalities of traditional AGVs to operate in highlydynamic environments. Traditionally, the reliable functioning ofAGVs relies on the availability of adequate infrastructure tosupport navigation. In the target environments of our system,such infrastructure is difficult to setup in an efficient way.Additionally, the location of objects to handle are unknown,which requires that the system be able to detect and track objectpositions at runtime. Another requirement of the system is to beable to generate trajectories dynamically, which is uncommon inindustrial AGV systems.
Freely moving autonomous mobile robots may leadto anxiety when operating in workspaces shared with humans.Previous works have given evidence that communicating in-tentions using Spatial Augmented Reality (SAR) in the sharedworkspace will make humans more comfortable in the vicinity ofrobots. In this work, we conducted experiments with the robotprojecting various patterns in order to convey its movementintentions during encounters with humans. In these experiments,the trajectories of both humans and robot were recorded witha laser scanner. Human test subjects were also equipped withan eye tracker. We analyzed the eye gaze patterns and thelaser scan tracking data in order to understand how the robot’sintention communication affects the human movement behavior.Furthermore, we used retrospective recall interviews to aid inidentifying the reasons that lead to behavior changes.
The difficulty to obtain ground truth (i.e. empirical evidence) about how a gas disperses in an environment is one of the major hurdles in the field of mobile robotic olfaction (MRO), impairing our ability to develop efficient gas source localization strategies and to validate gas distribution maps produced by autonomous mobile robots. Previous ground truth measurements of gas dispersion have been mostly based on expensive tracer optical methods or 2D chemical sensor grids deployed only at ground level. With the ever-increasing trend towards gas-sensitive aerial robots, 3D measurements of gas dispersion become necessary to characterize the environment these platforms can explore. This paper presents ten different experiments performed with a 3D grid of 27 metal oxide semiconductor (MOX) sensors to visualize the temporal evolution of gas distribution produced by an evaporating ethanol source placed at different locations in an office room, including variations in height, release rate and air flow. We also studied which features of the MOX sensor signals are optimal for predicting the source location, considering different lengths of the measurement window. We found strongly time-varying and counter-intuitive gas distribution patterns that disprove some assumptions commonly held in the MRO field, such as that heavy gases disperse along ground level. Correspondingly, ground-level gas distributions were rarely useful for localizing the gas source and elevated measurements were much more informative. We make the dataset and the code publicly available to enable the community to develop, validate, and compare new approaches related to gas sensing in complex environments.
This paper describes the development and validation of the currently smallest aerial platform with olfaction capabilities. The developed Smelling Nano Aerial Vehicle (SNAV) is based on a lightweight commercial nano-quadcopter (27 g) equipped with a custom gas sensing board that can host up to two in situ metal oxide semiconductor (MOX) gas sensors. Due to its small form-factor, the SNAV is not a hazard for humans, enabling its use in public areas or inside buildings. It can autonomously carry out gas sensing missions of hazardous environments inaccessible to terrestrial robots and bigger drones, for example searching for victims and hazardous gas leaks inside pockets that form within the wreckage of collapsed buildings in the aftermath of an earthquake or explosion. The first contribution of this work is assessing the impact of the nano-propellers on the MOX sensor signals at different distances to a gas source. A second contribution is adapting the ‘bout’ detection algorithm, proposed by Schmuker et al. (2016) to extract specific features from the derivative of the MOX sensor response, for real-time operation. The third and main contribution is the experimental validation of the SNAV for gas source localization (GSL) and mapping in a large indoor environment (160 m2) with a gas source placed in challenging positions for the drone, for example hidden in the ceiling of the room or inside a power outlet box. Two GSL strategies are compared, one based on the instantaneous gas sensor response and the other one based on the bout frequency. From the measurements collected (in motion) along a predefined sweeping path we built (in less than 3 min) a 3D map of the gas distribution and identified the most likely source location. Using the bout frequency yielded on average a higher localization accuracy than using the instantaneous gas sensor response (1.38 m versus 2.05 m error), however accurate tuning of an additional parameter (the noise threshold) is required in the former case. The main conclusion of this paper is that a nano-drone has the potential to perform gas sensing tasks in complex environments.
Gas distribution modelling can provide potentially life-saving information when assessing the hazards of gaseous emissions and for localization of explosives, toxic or flammable chemicals. In this work, we deployed a three-dimensional (3D) grid of metal oxide semiconductor (MOX) gas sensors deployed in an office room, which allows for novel insights about the complex patterns of indoor gas dispersal. 12 independent experiments were carried out to better understand dispersion patters of a single gas source placed at different locations of the room, including variations in height, release rate and air flow profiles. This dataset is denser and richer than what is currently available, i.e., 2D datasets in wind tunnels. We make it publicly available to enable the community to develop, validate, and compare new approaches related to gas sensing in complex environments.
In order to deal with the scaling problem of volumetric map representations, we propose spatially local methods for high-ratio compression of 3D maps, represented as truncated signed distance fields. We show that these compressed maps can be used as meaningful descriptors for selective decompression in scenarios relevant to robotic applications. As compression methods, we compare using PCA-derived low-dimensional bases to nonlinear auto-encoder networks. Selecting two application-oriented performance metrics, we evaluate the impact of different compression rates on reconstruction fidelity as well as to the task of map-aided ego-motion estimation. It is demonstrated that lossily reconstructed distance fields used as cost functions for ego-motion estimation can outperform the original maps in challenging scenarios from standard RGB-D (color plus depth) data sets due to the rejection of high-frequency noise content.
With the increased availability of GPUs and multicore CPUs, volumetric map representations are an increasingly viable option for robotic applications. A particularly important representation is the truncated signed distance field (TSDF) that is at the core of recent advances in dense 3D mapping. However, there is relatively little literature exploring the characteristics of 3D feature detection in volumetric representations. In this paper we evaluate the performance of features extracted directly from a 3D TSDF representation. We compare the repeatability of Integral invariant features, specifically designed for volumetric images, to the 3D extensions of Harris and Shi & Tomasi corners. We also study the impact of different methods for obtaining gradients for their computation. We motivate our study with an example application for building sparse stable scene graphs, and present an efficient GPU-parallel algorithm to obtain the graphs, made possible by the combination of TSDF and 3D feature points. Our findings show that while the 3D extensions of 2D corner-detection perform as expected, integral invariants have shortcomings when applied to discrete TSDFs. We conclude with a discussion of the cause for these points of failure that sheds light on possible mitigation strategies.
In this work we propose a method to effectively remove noise from depth images obtained with a commodity structured light sensor. The proposed approach fuses data into a consistent frame of reference over time, thus utilizing prior depth measurements and viewpoint information in the noise removal process. The effectiveness of the approach is compared to two state of the art, single-frame denoising methods in the context of feature descriptor matching and keypoint detection stability. To make more general statements about the effect of noise removal in these applications, we extend a method for evaluating local image gradient feature descriptors to the domain of 3D shape descriptors. We perform a comparative study of three classes of such descriptors: Normal Aligned Radial Features, Fast Point Feature Histograms and Depth Kernel Descriptors; and evaluate their performance on a real-world industrial application data set. We demonstrate that noise removal enabled by the dense map representation results in major improvements in matching across all classes of descriptors as well as having a substantial positive impact on keypoint detection reliability
Ego-motion estimation and environment mapping are two recurring problems in the field of robotics. In this work we propose a simple on-line method for tracking the pose of a depth camera in six degrees of freedom and simultaneously maintaining an updated 3D map, represented as a truncated signed distance function. The distance function representation implicitly encodes surfaces in 3D-space and is used directly to define a cost function for accurate registration of new data. The proposed algorithm is highly parallel and achieves good accuracy compared to state of the art methods. It is suitable for reconstructing single household items, workspace environments and small rooms at near real-time rates, making it practical for use on modern CPU hardware
Voxel volumes are simple to implement and lend themselves to many of the tools and algorithms available for 2D images. However, the additional dimension of voxels may be costly to manage in memory when mapping large spaces at high resolutions. While lowering the resolution and using interpolation is common work-around, in the literature we often find that authors either use trilinear interpolation or nearest neighbors and rarely any of the intermediate options. This paper presents a survey of geometric interpolation methods for voxel-based map representations. In particular we study the truncated signed distance field (TSDF) and the impact of using fewer than 8 samples to perform interpolation within a depth-camera pose tracking and mapping scenario. We find that lowering the number of samples fetched to perform the interpolation results in performance similar to the commonly used trilinear interpolation method, but leads to higher framerates. We also report that lower bit-depth generally leads to performance degradation, though not as much as may be expected, with voxels containing as few as 3 bits sometimes resulting in adequate estimation of camera trajectories.
A mobile robot communicating its intentions using Spatial Augmented Reality (SAR) on the shared floor space makes humans feel safer and more comfortable around the robot. Our previous work [1] and several other works established this fact. We built upon that work by adding an adaptable information and control to the SAR module. An empirical study about how a mobile robot builds trust in humans by communicating its intentions was conducted. A novel way of evaluating that trust is presented and experimentally shown that adaption in SAR module lead to natural interaction and the new evaluation system helped us discover that the comfort levels between human-robot interactions approached those of human-human interactions.
The upcoming new generation of autonomous vehicles for transporting materials in industrial environments will be more versatile, flexible and efficient than traditional AGVs, which simply follow pre-defined paths. However, freely navigating vehicles can appear unpredictable to human workers and thus cause stress and render joint use of the available space inefficient. Here we address this issue and propose on-board intention projection on the shared floor space for communication from robot to human. We present a research prototype of a robotic fork-lift equipped with a LED projector to visualize internal state information and intents. We describe the projector system and discuss calibration issues. The robot’s ability to communicate its intentions is evaluated in realistic situations where test subjects meet the robotic forklift. The results show that already adding simple information, such as the trajectory and the space to be occupied by the robot in the near future, is able to effectively improve human response to the robot.
Eye gaze can convey information about intentions beyond what can beinferred from the trajectory and head pose of a person. We propose eye-trackingglasses as safety equipment in industrial environments shared by humans androbots. In this work, an implicit intention transference system was developed and implemented. Robot was given access to human eye gaze data, and it responds tothe eye gaze data through spatial augmented reality projections on the sharedfloor space in real-time and the robot could also adapt its path. This allows proactivesafety approaches in HRI for example by attempting to get the human'sattention when they are in the vicinity of a moving robot. A study was conductedwith workers at an industrial warehouse. The time taken to understand the behaviorof the system was recorded. Electrodermal activity and pupil diameter wererecorded to measure the increase in stress and cognitive load while interactingwith an autonomous system, using these measurements as a proxy to quantifytrust in autonomous systems.
Robots in human co-habited environments need human-aware task and motion planning, ideally responding to people’s motion intentions as soon as they can be inferred from human cues. Eye gaze can convey information about intentions beyond trajectory and head pose of a person. Hence, we propose eye-tracking glasses as safety equipment in industrial environments shared by humans and robots. This paper investigates the possibility of human-to-robot implicit intention transference solely from eye gaze data. We present experiments in which humans wearing eye-tracking glasses encountered a small forklift truck under various conditions. We evaluate how the observed eye gaze patterns of the participants related to their navigation decisions. Our analysis shows that people primarily gazed on that side of the robot they ultimately decided to pass by. We discuss implications of these results and relate to a control approach that uses human eye gaze for early obstacle avoidance.