In robotics perception, numerous tasks rely on point cloud registration. However, currently there is no method that can automatically detect misaligned point clouds reliably and without environment-specific parameters. We propose "CorAl", an alignment quality measure and alignment classifier for point cloud pairs, which facilitates the ability to introspectively assess the performance of registration. CorAl compares the joint and the separate entropy of the two point clouds. The separate entropy provides a measure of the entropy that can be expected to be inherent to the environment. The joint entropy should therefore not be substantially higher if the point clouds are properly aligned. Computing the expected entropy makes the method sensitive also to small alignment errors, which are particularly hard to detect, and applicable in a range of different environments. We found that CorAl is able to detect small alignment errors in previously unseen environments with an accuracy of 95% and achieve a substantial improvement to previous methods.
This paper presents a control structure selection guideline based on Type-1 and Type-2 T-S fuzzy models for multi-input-multi-output (MIMO) processes. By virtue of relative normalized gain array (RNGA), the interactions between process inputs and outputs are analyzed to determine a decentralized control configuration as well as a richer control structure for an MIMO process controller design. Simple calculating procedures based on both Type-1 and Type-2 T-S fuzzy models are provided. An MIMO process is utilized as an example to demonstrate effectiveness and robustness of the proposed method.
In order to facilitate decentralized fuzzy controller designs for multi-input-multi-output (MIMO) processes, this paper presents a novel manner, called effective Takagi-Sugeno (T-S) fuzzy model (ETSM), to describe the interactions among the loops. For a certain control-loop of an MIMO process, in terms of relative normalized gain array (RNGA) based loop pairing criterion, simple calculating procedure is given to obtain an ETSM based on its individual open-loop T-S fuzzy model. With the ETSMs of control-loops, an MIMO process can be approximately regarded as multiple non-interacting single loops such that each local controller of a decentralized control system can be independently designed using linear single-input-single-output (SISO) control algorithms. Compared with the existing decentralized fuzzy control methods adding extra terms to individual open-loop models to characterize interactions, ETSM is a practical and low-cost way. While compared with the existing effective transfer function (ETF) methods, ETSM is an extension that can proceed without requiring exact process mathematical functions, and lays a basis to develop robust controller since fuzzy system is strong in handling uncertainties. In case study, a nonlinear MIMO process is used as an example to demonstrate the effectiveness of the proposed ETSM method.
In terms of relative normalized gain array (RNGA) criterion, this paper presents an interaction analysis and loop pairing method for multi-input-multi-output (MIMO) processes described by Type-2 Takagi-Sugeno (T-S) fuzzy models. Type2 fuzzy system offers a significant improvement on traditional (Type-1) fuzzy systems to handle the uncertainties. For each individual loop in the MIMO process, the steady-state gain and normalized integrated error can be derived from its Type-2 T-S fuzzy model through simple formulae, and the pairing results can then be obtained according to the rules of RNGA criterion. This method can be applied to measure the interactions and determine the control structure for multivariable control system design when the mathematical functions are unavailable and a large number of uncertainties present. The simulation proves that the proposed method can provide more accurate results than the existing RNGA paring method based on Type-1 fuzzy model under the influence of uncertainties.
Interaction measure determines decentralized and parse control configurations for a multivariable process control. This paper investigates interval type-2 Takagi–Sugeno fuzzy (IT2TSF) model based interactionmeasures using two different criteria, one is controllability and observability gramians, the other is relative normalized gain array (RNGA). The main contributions are: first, a data-driven IT2TSF modeling method is introduced; econd, explicit formulas to execute the two measures based on IT2TSF models are given; third, two interaction indexes are defined from RNGA to select sparse control configuration; fourth, the calculations to derive sensitivities of the two measures with respect to parametric variations in the IT2TSF models are developed; and fifth, the discussion to compare the two measures is presented. Three multivariable processes are used as examples to show that the results calculated from IT2TSF models are more accurate than that from their type-1 counterparts, and compared to gramian-basedmeasure, RNGA selectsmore reasonable control configurations and is more robust to the parametric uncertainties.
In order to better handle the coupling effects when controlling multiple-input multiple-output (MIMO) systems, taking the decentralized control structure as the basis, this paper proposes a sparse control strategy and a decoupling control strategy. Type-1 and type-2 Takagi-Sugeno (T-S) fuzzy models are used to describe the MIMO system, and the relative normalized gain array (RNGA) based criterion is employed to measure the coupling effects. The main contributions include: i). compared to the previous studies, a manner with less computational cost to build fuzzy models for the MIMO systems is provided, and a more accurate method to construct the so-called effective T-S fuzzy model (ETSM) to express the coupling effects is developed; ii). for the sparse control strategy, four indexes are defined in order to extend a decentralized control structure to a sparse one. Afterwards, an ETSM-based method is presented that a sparse control system can be realized by designing multiple independent single-input single-output (SISO) control-loops; iii). for the decoupling control strategy, a novel and simple ETSM-based decoupling compensator is developed that can effectively compensate for both steady and dynamic coupling effects. As a result, the MIMO controller design can be transformed to multiple non-interacting SISO controller designs. Both of the sparse and decoupling strategies allow to use linear SISO control algorithms to regulate a closely coupled nonlinear MIMO system without knowing its exact mathematical functions. Two examples are used to show the effectiveness of the proposed strategies.
This paper studies the fuzzy cluster-based point set registration (FuzzyPSReg). First, we propose a new metric based on Gustafson-Kessel (GK) fuzzy clustering to measure the alignment of two point clouds. Unlike the metric based on fuzzy c-means (FCM) clustering in our previous work, the GK-based metric includes orientation properties of the point clouds, thereby providing more information for registration. We then develop the registration quality assessment of the GK-based metric, which is more sensitive to small misalignments than that of the FCM-based metric. Next, by effectively combining the two metrics, we design two FuzzyPSReg strategies with global optimization: i). \textit{FuzzyPSReg-SS}, which extends our previous work and aligns two similar-sized point clouds with greatly improved efficiency; ii). \textit{FuzzyPSReg-O2S}, which aligns two point clouds with a relatively large difference in size and can be used to estimate the pose of an object in a scene. In the experiment, we use different point clouds to test and compare the proposed method with state-of-the-art registration approaches. The results demonstrate the advantages and effectiveness of our method.
This study presents a new point set registration method to align 3D range scans. In our method, fuzzy clusters are utilized to represent a scan, and the registration of two given scans is realized by minimizing a fuzzy weighted sum of the distances between their fuzzy cluster centers. This fuzzy cluster-based metric has a broad basin of convergence and is robust to noise. Moreover, this metric provides analytic gradients, allowing standard gradient-based algorithms to be applied for optimization. Based on this metric, the outlier issues are addressed. In addition, for the first time in rigid point set registration, a registration quality assessment in the absence of ground truth is provided. Furthermore, given specified rotation and translation spaces, we derive the upper and lower bounds of the fuzzy cluster-based metric and develop a branch-and-bound (BnB)-based optimization scheme, which can globally minimize the metric regardless of the initialization. This optimization scheme is performed in an efficient coarse-to-fine fashion: First, fuzzy clustering is applied to describe each of the two given scans by a small number of fuzzy clusters. Then, a global search, which integrates BnB and gradient-based algorithms, is implemented to achieve a coarse alignment for the two scans. During the global search, the registration quality assessment offers a beneficial stop criterion to detect whether a good result is obtained. Afterwards, a relatively large number of points of the two scans are directly taken as the fuzzy cluster centers, and then, the coarse solution is refined to be an exact alignment using the gradient-based local convergence. Compared to existing counterparts, this optimization scheme makes a large improvementin terms of robustness and efficiency by virtue of the fuzzy cluster-based metric and the registration quality assessment. In the experiments, the registration results of several 3D range scan pairs demonstrate the accuracy and effectiveness of the proposed method, as well as its superiority to state-of-the-art registration approaches.
tEffective model is a novel tool for decentralized controller design to handle the interconnected inter-actions in a multi-input-multi-output (MIMO) process. In this paper, Type-1 and Type-2 effectiveTakagi-Sugeno fuzzy models (ETSM) are investigated. By means of the loop pairing criterion, simple cal-culations are given to build Type-1/Type-2 ETSMs which are used to describe a group of non-interactingequivalent single-input-single-output (SISO) systems to represent an MIMO process, consequently thedecentralized controller design can be converted to multiple independent single-loop controller designs,and enjoy the well-developed linear control algorithms. The main contributions of this paper are: i)Compared to the existing T-S fuzzy model based decentralized control methods using extra terms tocharacterize interactions, ETSM is a simple feasible alternative; ii) Compared to the existing effectivemodel methods using linear transfer functions, ETSM can be carried out without requiring exact mathe-matical process functions, and lays a basis to develop robust controllers since fuzzy system is powerful tohandle uncertainties; iii) Type-1 and Type-2 ETSMs are presented under a unified framework to provideobjective comparisons. A nonlinear MIMO process is used to demonstrate the ETSMs’ superiority overthe effective transfer function (ETF) counterparts as well as the evident advantage of Type-2 ETSMs interms of robustness. A multi-evaporator refrigeration system is employed to validate the practicabilityof the proposed methods.
This article studies group-wise point set registration and makes the following contributions: "FuzzyGReg", which is a new fuzzy cluster-based method to register multiple point sets jointly, and "FuzzyQA", which is the associated quality assessment to check registration accuracy automatically. Given a group of point sets, FuzzyGReg creates a model of fuzzy clusters and equally treats all the point sets as the elements of the fuzzy clusters. Then, the group-wise registration is turned into a fuzzy clustering problem. To resolve this problem, FuzzyGReg applies a fuzzy clustering algorithm to identify the parameters of the fuzzy clusters while jointly transforming all the point sets to achieve an alignment. Next, based on the identified fuzzy clusters, FuzzyQA calculates the spatial properties of the transformed point sets and then checks the alignment accuracy by comparing the similarity degrees of the spatial properties of the point sets. When a local misalignment is detected, a local re-alignment is performed to improve accuracy. The proposed method is cost-efficient and convenient to be implemented. In addition, it provides reliable quality assessments in the absence of ground truth and user intervention. In the experiments, different point sets are used to test the proposed method and make comparisons with state-of-the-art registration techniques. The experimental results demonstrate the effectiveness of our method.The code is available at https://gitsvn-nt.oru.se/qianfang.liao/FuzzyGRegWithQA
Virtual Reality (VR) is regarded as a useful tool for teleoperation system that provides operators an immersive visual feedback on the robot and the environment. However, without any haptic feedback or physical constructions, VR-based teleoperation systems normally have poor maneuverability and may cause operational faults in some fine movements. In this paper, we employ Mixed Reality (MR), which combines real and virtual worlds, to develop a novel teleoperation system. New system design and control algorithms are proposed. For the system design, a MR interface is developed based on a virtual environment augmented with real-time data from the task space with a goal to enhance the operator’s visual perception. To allow the operator to be freely decoupled from the control loop and offload the operator’s burden, a new interaction proxy is proposed to control the robot. For the control algorithms, two control modes are introduced to improve long-distance movements and fine movements of the MR-based teleoperation. In addition, a set of fuzzy logic based methods are proposed to regulate the position, velocity and force of the robot in order to enhance the system maneuverability and deal with the potential operational faults. Barrier Lyapunov Function (BLF) and back-stepping methods are leveraged to design the control laws and simultaneously guarantee the system stability under state constraints. Experiments conducted using a 6-Degree of Freedom (DoF) robotic arm prove the feasibility of the system.
Manipulability ellipsoid on the Riemannian manifold serves as an effective criterion to analyze, measure, and control the dexterous performance of robots. For asymmetric bilateral telerobotics, due to the different structures of master and slave robots, it is difficult or even impossible for the operator to manually regulate the manipulability of the remote slave robot. Thus, it is desired that the slave robot can automatically regulate its manipulability to assist the operator in remote different task executions, like humans regulating their own postures to enhance manipulability and adapt to different task scenarios. This article proposes a novel framework for manipulability transfer from human to robot. In this framework, we develop a Type-2 fuzzy model-based imitation learning method to encode and reproduce manipulability ellipsoids from demonstrations. This method can achieve high performance in accuracy and computational efficiency. In addition, it supports learning from a single demonstration. Then, we combine this method with a Riemannian manifold-based quadratic programming control algorithm such that the robot manipulability can fast track the desired manipulability profile. This framework is applied to telerobotics, in which a bilateral teleoperation controller is designed that enables the robot to follow the operator's command and simultaneously self-regulate its manipulability to perform the task adaptively. Meanwhile, the operator can receive force feedback relating to the manipulability regulation. Evaluations using comparative studies and practical experiments with a 3-DoF haptic device and 7-DoF robots are presented to show the effectiveness of the proposed framework.
Environmental collision is a challenging issue in human-robot collaboration. This article proposes a novel fuzzy cluster-based framework for robots to have reactive responses to various environmental collision scenarios. This framework makes four contributions: First, a fuzzy cluster-based environmental collision detection algorithm is developed to efficiently classify the collision area and non-collision (free) area of the environment. Second, based on the collision detection algorithm, a p-norm approximation-based collision avoidance algorithm is proposed to enable robots to avoid environmental collisions with guaranteed stability. Third, by extending the collision avoidance algorithm, an environmental collision adaptation algorithm is proposed to allow robots to adapt to environmental collisions with intelligently regulated contact force. Fourth, a teleoperation controller is designed to strengthen haptic force rendering and enhance the operator’s perception of collisions. Going beyond existing methods, the proposed framework allows teleoperated robots to have real-time responses to collisions in quasi-static environments without suffering from local optima, where the environments can be unstructured, non-convex, and detected with noisy outliers. In addition, this framework is simple in implementation because the proposed collision avoidance and collision adaptation algorithms work as several linear Quadratic Programming (QP) constraints that can be flexibly used by Inverse Kinematics (IK) solvers. Several experiments using 7-Degree of Freedom (DoF) robots are conducted to test and compare the proposed framework with existing methods, demonstrating the effectiveness of our work.
In this article, we propose a new method for multi-robot systems to have reactive responses to various collision scenarios in real time. This method contains a novel p-norm approximation-based reactive approach, which allows multiple robots to avoid mutual collisions or to adapt to the collisions with intelligently regulated force. Compared with existing approaches, the implementation of the proposed method is simpler and more convenient since our reactive approach works as several linear Quadratic Programming (QP) constraints, allowing for flexible utilization by Inverse Kinematics (IK) solvers. In addition, it requires low computational complexity, achieves high accuracy, and does not need training. In the experiments, we employ a multi-robot system to conduct comprehensive comparisons between the proposed method and state-of-the-art approaches, effectively showcasing the efficacy of our work.
The asymmetry in bilateral teleoperation, i.e., the differences of mechanical structures, sizes, and number of joints between the master and slave robots, can introduce kinematicsr edundancy and workspace inequality problems. In this article, a novel shared autonomy control strategy is proposed for handling the asymmetry of bilateral teleoperation, which has two main contributions. First, to deal with kinematics redundancy, the proposed strategy provides a self-regulation algorithm of orientation that allows the operator to solely use the master position command to simultaneously control the slave’s position and orientation. Second, to deal with workspace inequality, the proposed strategy enables the slave’s workspace to be dynamically tunable to adapt to various task spaces without influencing the smoothness of the robot’s movement. The experiments on a platform consisting of a six-degree of freedom (DoF) UR10 robot and a 3-DoF haptic device are given to validate the effectiveness of the proposed control strategy.
This paper develops a novel shared control scheme for online-switching tele-operated and autonomous system with time-varying delays. Type-2 Takagi-Sugeno (T-S) fuzzy model is used to describe the dynamics of master and slave robots in this system. A novel non-singular fast terminal siding mode (NFTSM)-based algorithm combined with an extended wave-based time domain passivity approach (TDPA) is presented to enhance the master-slave motion synchronization in the tele-operated mode and reference-slave motion synchronization in the autonomous mode, while simultaneously ensuring the stability of the overall system in the presence of arbitrary time delays. In addition, based on the Type-2 Fuzzy model, a new torque observer is designed to estimate the external torques and then the torque tracking method is employed in the control laws to let the slave apply the designated force to further improve the operator’s force perception for the environment. The stability of the closed-loop system is proven using the Lyapunov-Krasovskii functions. Finally, experiments using two haptic devices prove the superiority of the proposed strategy.
This paper develops an innovative multilateral teleoperation system with two haptic devices in the master side and a newly-designed reconfigurable multi-fingered robot hand in the slave side. A novel non-singular fast terminal sliding mode (NFTSM) algorithm together with varying dominance factors for cooperation is proposed to offer this system fast position and force tracking, as well as an integrate perception for the operator on the robot hand. Type-2 fuzzy model is used to describe the overall system dynamics, and accordingly a new fuzzy-model-based state observer is proposed to compensate for the system uncertainties. A sliding mode adaptive controller is designed to deal with the varying zero drift of the force sensors and force observers. The stability of the closed-loop system under time-varying delays is proved using Lyapunov-Krasovskii functions. Finally, experiments to grasp different objects are performed to verify the effectiveness of this multilateral teleoperation system.
This study proposes a new shared mixed reality (MR)-bilateral telerobotic system. The main contribution of this study is to combine MR teleoperation and bilateral teleoperation, which takes advantage of the two types of teleoperation and compensates for each other's drawbacks. With this combination, the proposed system can address the asymmetry issues in bilateral teleoperation, such as kinematic redundancy and workspace inequality, and provide force feedback, which is lacking in MR teleoperation. In addition, this system effectively supports long-distance movements and fine movements. In this system, a new MR interface is developed to provide the operator with an immersive visual feedback of the workspace, in which a useful virtual controller known as an interaction proxy—is designed. Compared with previous virtual reality-based teleoperation systems, this interaction proxy can freely decouple the operator from the control loop, such that the operational burden can be substantially alleviated. Additionally, the force feedback provided by the bilateral teleoperation gives the operator an advanced perception about the remote workspace and can improve task performance. Experiments on multiple pick-and-place tasks are provided to demonstrate the feasibility and effectiveness of the proposed system.
This paper proposes a new single master bimanual teleoperation (SMBT) system with an efficient position, orientation and force regulation strategy. Unlike many existing studies that solely support motion synchronization, the first contribution of the proposed work is to propose a solution for orientation regulation when several slave robots have differing motions. In other words, we propose a solution for self-regulated orientation for dual-arm robots. A second contribution in the paper allows the master with fewer degrees of freedom to control the slaves (with higher degrees of freedom), while the orientation of the slaves is self-regulated. The system further offers a novel force regulation that enables the slave robots to have a smooth and balanced robot-environment interaction with proper force directions. Finally, the proposed approach provides adequate force feedback about the environment to the operator and assists the operator in identifying different motion situations of the slaves. Our approach demonstrates that the forces from the slaves will not interrupt the operator’s perception of the environment. To validate the proposed system, experiments are conducted using a platform consisting of two 7-Degree of Freedom (DoF) slave robots and one 3-DoF master haptic device. The experiments demonstrated good results in terms of position, orientation and force regulation.
Imitation learning is an important direction in the area of robot skill learning. It provides a user-friendly and straightforward solution to transfer human demonstrations to robots. In this article, we integrate fuzzy theory into imitation learning to develop a novel method called Type-2 Fuzzy Model-based Movement Primitives (T2FMP).In this method, a group of data-driven Type-2 fuzzy models are used to describe the input-output relationships of demonstrations. Based on the fuzzy models, T2FMP can efficiently reproduce the trajectory without high computational costs or cumbersome parameter settings. Besides, it can well handle the variation of the demonstrations and is robust to noise. In addition, we develop extensions that endow T2FMP with trajectory modulation and superposition to achieve real-time trajectory adaptation to various scenarios. Going beyond existing imitation learning methods, we further extend T2FMP to regulate the trajectory to avoid collisions in the environment that is unstructured, non-convex, and detected with noisy outliers. Several experiments are performed to validate the effectiveness of our method.
This paper firstly develops a novel force observer using Type-2 Fuzzy Neural Network (T2FNN)-based Moving Horizon Estimation (MHE) to estimate external force/torque information and simultaneously filter out the system disturbances. Then, by using the proposed force observer, a new bilateral teleoperation system is proposed that allows the slave industrial robot to be more compliant to the environment and enhances the situational awareness of the human operator by providing multi-level force feedback. Compared with existing force observer algorithms that highly rely on knowing exact mathematical models, the proposed force estimation strategy can derive more accurate external force/torque information of the robots with complex mechanism and with unknown dynamics. Applying the estimated force information, an external-force-regulated Sliding Mode Control (SMC) strategy with the support of machine vision is proposed to enhance the adaptability of the slave robot and the perception of the operator about various scenarios by virtue of the detected location of the task object. The proposed control system is validated by the experiment platform consisting of a universal robot (UR10), a haptic device and an RGB-D sensor.
Current path-tracking trajectory planning methods intrinsically suffer from a heavy computational burden, hindering them from modern autonomous robotic applications requiring real-time reactivity. To achieve flexible, accurate, and efficient motions, this article proposes a real-time trajectory planning framework for time-efficient path tracking. First, a receding horizon method is designed that generates local trajectories online while considering the global kinematic constraints. Second, an analytical closed-form method is developed for calculating the local time-minimal trajectory. Third, the models and solutions are established for handling dynamic factors. Compared to existing methods, this method exhibits lower computational complexity and can deal with path changes during motion executions while maintaining tracking accuracy and time efficiency. Experiments using Franka-Emika Panda robots are conducted in handover scenarios involving unpredictable dynamic obstacles and human interventions. The results demonstrate the low computational overhead. The robot flexibly reacts to online path changes, maintaining tracking accuracy while marginally compromising time efficiency.