Virtual Reality (VR) is regarded as a useful tool for teleoperation system that provides operators an immersive visual feedback on the robot and the environment. However, without any haptic feedback or physical constructions, VR-based teleoperation systems normally have poor maneuverability and may cause operational faults in some fine movements. In this paper, we employ Mixed Reality (MR), which combines real and virtual worlds, to develop a novel teleoperation system. New system design and control algorithms are proposed. For the system design, a MR interface is developed based on a virtual environment augmented with real-time data from the task space with a goal to enhance the operator’s visual perception. To allow the operator to be freely decoupled from the control loop and offload the operator’s burden, a new interaction proxy is proposed to control the robot. For the control algorithms, two control modes are introduced to improve long-distance movements and fine movements of the MR-based teleoperation. In addition, a set of fuzzy logic based methods are proposed to regulate the position, velocity and force of the robot in order to enhance the system maneuverability and deal with the potential operational faults. Barrier Lyapunov Function (BLF) and back-stepping methods are leveraged to design the control laws and simultaneously guarantee the system stability under state constraints.  Experiments conducted using a 6-Degree of Freedom (DoF) robotic arm prove the feasibility of the system.

This paper firstly develops a novel force observer using Type-2 Fuzzy Neural Network (T2FNN)-based Moving Horizon Estimation (MHE) to estimate external force/torque information and simultaneously filter out the system disturbances. Then, by using the proposed force observer, a new bilateral teleoperation system is proposed that allows the slave industrial robot to be more compliant to the environment and enhances the situational awareness of the human operator by providing multi-level force feedback. Compared with existing force observer algorithms that highly rely on knowing exact mathematical models, the proposed force estimation strategy can derive more accurate external force/torque information of the robots with complex mechanism and with unknown dynamics. Applying the estimated force information, an external-force-regulated Sliding Mode Control (SMC) strategy with the support of machine vision is proposed to enhance the adaptability of the slave robot and the perception of the operator about various scenarios by virtue of the detected location of the task object. The proposed control system is validated by the experiment platform consisting of a universal robot (UR10), a haptic device and an RGB-D sensor.

This paper develops an innovative multilateral teleoperation system with two haptic devices in the master side and a newly-designed reconfigurable multi-fingered robot hand in the slave side. A novel non-singular fast terminal sliding mode (NFTSM) algorithm together with varying dominance factors for cooperation is proposed to offer this system fast position and force tracking, as well as an integrate perception for the operator on the robot hand. Type-2 fuzzy model is used to describe the overall system dynamics, and accordingly a new fuzzy-model-based state observer is proposed to compensate for the system uncertainties. A sliding mode adaptive controller is designed to deal with the varying zero drift of the force sensors and force observers. The stability of the closed-loop system under time-varying delays is proved using Lyapunov-Krasovskii functions. Finally, experiments to grasp different objects are performed to verify the effectiveness of this multilateral teleoperation system.

In order to better handle the coupling effects when controlling multiple-input multiple-output (MIMO) systems, taking the decentralized control structure as the basis, this paper proposes a sparse control strategy and a decoupling control strategy. Type-1 and type-2 Takagi-Sugeno (T-S) fuzzy models are used to describe the MIMO system, and the relative normalized gain array (RNGA) based criterion is employed to measure the coupling effects. The main contributions include: i). compared to the previous studies, a manner with less computational cost to build fuzzy models for the MIMO systems is provided, and a more accurate method to construct the so-called effective T-S fuzzy model (ETSM) to express the coupling effects is developed; ii). for the sparse control strategy, four indexes are defined in order to extend a decentralized control structure to a sparse one. Afterwards, an ETSM-based method is presented that a sparse control system can be realized by designing multiple independent single-input single-output (SISO) control-loops; iii). for the decoupling control strategy, a novel and simple ETSM-based decoupling compensator is developed that can effectively compensate for both steady and dynamic coupling effects. As a result, the MIMO controller design can be transformed to multiple non-interacting SISO controller designs. Both of the sparse and decoupling strategies allow to use linear SISO control algorithms to regulate a closely coupled nonlinear MIMO system without knowing its exact mathematical functions. Two examples are used to show the effectiveness of the proposed strategies

Interaction measure determines decentralized and parse control configurations for a multivariable process control. This paper investigates interval type-2 Takagi–Sugeno fuzzy (IT2TSF) model based interactionmeasures using two different criteria, one is controllability and observability gramians, the other is relative normalized gain array (RNGA). The main contributions are: first, a data-driven IT2TSF modeling method is introduced; econd, explicit formulas to execute the two measures based on IT2TSF models are given; third, two interaction indexes are defined from RNGA to select sparse control configuration; fourth, the calculations to derive sensitivities of the two measures with respect to parametric variations in the IT2TSF models are developed; and fifth, the discussion to compare the two measures is presented. Three multivariable processes are used as examples to show that the results calculated from IT2TSF models are more accurate than that from their type-1 counterparts, and compared to gramian-basedmeasure, RNGA selectsmore reasonable control configurations and is more robust to the parametric uncertainties.

This paper develops a novel shared control scheme for online-switching tele-operated and autonomous system with time-varying delays. Type-2 Takagi-Sugeno (T-S) fuzzy model is used to describe the dynamics of master and slave robots in this system. A novel non-singular fast terminal siding mode (NFTSM)-based algorithm combined with an extended wave-based time domain passivity approach (TDPA) is presented to enhance the master-slave motion synchronization in the tele-operated mode and reference-slave motion synchronization in the autonomous mode, while simultaneously ensuring the stability of the overall system in the presence of arbitrary time delays. In addition, based on the Type-2 Fuzzy model, a new torque observer is designed to estimate the external torques and then the torque tracking method is employed in the control laws to let the slave apply the designated force to further improve the operator’s force perception for the environment. The stability of the closed-loop system is proven using the Lyapunov-Krasovskii functions. Finally, experiments using two haptic devices prove the superiority of the proposed strategy.

tEffective model is a novel tool for decentralized controller design to handle the interconnected inter-actions in a multi-input-multi-output (MIMO) process. In this paper, Type-1 and Type-2 effectiveTakagi-Sugeno fuzzy models (ETSM) are investigated. By means of the loop pairing criterion, simple cal-culations are given to build Type-1/Type-2 ETSMs which are used to describe a group of non-interactingequivalent single-input-single-output (SISO) systems to represent an MIMO process, consequently thedecentralized controller design can be converted to multiple independent single-loop controller designs,and enjoy the well-developed linear control algorithms. The main contributions of this paper are: i)Compared to the existing T-S fuzzy model based decentralized control methods using extra terms tocharacterize interactions, ETSM is a simple feasible alternative; ii) Compared to the existing effectivemodel methods using linear transfer functions, ETSM can be carried out without requiring exact mathe-matical process functions, and lays a basis to develop robust controllers since fuzzy system is powerful tohandle uncertainties; iii) Type-1 and Type-2 ETSMs are presented under a unified framework to provideobjective comparisons. A nonlinear MIMO process is used to demonstrate the ETSMs’ superiority overthe effective transfer function (ETF) counterparts as well as the evident advantage of Type-2 ETSMs interms of robustness. A multi-evaporator refrigeration system is employed to validate the practicabilityof the proposed methods.