Environmental collision is a challenging issue in human-robot collaboration. This article proposes a novel fuzzy cluster-based framework for robots to have reactive responses to various environmental collision scenarios. This framework makes four contributions: First, a fuzzy cluster-based environmental collision detection algorithm is developed to efficiently classify the collision area and non-collision (free) area of the environment. Second, based on the collision detection algorithm, a p-norm approximation-based collision avoidance algorithm is proposed to enable robots to avoid environmental collisions with guaranteed stability. Third, by extending the collision avoidance algorithm, an environmental collision adaptation algorithm is proposed to allow robots to adapt to environmental collisions with intelligently regulated contact force. Fourth, a teleoperation controller is designed to strengthen haptic force rendering and enhance the operator’s perception of collisions. Going beyond existing methods, the proposed framework allows teleoperated robots to have real-time responses to collisions in quasi-static environments without suffering from local optima, where the environments can be unstructured, non-convex, and detected with noisy outliers. In addition, this framework is simple in implementation because the proposed collision avoidance and collision adaptation algorithms work as several linear Quadratic Programming (QP) constraints that can be flexibly used by Inverse Kinematics (IK) solvers. Several experiments using 7-Degree of Freedom (DoF) robots are conducted to test and compare the proposed framework with existing methods, demonstrating the effectiveness of our work.