For the purpose of vibration-based condition monitoring and to prevent occurrence of catastrophic gear failures it is important to improve the simulated dynamic response of the studied gear model. The time-varying gear mesh stiffness will contribute to the dynamic response of a geared system. Some previously applied methods for stiffness calculation, described in the literature, show good agreement with the results obtained with FEM simulation for smaller crack sizes. However, when larger crack sizes are reached, these methods show an increasing deviation from FEM simulation results. A reduction in the gear mesh stiffness can be considered to assess the status of tooth damage and, therefore, by increasing the accuracy of the calculated mesh stiffness, dynamic simulations of a gear can be improved. In this paper a new method is presented for calculating the gear mesh stiffness for a propagating crack in the tooth root. The influence of gear mesh stiffness on the vibration-based fault detection indicators, the RMS, kurtosis and the crest factor, is investigated. Different crack sizes are examined by using this new method for sizes up to around 50% of the total tooth root thickness. When compared to FEM simulations, the presented method shows more accurate results for calculations of the gear mesh stiffness (for the studied model) than the previously suggested methods. (C) 2013 Elsevier Ltd. All rights reserved.