Abstarct: In this contribution, a new frxamework for effective tracking of moving objects is introduced. In order to apply this object tracking method to hand gesture recognition in sign language, more importance is given to hand tracking. Through developing a two-step approach of Mean-Shift algorithm and corresponding extracted features namely colors, reasonable detection ability is obtained which yields desired fast tracking results feasible for real-time applications. Particle filtering is also another well-known approach in object tracking which is in fact more robust than Mean-Shift tracker but much slower. Both particle filtering (PF) and mean shift (MS) are two successful approaches to visual tracking. Both have their respective strengths and weaknesses. Mean Shift is a fast tracker and Particle filter is a robust tracker. In this Study, it is proposed to integrate advantages of these two algorithms into one procedure in order to improve hand tracking results by incorporating the MS optimization into particle filtering to move particles to local peaks in the likelihood, so that the proposed method improves the sampling efficiency considerably. The experimental results demonstrate that the proposed method, called CMP (Combination of Mean shift and Particle filter), can track hands two time faster than PF. CMP also tracks hand as robust as PF using just 20% number of total required particles in PF. Conclusively, Extensive experimental results illustrate that CMP outperforms MS and PF in hand tracking. Having proposed a capable hand tracker (CMP), a simple procedure for extracting hand trajectory from a sequence of images in the utilized sign language databaxse lead us to hand gesture recognition reasonably and acceptably.