This paper proposes a new game-control interface using hand poses captured from a digital camera. Extracting skin color from the background-subtracted image is generally used for segmenting hand region. But it suffers from change in illumination, shadows and skin- colored objects so that unwanted subsets reduce accuracy in hand pose recognition. To deal with these problems, first, subtracting skin colors from the background image must be done and second, it mustn 't get affected by changes in background. The paper describes an approach to using variance of homogeneity for skin color extraction. Homogeneity which represents contrast in a window is robust against change in illumination and can be used to effectively extract skin color rather than color or brightness based subtraction from background. The recognition of hand poses proceeds by computing fourier descriptor of extracted pixels. This approach to recognition can robustly identify hand poses comparing to the others. Experiments on real-time game demonstrate the robustness of the proposed method.