This paper presents a machine learning algorithm for smile detection in images. To come to the point where it is necessary to detect the presence or absence of smile, it is necessary to go through a couple of phases, such as: the determination of the facial region based on skin color, eye detection using Harris detector and the determination of the mouth region which is based on morphological face characteristics. In the final stage of the algorithm is analyzed relative position of local extreme in the mouth region in order to detect a smile. The contributions of this paper are reducing the area for locating the eyes using Harris detector and smile detection based on the mutual position of the local extremes. The advantage is a high degree of accuracy, and the lack is relatively long period required for its execution.