This paper proposes a motion generator approach to translating human motion from video image sequences to computer animations in real time. In this approach, a motion generator makes inferences about the current human motion and/or posture from the data obtained by processing the source video images, and then generates a set of joint angles for the target human body model. Compared with conventional motion capture methods, our approach is more robust, and tolerant of broader environmental and postural conditions. Experiments on a prototype system show that an animated virtual human can walk, sit, and lie as the real human performs without special illumination control. Copyright © 2000 John Wiley & Sons, Ltd.