In this paper, we propose a vision-based system for human detection and activity analysis in indoor environment. The developed presence sensor is based on video analysis, using a static camera. Composed of three main steps, the first one consists in change detection using a background model updated at different levels to manage the most common variations of the environment. Second, a moving objects tracking, based on interest points, is performed. Third, in order to know the nature of the various objects that could be present in the scene, multiple cascades of boosted classifiers are used. The validation protocol, defined by the industrial partners involved in the CAPTHOM project focusing among other things on “Energy Management in Building”, is then detailed. Three applications integrated into the CAPTHOM project illustrate how the developed system can help in collecting useful information for the building management system. Occupancy detection and people counting as well as activity characterization and 3D location extend to a wide variety of buildings technology research areas such as human-centered environmental control including heating adjustment and demand-controlled ventilation, but also security and energy efficient buildings.