•A comprehensive and critical review is conducted of CV applications.•Construct and visualize bibliometric networks to review literature.•A contents analysis method is used to quantitatively review literature.•The limitations of existing studies and future research directions are discussed. Globally, the occupational health and safety (OHS) of construction workers has long been a serious concern. To address this issue, there is an urgent need for an efficient means to continuously monitor the construction site to eliminate potential hazards in a timely manner. As robust and automated video and image information extraction and processing tools for construction sites, computer vision techniques have been considered to be effective solutions and been applied for the occupational health and safety monitoring of construction site workers. This paper aims to use bibliometric and content-based analysis methods to review the previous attempts in related fields and present the current research status in this field. The results clarify the major limitations and challenges of the current research from both technical and practical perspectives, in turn suggesting the direction of future research.