A head-mounted display (HMD) with inappropriate mass and center of mass (COM) increases the physical workload of HMD users. The aim of this study was to investigate the effects of mass and COM of HMD on physical workload. Twelve subjects participated in this study. The mass and posteroanterior COM position were 0.8, 1.2, or 1.6 kg and −7.0, 0.0, or 7.0 cm, respectively. The subjects gazed at the target objects in four test postures: the neutral, look-up, body-bending, and look-down postures. The normalized joint torques for the neck and the lumbar region were calculated based on the measured segment angles. The results showed that the neck joint torque was significantly affected by mass and COM and it increased as the HMD mass increased for all test postures. The COM position that minimized the neck joint torque varied depending on the test postures, and the recommended ranges of COM were identified. •A gazing task while wearing head mounted display (HMD) was conducted.•The HMD had different levels of the mass and center of mass (COM) positions.•The joint torques of neck and lumbar region were calculated during the task.•The neck joint torque was significantly affected by the mass and COM of HMD.•The proper COM position varied depending on the position of the target object.