This study aims to assess various perceived adverse effects of noise on construction sites according to the different stages and machinery used in the stages, and to examine whether or not personal-situational factors affect the judgment of managers regarding the adverse impact of noise at work through a self-reported survey method. Four primary construction stages and twenty-four types of construction machines were evaluated. The effects of personal (noise sensitivity) and situational (types of jobs, and years of working) factors on the adverse impacts of noise on annoyance, work performance, work safety, and speech interference at work were examined. The results show significant differences in perceived noise annoyance in the various construction stages. The demolition stage is the most annoying, followed by the foundation, earthwork, and concrete framing stages. In addition, the annoying equipment differs for each construction stage. A breaker, pile driver, and hammer compactor are rated as the most annoying construction machines in the demolition, foundation, and earthwork stages, respectively. Individual noise sensitivity appears to have the most significant influence on the adverse impacts of noise on annoyance, work performance, work safety, and speech interference. A high noise sensitivity group tends to judge construction noise to be more adverse than the rest. In addition, different interrelationships between the adverse items of noise are found across their types of jobs: building construction, civil construction, and safety management. The findings of this study will provide further knowledge to facilitate better noise management planning on construction sites. •Demolition stage is the most annoying stage on construction sites.•Machines that emits high noise level with high temporal variation are more annoying.•Noise sensitivity appears to have the most significant influence on the adverse impacts.•Significantly different correlations between the adverse impacts of noise are found across job types.