Damage due to age or accumulated effects from hazards on existing structures poses a worldwide problem. In order to evaluate the current status of aging, deteriorating and damaged structures, it is vital to accurately assess the present conditions. It is possible to capture the in situ condition of structures by using laser scanners that create dense three-dimensional point clouds. This paper investigates the use of high resolution three-dimensional terrestrial laser scans coupled with images to capture geometric range data of complex scenes for surface damage detection and quantification. Although using images with varying resolution to detect cracks is an extensively researched topic, damage detection using laser scanners with and without color images is a new research area that holds many opportunities for enhancing the current practice of visual inspections. Thus, this paper mainly focuses on combining the best features of laser scans and images to create an automatic and effective surface damage detection method, which will reduce the need for skilled labor during visual inspections and allow automatic documentation of related information. A novel surface normal-based damage detection and quantification method that uses the local surface properties extracted from laser scanner data along with color information is presented. Color data provides information in the fourth dimension that enables detecting damage types such as cracks, corrosion, and related surface defects that are generally difficult to identify using only laser scanner. Display omitted •Novel surface damage detection strategy for locating and quantifying defects•Using model properties of surfaces to locate the defective areas on structures•Clustering methodology for grouping defect point into individual damage clusters•Silhouette-based cluster evaluation for optimizing the number of defect clusters•Mesh-grid based quantification system for damage area and volume determination