Precision viticulture has arisen in recent years as a new approach in grape production. It is based on assessing field spatial variability and implementing site-specific management strategies, which can require georeferenced information of the three dimensional (3D) grapevine canopy structure as one of the input data. The 3D structure of vineyard fields can be generated applying photogrammetric techniques to aerial images collected with Unmanned Aerial Vehicles (UAVs), although processing the large amount of crop data embedded in 3D models is currently a bottleneck of this technology. To solve this limitation, a novel and robust object-based image analysis (OBIA) procedure based on Digital Surface Model (DSM) was developed for 3D grapevine characterization. The significance of this work relies on the developed OBIA algorithm which is fully automatic and self-adaptive to different crop-field conditions, classifying grapevines, and row gap (missing vine plants), and computing vine dimensions without any user intervention. The results obtained in three testing fields on two different dates showed high accuracy in the classification of grapevine area and row gaps, as well as minor errors in the estimates of grapevine height. In addition, this algorithm computed the position, projected area, and volume of every grapevine in the field, which increases the potential of this UAV- and OBIA-based technology as a tool for site-specific crop management applications.