View factors for sky, trees, and buildings are three important parameters of the urban outdoor environment that describe the geometrical relationship between different surfaces from the perspective of radiative energy transfer. This study develops an approach for accurately estimating sky view factor (SVF), tree view factor (TVF), and building view factor (BVF) of street canyons in the high-density urban environment of Hong Kong using publicly available Google Street View (GSV) images and a deep-learning algorithm for extraction of street features (sky, trees, and buildings). As a result, SVF, TVF, and BVF maps of street canyons are generated. Verification using reference data of hemispheric photography from field surveys in compact high-rise and low-rise areas shows that the GSV-based VF estimates have a satisfying agreement with the reference data (all with R2 > 0.95), suggesting the effectiveness and high accuracy of the developed method. This is the first reported use of hemispheric photography for direct verification in a GSV-based streetscape study. Furthermore, a comparison between GSV-based and 3D-GIS-based SVFs shows that the two SVF estimates are significantly correlated (R2 = 0.40, p < 0.01) and show better agreement in high-density areas. However, the latter overestimates SVF by 0.11 on average, and the differences between them are significantly correlated with street trees (R2 = 0.53): the more street trees, the larger the difference. This suggests that a lack of street trees in a 3D-GIS model of street environments is the dominant factor contributing to the large discrepancies between the two datasets. •GSV-based view factor estimation method with a deep-learning algorithm is proposed.•Sky, tree, and building view factors of street canyons are accurately quantified.•Hemispheric photography is used for direct verification of view factor estimates.•Comparison analysis of GSV-based and 3D-GIS-based methods is conducted.•The impacts of tree canopy and building density on SVF estimates are investigated.