The excitation functions for quasielastic scattering of ^{22}Ne+^{248}Cm, ^{26}Mg+^{248}Cm, and ^{48}Ca+^{238}U are measured using a gas-filled recoil ion separator. The quasielastic barrier distributions are extracted for these systems and are compared with coupled-channel calculations. The results indicate that the barrier distribution is affected dominantly by deformation of the actinide target nuclei, but also by vibrational or rotational excitations of the projectile nuclei, as well as neutron transfer processes before capture. From a comparison between the experimental barrier distributions and the evaporation residue cross sections for Sg (Z=106), Hs (108), Cn (112), and Lv (116), it is suggested that the hot fusion reactions take advantage of a compact collision, where the projectile approaches along the short axis of a prolately deformed nucleus. A new method is proposed to estimate the optimum incident energy to synthesize unknown superheavy nuclei using the barrier distribution.