•Ultrasonic vibratory stress relief (UVSR) was first applied to a random size plate.•The local yield stress of AA6082 is reduced by at least 27% during the UVSR.•The vibratory stress introduced by ultrasonic vibration was obtained accurately.•The stress relief ratio was 53% and the distribution interval was reduced by 57%. Ultrasonic vibratory stress relief (UVSR) is a novel method for residual stress relieving of metal parts. In this paper, the UVSR method was applied to treat a random-sized plate, which is not designed to match with the resonant frequency of the ultrasonic transducer. The UVSR system includes an ultrasonic generator, a 20 kHz ultrasonic transducer and specially designed fixtures to exert ultrasonic vibration to the treated parts. Based on the experimental results, the average stress relief ratio reached 53% and the final residual stress distribution interval was reduced by 57. The vibratory stress introduced by the ultrasonic vibration was obtained accurately by matching the FE model with the measured vibration amplitude distribution. It is found that local yield stress, instead of macroscopic yield stress, should be used as the threshold of residual stress relieving. Moreover, if the local yield stress of a material is constant, the local yield stress of 6082 aluminum alloy is reduced by at least 27% during the UVSR. Finally, it is found that the residual stress relief rate in one direction is proportional to the vibratory stress introduced in that direction.