The ground-borne vibration induced by the high-speed train operation in the loess area will adversely affect the safety of surrounding buildings and the daily life of residents. The ground vibration responses of the viaduct section and the subgrade-viaduct transition section of the Xi'an-Baoji high-speed railroad in China were measured in the longitudinal, transversal, and vertical directions. The vibration characteristics in time and frequency domains were analyzed based on the Fourier transform and the continuous wavelet transform (CWT). The acceleration levels were further calculated to evaluate the vibration exceedance of the viaduct section and the transition section. The CWT-based energy analysis method was used to analyze the vibration energy transmission pattern and the cumulative rate in the surrounding site. The analysis results show that both the viaduct section and the transition section exists vibration amplification zone, and the degree of vibration amplification in the horizontal direction is more significant than that in the vertical direction. It is also found that the horizontal direction is more prone to amplification than the vertical direction and it has the greater amplification degree because of the wave refraction. Two indicators of energy accumulation level and energy accumulation rate are proposed based on CWT. The two indicators are greater in the amplification zone than in other regions, and greater in the viaduct section than in the transition section. It shows that the acceleration vibration levels of both the viaduct section and the transition section exceed the vibration limits specified in the code. Some effective measures for the track vibration and noise reduction are finally recommended. •The CWT-based energy analysis method is used to analyze the vibration energy transmission pattern.•The vibration amplification phenomenon is found at 20 m and 55 m in the transition section.•The ground vibration dominant frequencies are characteristic frequencies integer multiples of carriage bogies.•The horizontal direction is more prone to amplification than the vertical direction.•The energy accumulation and energy transfer efficiency are higher at the amplification area.