Excessive vibration and noise radiation of the track structure can be caused by the operation of high speed trains. Though the track structure is characterized by obvious periodic properties and band gaps, the bandwidth is narrow and the elastic wave attenuation capability within the band gap is weak. In order to effectively control the vibration and noise of track structure, the local resonance mechanism is introduced to broaden the band gap and realize wave propagation control. The locally resonant units are attached periodically on the rail, forming a new locally resonant phononic crystal structure. Then the tuning of the elastic wave band gaps of track structure is discussed, and the formation mechanism of the band gap is explicated. The research results show that a new wide and adjustable locally resonant band gap is formed after the resonant units are introduced. The phenomenon of coupling and transition can be observed between the new locally resonant band gap and the original band gap of the periodic track structure with the band gap width reaching the maximum at the coupling position. The broader band gap can be applied for vibration and noise reduction in high speed railway track structure.