A zirconium(IV)‐based metal organic framework, composed of 4,4′‐biphenyldicarboxylic acid ligands and Zr6O4(OH)4 clusters, was successfully fabricated. Characterizations were performed on fourier transform infrared spectroscopy, scanning electron microscopy, X‐ray diffraction, Brunauer‐Emmett‐Teller, and thermogravimetric analysis, which confirmed that it possessed large specific surface area, high pore volume, and strong acid resistant stability. Furthermore, the prepared material containing biphenyl skeleton and a large number of Zr‐O bonds, can grasp acid herbicides especially phenoxyacetic acid herbicides with aromatic structures through π‐π interaction, hydrophobic interaction and Zr‐O‐H+ bonds. Based on these advantages, a method was developed for the determination of four phenoxyacetic acid herbicides from vegetable samples. Under the optimal conditions, wide linearities from 0.3 to 250 μg/L and low limits of detection from 0.1 to 0.5 μg/L were obtained. The intra‐ and interday relative standard deviations were 1.56‐3.92 and 5.01‐7.65%, respectively. The proposed method was applied to analyze phenoxyacetic acid herbicides residues in the tomato, cucumber, and white gourd samples. The satisfactory recoveries (86.12–103.44%) for the spiked samples in vegetable samples were achieved which demonstrated the method was an efficient pretreatment procedure and has a potential application for the trace determination of phenoxyacetic acids from a complex matrix.