This study conducted a seepage experiment of broken shale under cyclic loading and unloading conditions to investigate the permeability characteristics of broken shale with different particle sizes under stress disturbance. There were six groups of broken shale samples with different particle sizes, and three cycles of loading and unloading were conducted to test the permeability of broken shale. The influence of the number of cycles and shale particle size on permeability damage was analyzed. Furthermore, the stress-seepage model of broken shale was established to analyze the influence of the number of cycles and shale particle size on the variation of stress sensitivity of broken shale under different loading and unloading stages. The results show that the initial permeability, permeability damage, and stress sensitivity of broken shale are negatively correlated with the number of cycles and that the first cycle has a significant impact on them. As the size of the broken shale particles increases, the rate of change in shale permeability also increases before and after conducting experiments. This results in a greater shale damage permeability. However, the impact of the number of cycles on shale permeability damage decreases. Additionally, the shale’s stress sensitivity becomes stronger. It is also found that the seepage characteristics between the small particle size (G1–G4, 0–6.8 mm) and the large particle size (G5–G6, 7–15 mm) are different in the broken shale samples with six different particle sizes. In addition, the change of the shale particle structure under cyclic loading and unloading conditions is discussed, and the mechanism of the shale permeability change is studied. This study indicates the seepage characteristics of broken shale under cyclic loading and unloading conditions, which has a certain guiding significance for the safe and efficient extraction of shale gas wells.