Excavated rock and sediment in which arsenic release naturally occurs is necessary to be treated with an appropriate technique such as chemical immobilization. The physical condition such as particle size ratio of rock to the immobilization material possibly enhances/suppresses the effects of arsenic immobilization because arsenic should be contacted and reacted with the immobilization material in the rock pores for the immobilization. This study prepared the up-flow column percolation test using the excavated rocks with the different particle sizes and immobilization material with constant particle size to determine a suitable ratio of the diameter of the average particle size in the rock samples to that in the immobilization material. The column percolation test showed that the contribution of arsenic immobilization by the immobilization material was higher in the larger particle size of excavated rock while the arsenic retention capacity of excavated rock was high in the smaller particle size. The arsenic concentrations in the leaching water were the lowest and the percentage of arsenic retained during the column percolation test was the highest in the particle size of 1.0–2.0 mm of excavated rock. Based on these results, this study suggests that the suitable particle size ratio of excavated rock to the immobilization material was 2 to 10.