The “requisition-compensation balance policy” is a basic arable land protection policy promulgated by the Chinese government to maintain the dynamic balance of the total arable land. Since the implementation of the “requisition-compensation balance policy”, its important role in the conservation of arable land has been widely noted. Multiple studies indicate the effectiveness of the policy by estimating the quantity and quality change of arable land. In addition, the defects and negative externalities of the policy are disputed. The overall goal of this study is to estimate the provincial-level effectiveness of the “requisition-compensation balance policy” in mainland China in two periods (i.e., 2000—2010; 2010—2020) from four perspectives: arable land quantity dynamic balance; arable land productivity balance; farming distance; and sustainability of arable land use. The results showed that, first, the arable land quantity balance was achieved during 2000–2020 from a national perspective. There are still 19–23% of provinces have failed to reach the quantity balance. Second, the decrease in China’s total arable land productivity was exacerbated from 9612.1 thousand tons. to 31254.6 thousand ton. The average potential yield balance index was less than 1 for nearly all provinces during 2000–2010 and became even worse in the next decade because of occupying superior arable land while compensating for inferior arable land. The conservation of arable land productivity has become more important than the conservation of quantity. Third, due to the lack of constraints on the farming distance changes in the “requisition-compensation balance policy”, most provinces convert arable land around urban and rural areas to built-up land while replenishing land far from residential areas without providing adequate agricultural infrastructure, which leads to an increase in the cost of farming and consequently to an increase in the marginalization of arable land. During 2010–2020, the average farming distance of compensated arable land at the provincial level reached 2–7 times that of occupied arable land. However, the average farming distance of the whole arable land at the provincial level decreased by 3.82–63.88% during the same period. This contradiction is mainly due to increasing marginalization and opportunity costs resulting in arable land with high farming distance (including arable land that was compensated in the past) to be used with low intensity or even abandoned and thus identified as other land use types by remote sensing classification models. This factor outweighed that of “occupy nearby arable land while compensating farther one”, resulting in a reduction in the provincial average farming distance. Fourth, the percentage of sustainable compensated arable land in most provinces was lower than 70%. This indicated that the utilization and protection of arable land in these provinces was insufficiently implemented and monitored. Challenges along optimization of the “requisition-compensation balance policy” were discussed from two respects: data, theory and methodology and policy design and implementation. The authors argue that a more comprehensive “requisition-compensation balance policy” should be designed considering not only the quantity and productivity of arable land but also the farming distance, sustainability and ecological protection. A differentiated regulation mechanism of arable land requisition compensation in trans-provincial areas should be formulated. This study can provide guidance for optimizing the implementation of regional arable land protection and can also provide a reference for other countries to protect arable land. •Estimate effectiveness of the arable land requisition-compensation balance policy in mainland China in the last 20 years.•Estimate quantity and productivity change of arable land in mainland China in the last 20 years.•The average farming distance of the compensated arable land was 2–7 times that of the occupied one.•The average farming distance decrease 3.82–63.88% because of explicit or implicit farmland abandoned.•There is a lack of control over the sustainable use of compensated arable land in mainland China.