Development of subclassification of intermediate-stage hepatocellular carcinoma (HCC) by treatment suitability is in demand. We aimed to identify predictors that define treatment refractoriness against locoregional(transarterial chemoembolization(TACE) or thermal ablation) and surgical therapy. This multicenter retrospective study enrolled 1167 HCC patients between 2015 and 2021. Of those, 209 patients were initially diagnosed with intermediate-stage HCC. Treatment refractoriness was defined as clinical settings that meets the following untreatable progressive conditions by TACE (1) 25% increase of intrahepatic tumor, (2) transient deterioration to Child-Pugh class C, (3) macrovascular invasion or extrahepatic spread, within one year. We then analyzed factors contributing to treatment refractoriness. The Child-Pugh score/class, number of tumors, infiltrative radiological type, and recurrence were significant factors. Focusing on recurrence as a predictor, median time to untreatable progression (TTUP) was 17.2 months in the recurrence subgroup whereas 35.5 months in the initial occurrence subgroup (HR, 2.06; 95% CI, 1.44-2.96; P = 0.001). Median TTUP decreased in cases with more later times of recurrence (3-5 recurrences, 17.3 months; ≥ 6 recurrences, 7.7 months). Recurrence, even more at later times, leads to increased treatment refractoriness. Early introduction of multidisciplinary treatment should be considered against HCC patients after multiple recurrent episodes.