Background Neoadjuvant chemotherapy may improve outcomes in gastric cancer. Tumor responses can be evaluated with RECIST, Japanese Classification of Gastric Carcinoma (JCGC), and histological criteria. These approaches have not yet been compared. Methods We analyzed two phase II trials of neoadjuvant chemotherapy using S-1 plus cisplatin. JCOG0210 included patients with linitis plastica and large ulcero-invasive tumors, whereas JCOG0405 comprised those with para-aortic or bulky lymph node metastases. Radiologic evaluations were conducted using RECIST in JCOG0405 and JCGC criteria in JCOG0210, because the latter included many patients without measurable lesions. A histological responder was defined as a patient in whom one third or more of the tumor was affected. The hazard ratios (HR) for death between responders and non-responders and response rate differences between short- and long-term survivors were estimated. Results In JCOG0210 ( n  = 49), HR was 0.54 in JCGC responders ( P  = 0.059) and 0.40 in histological responders ( P  = 0.005). The difference in response rates between short- and long-term survivors using histological criteria (34 %, P  = 0.023) was greater than that using JCGC criteria (24 %, P  = 0.15). In JCOG0405 ( n  = 51), HR was 0.67 in RECIST responders ( P  = 0.35) and 0.39 in histological responders ( P  = 0.030). In short- and long-term survivors, respectively, RECIST response rates were 62  and 67 % ( P  = 0.77), whereas histological response rates were 33  and 63 % ( P  = 0.048). Conclusions Histological criteria showed higher response assessment validity than RECIST or JCGC criteria and yielded the best surrogate endpoint for overall survival.