The study investigated the evocation of mental imagery and emotional responses when English–Chinese bilinguals read classic Chinese poems and their English translations to examine (1) the target readers' formation of non-verbal text representations of Chinese poetry and (2) whether different translations affect the target readers' imagery cognition. A total of 20 English–Chinese speaker students enrolled in a Chinese university read a classic Chinese poem in Chinese and its four versions of translation in English. Through questionnaires and interviews, participants rated the visualized words used in the poems for the degree of mental imagery and emotional response evoked based on three indicators of narrative process, salience value, and emotive validity in the theoretical framework of visual grammar. Results showed considerable individual variances in the cognitive differences in forming mental imagery in all versions of the poems and there were also effects of translation strategy. Moreover, visual language information in poetry reading and its translations evoked different emotional responses depending on the use of visual words with cultural features. Our study demonstrates the applicability and accessibility of visual language in describing different readers' mental imagery and the interrelation and interaction between the poetry language system and the emotional, social, and cultural contexts involved in poetry translation.