•A novel sequence coding-based 1D-GLCM method was proposed.•The recognition accuracy for mild mental stress increased from 88.57% to 98.57%.•The Energy feature is more discriminating compared with other texture features. Mild mental stress, which represents the type of stress that occurs in daily life without a specific time limit, can accumulate and eventually lead to many physical and psychological disorders. Moreover, the overall heart rate fluctuations induced by mild mental stress are less significant, posing a great challenge to stress detection. However, current traditional methods have the drawback of limited ability to quantify local details and spatial variations in the pattern of autonomic fluctuations during early mental stress. To address these issues, in this study, a novel inter-beat interval-based analysis model is proposed, which is comprised of sequence coding-based transformation and construction of gray level co-occurrence matrix (GLCM). The inter-beat intervals were first mapped into binary symbolic sequences, and then the word sequences were constructed. Then four texture features, i.e., Contrast, Correlation, Energy, and Homogeneity, were extracted from the derived GLCM constructed from the word sequences. Additionally, other classic multi-domain features were also extracted. Statistical analysis and Spearman correlation analysis were then performed, respectively. After feature selection by recursive feature elimination, different feature subsets were fed into the support vector machine to identify whether subjects were suffering from mental stress. The results showed that Energy was not only statistically different but also associated with altered mental states. The accuracy using only classic features was 88.57%, while combining classic and texture features increased the highest accuracy to 98.57%. This study provides promising methodology to detecting mild mental stress and potential insight into the mechanisms underlying its pathophysiology.