This paper suggests a method to assess the extent to which ultra-short Heart Rate Variability (HRV) features (less than 5 min) can be considered as valid surrogates of short HRV features (nominally 5 min). Short term HRV analysis has been widely investigated for mental stress assessment, whereas the validity of ultra-short HRV features remains unclear. Therefore, this study proposes a method to explore the extent to which HRV excerpts can be shortened without losing their ability to automatically detect mental stress. ECGs were acquired from 42 healthy subjects during a university examination and resting condition. 23 features were extracted from HRV excerpts of different lengths (i.e., 30 s, 1 min, 2 min, 3 min, and 5 min). Significant differences between rest and stress phases were investigated using non-parametric statistical tests at different time-scales. Features extracted from each ultra-short length were compared with the standard short HRV features, assumed as the benchmark, via Spearman's rank correlation analysis and Bland-Altman plots during rest and stress phases. Using data-driven machine learning approaches, a model aiming to detect mental stress was trained, validated and tested using short HRV features, and assessed on the ultra-short HRV features. Six out of 23 ultra-short HRV features (MeanNN, StdNN, MeanHR, StdHR, HF, and SD2) displayed consistency across all of the excerpt lengths (i.e., from 5 to 1 min) and 3 out of those 6 ultra-short HRV features (MeanNN, StdHR, and HF) achieved good performance (accuracy above 88%) when employed in a well-dimensioned automatic classifier. This study concluded that 6 ultra-short HRV features are valid surrogates of short HRV features for mental stress investigation.