► Support Vector Regression approach is proposed to analyze the asymmetry in car following behavior. ► Intensity difference leads to a neutral line separating the speed-space diagram into two areas. ► The neutral line determines the tilt angle and position of the hysteresis in the speed-space diagram. ► The leading vehicle’s states affect the internal and external shapes of the hysteresis. ► Magnitudes of deviations from equilibrium for micro- and macro-hysteresis are related. This paper presents a self-learning Support Vector Regression (SVR) approach to investigate the asymmetric characteristic in car-following and its impacts on traffic flow evolution. At the microscopic level, we find that the intensity difference between acceleration and deceleration will lead to a ‘neutral line’, which separates the speed-space diagram into acceleration and deceleration dominant areas. This property is then used to discuss the characteristics and magnitudes of microscopic hysteresis in stop-and-go traffic. At the macroscopic level, according to the distribution of neutral lines for heterogeneous drivers, different congestion propagation patterns are reproduced and found to be consistent with Newell’s car following theory. The connection between the asymmetric driving behavior and macroscopic hysteresis in the flow-density diagram is also analyzed and their magnitudes are shown to be positively related.