Owning to the powerful non-linear fitting ability and limited battery knowledge requirement, the deep learning has drawn increasing attention to state of charge (SOC) estimation for Li-ion batteries in electric vehicles. However, most of the previous research has tended to focus on the estimating precision rather than computational efficiency and hardware feasibility. This paper proposed a complete hardware-software co-design solution based on a modified self-attention network. On Turnigy Graphene battery dataset, the proposed light-weighted model achieves 1.31% RMSE with only 457 trainable parameters and 1.47K floating point operations. To map and infer the proposed model more efficiently, a dedicated hardware with high performance mode and low power mode was brought forward, which costs less computing time to estimate SOC than previous approaches on MCU and GPU. To the best of our knowledge, it is the first study developing specialized accelerator circuit for deep learning SOC estimation.