Some previous studies have shown that chaotic dynamics in the balanced state, i.e., one with balanced excitatory and inhibitory inputs into cortical neurons, is the underlying mechanism for the irregularity of neural activity. In this work, we focus on networks of current-based integrate-and-fire neurons with delta-pulse coupling. While we show that the balanced state robustly persists in this system within a broad range of parameters, we mathematically prove that the largest Lyapunov exponent of this type of neuronal networks is negative. Therefore, the irregular firing activity can exist in the system without the chaotic dynamics. That is the irregularity of balanced neuronal networks need not arise from chaos.