•Habitat selection and compensation for Doppler effect are incorporated into algorithm.•Algorithm possesses the quantum search operator and mechanical search operator.•Self-adaptive local search is proposed.•Algorithm shows significant performance in comparison with more than 20 methods. A novel bat algorithm (NBA) is proposed for optimization in this paper, which focuses on further mimicking the bats’ behaviors and improving bat algorithm (BA) in view of biology. The proposed algorithm incorporates the bats’ habitat selection and their self-adaptive compensation for Doppler effect in echoes into the basic BA. The bats’ habitat selection is modeled as the selection between their quantum behaviors and mechanical behaviors. Having considered the bats’ self-adaptive compensation for Doppler effect in echoes and the individual’s difference in the compensation rate, the echolocation characteristics of bats can be further simulated in NBA. A self-adaptive local search strategy is also embedded into NBA. Simulations and comparisons based on twenty benchmark problems and four real-world engineering designs demonstrate the effectiveness, efficiency and stability of NBA compared with the basic BA and some well-known algorithms, and suggest that to improve algorithm based on biological basis should be very efficient. Further research topics are also discussed.