Feature selection (FS) is an important task which can significantly affect the performance of image classification and recognition. In this paper, we present a feature selection algorithm based on ant colony optimization (ACO). For n features, existing ACO-based feature selection methods need to traverse a complete graph with O(n2) edges. However, we propose a novel algorithm in which the artificial ants traverse on a directed graph with only O(2n) arcs. The algorithm incorporates the classification performance and feature set size into the heuristic guidance, and selects a feature set with small size and high classification accuracy. We perform extensive experiments on two large image databases and 15 non-image datasets to show that our proposed algorithm can obtain higher processing speed as well as better classification accuracy using a smaller feature set than other existing methods. ► A feature selection algorithm based on ant colony optimization is presented. ► The algorithm can obtain higher processing speed than other existing methods. ► The algorithm can select a smaller feature set than other existing methods. ► Higher quality classification results are obtained using such smaller feature set. ► The advantages of the algorithm are proved empirically.