The importance of discovering significant variables from a large candidate pool is now widely recognized in many fields. There exist a number of algorithms for variable selection in the literature. Some are computationally efficient but only provide a necessary condition for, not a sufficient and necessary condition for, testing if a variable contributes or not to the system output. The others are computationally expensive. The goal of the paper is to develop a directional variable selection algorithm that performs similar to or better than the leading algorithms for variable selection, but under weaker technical assumptions and with a much reduced computational complexity. It provides a necessary and sufficient condition for testing if a variable contributes or not to the system. In addition, since indicators for redundant variables aren’t exact zero’s, it is difficult to decide variables whether are redundant or not when the indicators are small. This is critical in the variable selection problem because the variable is either selected or unselected. To solve this problem, a penalty optimization algorithm is proposed to ensure the convergence of the set. Simulation and experimental research verify the effectiveness of the directional variable selection method proposed in this paper.