Face recognition is one of the most important and widely applicable research problems in the subject area of machine learning and computer vision. Extraction of features, local or holistic, is the fundamental step and subspace method has been a natural choice for facial feature extraction. Among these, methods like PCA, ICA, LDA aim to reduce the dimension of the data while retaining the statistical separation property between distinct classes. Unlike the traditional ICA, in which the whole face image is stretched into a vector before calculating the independent components (ICs), Block ICA (B-ICA) partitions the facial images into blocks and takes the block as the training vector. Since the dimensionality of the training vector in B-ICA is much smaller than that in traditional ICA, reduction in face recognition error is expected. The objective of ICA is to find a separation matrix and it is achieved by a process of optimization, such as maximization of non-Gaussianity, maximum likelihood estimation, and minimization of mutual information. We observe here that the gradient-based learning can be efficiently and effectively achieved by the application of swarm-based optimization. We propose here the application of our Gradient-based Swarm Optimization method for Block ICA, where gradient information is combined with conventional swarm search to optimize the contrast function. We compare our method with B-ICA on three benchmark image data sets and show that our method achieved a better recognition rate compared to B-ICA in different block sizes with 70 % , 80 % and 90 % data used for training the model.