Recent advances on remote sensing techniques allow easier access to imaging spectrometer data. Manually labeling and processing of such collected hyperspectral images (HSIs) with a vast quantities of samples and a large number of bands is labor and time consuming. To relieve these manual processes, machine learning based HSI processing methods have attracted increasing research attention. A major assumption in many machine learning problems is that the training and testing data are in the same feature space and follow the same distribution. However, this assumption doesn't always hold true in many real world problems, especially in certain HSI processing problems with extremely insufficient or even without training samples. In this letter, we present a transfer learning framework to address this unsupervised challenge (i.e., without training samples in the target domain), by making the following three main contributions: 1) to the best of our knowledge, this is the first time for transfer learning framework to be used for the classification of totally unknown target HSI data with no training samples; 2) the characteristics of HSI are learned on dual spaces to exploit its structure knowledge to better label HSI samples; and 3) two specific new scenarios suitable for transfer learning are investigated. Experimental results on several real world HSIs support the superiority of the proposed work.