•Identify noise local image features by comparing features from disease group and healthy control group.•Support vector machine is used to classify image features.•Disease related regions are identified from three different diseases (Alzheimer's disease, Parkinson's disease and bipolar disorder).•The algorithm can be used to analyze MR images from heterogeneous datasets. Detecting brain structural changes from magnetic resonance (MR) images can facilitate early diagnosis and treatment of neurological and psychiatric diseases. Many existing methods require an accurate deformation registration, which is difficult to achieve and therefore prevents them from obtaining high accuracy. We develop a novel local feature based support vector machine (SVM) approach to detect brain structural changes as potential biomarkers. This approach does not require deformation registration and thus is less influenced by artifacts such as image distortion. We represent the anatomical structures based on scale invariant feature transform (SIFT). Likelihood scores calculated using feature-based morphometry is used as the criterion to categorize image features into three classes (healthy, patient and noise). Regional SVMs are trained to classify the three types of image features in different brain regions. Only healthy and patient features are used to predict the disease status of new brain images. An ensemble classifier is built from the regional SVMs to obtain better prediction accuracy. We apply this approach to 3D MR images of Alzheimer's disease, Parkinson's disease and bipolar disorder. The classification accuracy ranges between 70% and 87%. The highly predictive disease-related regions, which represent significant anatomical differences between the healthy and diseased, are shown in heat maps. The common and disease-specific brain regions are identified by comparing the highly predictive regions in each disease. All of the top-ranked regions are supported by literature. Thus, this approach will be a promising tool for assisting automatic diagnosis and advancing mechanism studies of neurological and psychiatric diseases.