This article develops an adaptive group lasso estimator for factor models with both global and group-specific factors. The global factors can affect all variables, whereas the group-specific factors are only allowed to affect the variables within a certain group. We propose a new method to separately identify the spaces spanned by global and group-specific factors, and we develop a new shrinkage estimator that can consistently estimate the factor loadings and determine the number of factors simultaneously. The asymptotic result shows that the proposed estimator can select the true model specification with a probability approaching one. An information criterion is developed to select the optimal tuning parameters in the shrinkage estimation. Monte Carlo simulations confirm our asymptotic theory, and the proposed estimator performs well in finite samples. In an empirical application, we implement the proposed method to a dataset consisting of Eurozone, United States, and United Kingdom macroeconomic variables, and we detect one global factor, one U.S.-specific factor, and one Eurozone-specific factor.