We present the results of deep near-infrared imaging observations of the $z=3.1$ proto-cluster region in the SSA 22a field taken by MOIRCS mounted on the Subaru Telescope. We observed a 21.7 arcmin $^2$ field to depths of $J=$ 24.5, $H=$ 24.3, and $K=$ 23.9 (5 $\sigma$ ). We examined the distribution of the $K$ -selected galaxies at $z \sim 3$ by using a simple color cut for distant red galaxies (DRGs) as well as a photometric-redshift selection technique. The marginal density excess of DRGs and the photo- $z$ selected objects were found around the two most luminous Ly $\alpha$ blobs (LABs). We investigated the correlation between the $K$ -selected objects and the LABs, and found that several galaxies with stellar mass, $M_* = 10^{9}$ – $10^{11} M_\odot$ , exist in the vicinity of LABs, especially around the two most luminous ones. We also found that 7 of the 8LABs in the field have plausible $K_{\rm s}$ -band counterparts, and the sum of the stellar mass possibly associated with LABs correlates with their luminosity and surface brightness, which implies that the origin of Ly $\alpha$ emission may be closely correlated with their previous star-formation phenomena.