Layered gallium telluride (GaTe) has attracted much attention recently, due to its extremely high photoresponsivity, short response time, and promising thermoelectric performance. Different from most commonly studied two-dimensional (2D) materials, GaTe has in-plane anisotropy and a low symmetry with the C 2h 3 space group. Investigating the in-plane optical anisotropy, including the electron–photon and electron–phonon interactions of GaTe is essential in realizing its applications in optoelectronics and thermoelectrics. In this work, the anisotropic light-matter interactions in the low-symmetry material GaTe are studied using anisotropic optical extinction and Raman spectroscopies as probes. Our polarized optical extinction spectroscopy reveals the weak anisotropy in optical extinction spectra for visible light of multilayer GaTe. Polarized Raman spectroscopy proves to be sensitive to the crystalline orientation of GaTe, and shows the intricate dependences of Raman anisotropy on flake thickness, photon and phonon energies. Such intricate dependences can be explained by theoretical analyses employing first-principles calculations and group theory. These studies are a crucial step toward the applications of GaTe especially in optoelectronics and thermoelectrics, and provide a general methodology for the study of the anisotropy of light-matter interactions in 2D layered materials with in-plane anisotropy.