We generalize the modular invariance approach to include the half-integral weight modular forms. Accordingly the modular group should be extended to its metaplectic covering group for consistency. We introduce the well-defined half-integral weight modular forms for congruence subgroup Γ(4N) and show that they can be decomposed into the irreducible multiplets of finite metaplectic groupΓ4N. We construct concrete expressions of the half-integral/integral modular forms for Γ (4) up to weight 6 and arrange them into the irreducible multiplets of Γ4. We present three typical models with Γ4 modular symmetry for neutrino masses and mixing, and the phenomenological predictions of each model are analyzed numerically.