The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment. And they can be well modeled by a hyper-Laplacian. However, the use of such distributions generally leads to challenging non-convex, non-smooth and non-Lipschitz problems, and makes existing algorithms very slow for large-scale applications. Together with the analytic solutions to <inline-formula><tex-math notation="LaTeX">\ell _{p}</tex-math> <inline-graphic xlink:href="shang-ieq1-2748590.gif"/> </inline-formula>-norm minimization with two specific values of <inline-formula><tex-math notation="LaTeX">p</tex-math> <inline-graphic xlink:href="shang-ieq2-2748590.gif"/> </inline-formula>, i.e., <inline-formula> <tex-math notation="LaTeX">p=1/2</tex-math> <inline-graphic xlink:href="shang-ieq3-2748590.gif"/> </inline-formula> and <inline-formula><tex-math notation="LaTeX">p=2/3</tex-math> <inline-graphic xlink:href="shang-ieq4-2748590.gif"/> </inline-formula>, we propose two novel bilinear factor matrix norm minimization models for robust principal component analysis. We first define the double nuclear norm and Frobenius/nuclear hybrid norm penalties, and then prove that they are in essence the Schatten-<inline-formula> <tex-math notation="LaTeX">1/2</tex-math> <inline-graphic xlink:href="shang-ieq5-2748590.gif"/> </inline-formula> and <inline-formula><tex-math notation="LaTeX">2/3</tex-math> <inline-graphic xlink:href="shang-ieq6-2748590.gif"/> </inline-formula> quasi-norms, respectively, which lead to much more tractable and scalable Lipschitz optimization problems. Our experimental analysis shows that both our methods yield more accurate solutions than original Schatten quasi-norm minimization, even when the number of observations is very limited. Finally, we apply our penalties to various low-level vision problems, e.g., text removal, moving object detection, image alignment and inpainting, and show that our methods usually outperform the state-of-the-art methods.