As a byproduct of the industrial synthesis of phosphoric acid, there was a large quantity of phosphor gypsum (PG) accumulation, which needs enormous storage space and endangers the environment. The preparation of calcium sulfate whisker (CSW) from waste PG could solve the large amounts of PG accumulation and substitute the plant-based pulp fiber, reducing the pollution of wastewater in paper and pulp industries. In this work, the CSW, which length could reach 230 μm, was fabricated from the PG in a glycerol-water solution. It could be found that the glycerol-water volume ratio of 4:6 was the best ratio that could reduce the water activity and accelerate the phase transition from calcium sulfate dihydrate (CSD) to CSW. Meanwhile, the CSW, after calcination, could prepare the anhydrous calcium sulfate whisker (ACSW) with low solubility in water; when the ACSW substitution rate reached 50%, the property of the complex paper could still satisfy the standard of mechanical and optical for offset book paper.