Background: Direct hemoperfusion using polymyxin B-immobilized fiber column (PMX-DHP) therapy has been approved for sepsis-associated acute respiratory distress syndrome, but its efficacy for other rapidly progressive interstitial pneumonias (RPIPs) is unclear. The purpose of this study was to examine the efficacy of PMX-DHP therapy for acute respiratory failure in patients with RPIPs, when compared with a historical control receiving conventional treatment without PMX-DHP. Methods: This study comprised 77 patients with RPIPs in our institute between January 2002 and December 2015. The initial 36 patients between January 2002 and March 2007 were treated without PMX-DHP (historical control group), and the following 41 patients between April 2007 and December 2015 were treated with PMX-DHP (PMX-DHP group) once daily for two successive days concurrently with corticosteroids and/or immunosuppressive agents. The 90-day mortality and clinical factors were compared between the groups. Cox proportional hazards models were constructed to analyze 90-day mortality and identify predictors. Results: The 90-day mortality rate was significantly lower in the PMX-DHP group than in the controls (41.5% versus 66.7%, p = 0.019). PMX-DHP therapy was significantly associated with mortality (hazard ratio 0.505; 95% confidence interval, 0.270–0.904; p = 0.032). There were significant differences in the serial changes in the PaO2/FiO2 ratio, SOFA score, and blood neutrophil counts from days 0–5 after PMX-DHP between the survivor and non-survivor groups (p = 0.015, p < 0.001, p = 0.035, respectively). The improved PaO2/FiO2 ratio on day 3 significantly correlated with the change in blood neutrophil counts (rs = −0.431, p = 0.006). Conclusions: PMX-DHP therapy may be effective in RPIPs patients accompanied by acute respiratory failure and is expected to reduce mortality rates.