Abstract The recovery of low-quality waste heat is a major problem in energy utilization. In order to solve this problem and improve energy utilization, the research group designed a low-quality waste heat power generation device with Roots power machine as the core. However, the device has poor ability to adjust the rotation speed and it is difficult to generate electricity stably. The fundamental reason is that the system has many variables and strong coupling. According to the actual working conditions, the power of the device is 10 kW, and the fluctuation range should be within ± 7%. On the one hand, it can be improved by hardware, on the other hand, the design of software is also very critical. At present, through the investigation of domestic and foreign researches on the control system, it is found that the stability of the system is gradually improved, but the problem of strong coupling between variables has not been effectively solved. Therefore, the research group modeled the variables in the system and obtained a coupled model. Based on the couple model, the research group introduced nonlinear multi-model adaptive closed-loop decoupling control and designed a control system. The simulation results show that the maximum overshoot of the control system is 3.9%, the adjustment time is also reduced, and it is stable in low quality waste heat recovery device. Experimental results show that under the control of the system, the rotational speed of roots motor can keep stable, the maximum deviation is not more than 21.4 r/min, and the fluctuation range is within ± 7%, which meets the requirements of the index. This has laid the foundation for the follow-up research of grid-connected power generation.