One source of learning in universities is a digital library. In the era of industry 4.0, most universities have implemented digital libraries in supporting the learning process. However, the reality shows that digital library management is still ineffective. Therefore, the implementation of digital libraries needs to be evaluated for determining the digital library effectiveness used as learning resources in supporting the learning process in universities. Many evaluation tools are used to evaluate the effectiveness of digital libraries but have not provided accurate recommendation results to support decision-making. This research presents an innovation in the form of an evaluation tool that can be used to evaluate the digital library effectiveness in universities. That evaluation tool is called the Alkin-WP-based digital library evaluation software. This software is a desktop platform that contains aspects of measuring the digital library effectiveness by referring to the components of the Alkin evaluation model and the WP (Weighted Product) method. This research aimed to show the effectiveness level of the utilization of Alkin-WP-based digital library evaluation software. This research method was R & D (Research & Development) which refers to the ten development stages of the Borg and Gall model. In this research, development was focused only on a few stages, included: usage trials, final product revision, dissemination, and implementation. The subjects involved in assessing the implementation/utilization of the Alkin-WP-based digital library evaluation software were 35 people, in the usage trials were six people, in product revision were three people, and at the stage of dissemination were 15 people. The tools used to collect data were questionnaires and interview guidelines. The data analysis technique used was descriptive quantitative. The effectiveness level of utilizing the Alkin-WP-based digital library evaluation software was 88.34%. It showed that the evaluation software had effective. The impact of this research results on the scientific field of educational evaluation is being able to show the existence of a new evaluation tool based on educational evaluation and artificial intelligence. That evaluation tool can easier for library heads to make policies for revamping digital library services based on accurate recommendations. Doi: 10.28991/esj-2021-01308 Full Text: PDF