In order to find a highly efficient, environment-friendly magnetic refrigerant, direct measurements of the adiabatic temperature change Δ T adb is required. Here, in this work a simple setup for the Δ T adb measurement is presented. Using a permanent magnet Halbach array with a maximum magnetic field of 1.8 T and a rate of magnetic field change of 5 T/s, accurate determination of Δ T adb is possible in this system. The operating temperature range of the system is from 100 K to 400 K, designed for the characterization of materials with potential for room temperature magnetic refrigeration applications. Using the setup, the Δ T adb of a first-order and two second-order compounds have been studied. Results from the direct measurement for the first-order compound have been compared with Δ T adb calculated from the temperature and magnetic field dependent specific heat data. By comparing results from direct and indirect measurements, it is concluded that for a reliable characterization of the magnetocaloric effect, direct measurement of Δ T adb should be adopted.