This study examined the effects of alloying elements such as Re and Ta on the microstructural evolution of recrystallized W under proton (1 MeV H+) and self-ion (18 MeV W6+) irradiations at 800 °C. Although the number density of voids increased with increasing proton-induced damage level, the void density in W-Re (R) and W-Ta (R) alloys were lower than that of pure W (R). Herein, the addition of Re and Ta to W suppresses the void formation process. In the proton-irradiated W-3%Re (R), a lot of dislocation loops were observed at 0.05 dpa which is the stage of nucleation. The evolution process up to 0.2 dpa was characterized by loop growth via the absorption of clusters and point defects. The dislocation loops then coalesce and grow large, and the dislocation lines become tangled at 1 dpa. At 0.05 dpa, the dislocation loops in pure W (R) have already evolved into the tangled dislocations. Solute Re may inhibit the mobility of small dislocation loops and SIA clusters. In W-3%Ta (R) irradiated at 0.05 and 0.2 dpa, the coalescence process of the elongated dislocation loops was observed. Solute Ta may inhibit the mobility of SIA clusters. Although no voids and rafts were observed in self-ion irradiated W-3%Re (R) to 0.2 dpa, not only dislocation loops but also voids and rafts were observed in pure W (R) to 0.2 dpa. Solute Re may inhibit the mobility of small dislocation loops and SIA clusters, so that solute Re would suppress the raft formation and then the void formation under self-ion irradiation. The dispersed barrier model was applied to the irradiation hardening of the proton-irradiated W-3%Re (R) up to 0.2 dpa before developing dislocation lines. The hardening in the proton-irradiated W-3%Re (R) up to 0.2 dpa is mainly due to TEM-visible voids and dislocation loops.