•Effects of rubber modification on fracture toughness of epoxy were investigated.•Fracture toughness tests were conducted under mode I and II loadings.•The fracture toughness values were evaluated using two types of J-integral values.•The rubber modification effect depended on the type of J-integral value. In the present study, the effects of rubber modification on the joint strength and fracture toughness of epoxy adhesives under mode I and II loading were investigated using Arcan-type adhesively bonded joints. The results of the fracture tests under mode I loading revealed the occurrence of stress whitening owing to triaxial tensile stresses near the crack tip. The joint strength and fracture toughness were calculated using the J-integral value, where the adhesive layer was considered as an elastic–plastic material. The crack-growth energy was also calculated using the J-integral value, where the adhesive layer was considered as an elastic material. The results revealed that rubber modification under mode I loading induced an increase in the joint strength, fracture toughness, and crack-growth energy. Rubber modification under mode II loading did not induce stress whitening near the crack tip owing to low triaxial tensile stresses. However, macroscopic void growth and significant plastic deformation were observed in the adhesive layer. Rubber modification induced a slight decrease in the joint strength; however, it induced a significant increase in the fracture toughness owing to the growth of the plastic zone in the adhesive layer. The calculated crack-growth energy decreased slightly, in accordance with the joint strength. It was concluded that rubber modification contributed to an increase in the crack-growth resistance under mode I loading and not under mode II loading. Furthermore, a correlation was observed between the joint strength and crack-growth energy.