•Co/Gd3Ga5O 12/Y3Fe5O12 heterostructures were grown by laser molecular beam epitaxy.•Selective YIG and Co magnetization curves were measured by MOKE and XMCD techniques.•Clear proximity effects were observed in heterostructures with 1 nm thick GGG spacer.•Ferromagnetic interaction between Co and YIG was explained by “orange peal” effect. Interlayer magnetic coupling has been studied in the epitaxial system consisting of few nanometer thick magnetic layers of cobalt and yttrium iron garnet (YIG) separated by non-magnetic spacers of gadolinium gallium garnet (GGG). The samples were grown by laser molecular beam epitaxy on GGG(1 1 1) substrates. The layer morphology and crystal structure were characterized by atomic force microscopy, X-ray and electron diffraction techniques. MOKE and XMCD methods were applied for element selective study of the magnetization reversal in Co and YIG layers. For the Co/YIG/GGG heterostructures it has been found that magnetization loops of YIG and Co exhibit the same values of coercive field and are of the same shape indicating strong magnetic coupling between Co and YIG layers. In opposite, when YIG and Co are separated by a 1 nm thick GGG spacer layer, the coercive field of Co becomes very different from that of YIG. Moreover the center of YIG loop gets shifted in field whereas the sign and value of this shift depend on the current magnetic state of the Co layer. The magnetic interaction between Co and YIG can be interpreted in the terms of 2–3 Oe magnetic field induced in the YIG layer by the adjacent Co layer. This magnetic field is likely caused by the “orange peel” effect arising due to granular structure of Co layer.