We study the performance of two-phase avalanche detectors based on Gas Electron Multipliers (GEMs) and operated in an electron-avalanching mode in Ar and Xe. Emission, gain, energy resolution and stability characteristics of the detectors were studied. Rather high gains, reaching 5000, and stable operation for several hours were observed in the two-phase Ar avalanche detector using a triple-GEM multiplier. The signals induced by X-rays, β-particles and γ-rays were successfully recorded. Preliminary results were obtained in the two-phase Xe avalanche detector: the maximum gain of the triple-GEM in two-phase Xe and Xe+2%CH 4 was about 200. The results obtained are relevant in the field of two-phase detectors for dark matter searches, coherent neutrino scattering, PET and digital radiography.