•Independent laser beams are spatially modulated as orthogonal optical modes.•The transmission is optimized by correlating the transverse intensities.•Separation and identification of modes use matching filter based on inner product model.•Experimental identification is performed by crossing analysis on two copies of the filtered modes.•Identification quality is quantified by Sarle's bimodality coefficient. The study proves the concept of using a system with two inputs laser-beam, modulated independently by helical and conical phase distributions followed by their superposition in the same physical path for free space optical communications. Both phase distributions generate annular intensities with specific geometry according to the modulating parameters. The modes are multiplexed in an emission unit and propagate via laboratory free space as rings with optimized radii, without interfering, to the reading phase mask of the receiving unit. The reading mask acts as matching filter which embeds planar phase information such that to distribute the helical and conical components on mutual perpendicular directions. Identification occurs when the parameters of the reading mask match those of the input modes as predicted by a 2-dim inner product model developed here and confirmed experimentally by the occurrence of maximum intensities in the specific centers. Splitting the matrix-like diffracted beam in enabler and processing branches allows to double check and to identify the matching cases by the properties of the associated histogram without affecting subsequent signal retrieval. Crosstalk levels are examined.