We have investigated the impact of out-of-plane ferromagnetic (FM) anisotropy (which can be coincident with the direction of unidirectional anisotropy), where antiferromagnetic (AF) anisotropy is along the film plane. This provides a platform for non-collinear exchange coupling in an archetypal exchange coupled system in an unconventional way. We probe the in-plane magnetization by the depth-sensitive vector magnetometry technique. The experimental findings reveal a magnetization reversal (i) that is symmetric for both the branches of the hysteresis loop, (ii) that is characterized by vertically correlated domains associated with a strong transverse component of magnetization and (iii) that remains untrained (suppression of trained state) with field cycling. This scenario has been compared with in-plane magnetization reversal for a conventional in-plane unidirectional anisotropic case in the same system that shows usual asymmetric reversal and training for vertically uncorrelated domains. We explain the above observations for the out-of-plane case in terms of inhomogeneous magnetic states due to competing perpendicular anisotropies that result in non-collinear FM-AF coupling. This study provides direct evidence for the vertical correlation of domains mediated by out-of-plane exchange coupling.