Spin Frustration and Unconventional Surface Spin Canting State in Van der Waals Ferromagnet/Antiferromagnet Heterostructures
Atomically flat surfaces of van der Waals (vdW) materials pave an avenue for addressing a long-standing fundamental issue of how a compensated antiferromagnet (AFM) surface frustrates a ferromagnetic (FM) overlayer in FM/AFM heterostructures. We investigate Fe5GeTe2/NiPS3 vdW heterostructures by characterizing AFM and FM spins separately. We find that in-plane zig-zag AFM NiPS3 develops three equivalent AFM domains, which are robust against external magnetic field and magnetic coupling with Fe5GeTe2. Moreover, evidence is provided of in-plane-AFM-induced perpendicular magnetic anisotropy (PMA) in adjacent Fe5GeTe2, and an unconventional out-of-plane surface spin canting state with the Fe5GeTe2 spins spatially turn from out-of-plane direction near the interface to in-plane direction away from the interface in Fe5GeTe2/NiPS3. The out-of-plane surface spin canting is a unique property of spin frustration in vdW magnetic heterostructures.