Novel Magnetic Phases of Quantum Matter from Geometric, Spin-Space and Chemical Frustration