Frustration, or the competition between interacting components of a network,
is often responsible for the complexity of many body systems, from social and
neural networks to protein folding and magnetism. In quantum magnetic systems,
frustration arises naturally from competing spin-spin interactions given by the
geometry of the spin lattice or by the presence of long-range antiferromagnetic
couplings. Frustrated magnetism is a hallmark of poorly understood systems such
as quantum spin liquids, spin glasses and spin ices, whose ground states are
massively degenerate and can carry high degrees of quantum entanglement. The
controlled study of frustrated magnetism in materials is hampered by short
dynamical time scales and the presence of impurities, while numerical modeling
is generally intractable when dealing with dynamics beyond N~30 particles.
Alternatively, a quantum simulator can be exploited to directly engineer
prescribed frustrated interactions between controlled quantum systems, and
several small-scale experiments have moved in this direction. In this article,
we perform a quantum simulation of a long-range antiferromagnetic quantum Ising
model with a transverse field, on a crystal of up to N = 16 trapped Yb+ atoms.
We directly control the amount of frustration by continuously tuning the range
of interaction and directly measure spin correlation functions and their
dynamics through spatially-resolved spin detection. We find a pronounced
dependence of the magnetic order on the amount of frustration, and extract
signatures of quantum coherence in the resulting phases.