Competition between ferromagnetic and antiferromagnetic phases on frustrated lattices in hexagonal Laves phase compound Hf0.86Ta0.14Fe2 is investigated by using neutron diffraction as a function of temperature and magnetic fields and density-functional-theory calculations. At 325 K, the compound orders into the 120° frustrated antiferromagnetic state with a well-reduced magnetic moment, and an in-plane lattice contraction simultaneously sets in. With further cooling down, however, the accumulated distortion in turn destabilizes this susceptible frustrated structure. The frustration is completely relieved at 255 K when the first-order transition to the ferromagnetic state takes place, where a colossal negative volumetric thermal expansion, -123×10-6/K, is obtained. Meanwhile, the antiferromagnetic state can be suppressed by few-tesla magnetic fields, which results in a colossal positive magnetostriction. Such delicate competition is attributed to the giant magnetic fluctuation inherent in the frustrated antiferromagnetic state. Therefore, the magnetoelastic instability is approached even under a small perturbation.