We investigated the structural, vibrational, magnetic, and electronic properties of the recently synthesized CaCo3V4O12 double perovskite with the high-spin (HS) Co2+ ions in a square-planar oxygen coordination at extreme conditions of high pressures and low temperatures. The single-crystal X-ray diffraction and Raman spectroscopy studies up to 60 GPa showed a conservation of its cubic crystal structure but indicated a crossover near 30 GPa. Above 30 GPa, we observed both an abnormally high "compressibility" of the Co-O bonds in the square planar oxygen coordination and a huge anisotropic displacement of HS-Co2+ ions in the direction perpendicular to the oxygen planes. Although this effect is reminiscent of a continuous HS -> LS transformation of the Co2+ ions, it did not result in the anticipated shrinkage of the cell volume because of a certain "stiffing" of the bonds of the Ca and V cations. We verified that the oxidation states of all the cations did not change across this crossover, and hence, no charge-transfer effects were involved. Consequently, we proposed that CaCo(3)V(4)o(12) could undergo a phase transition at which the large HS-Co2+ ions were pushed out of the oxygen planes because of lattice compression. The antiferromagnetic transition in CaCo3V4O12 at 100 K was investigated by neutron powder diffraction at ambient pressure. We established that the magnetic moments of the Co2+ ions were aligned along one of the cubic axes, and the magnetic structure had a 2-fold periodicity along this axis, compared to the crystallographic one.