In this article, we report the structural phase transitions in Bi4V2O11 as observed from temperature-dependent Raman scattering and X-ray diffraction measurements. Four different types of highly disordered coordination polyhedra around the vanadium atoms with large dispersion of V-O bond lengths are observed in Bi4V2O11 at ambient temperature. The observed V-O bond lengths could be grouped into two categories, viz. shorter <1.7 angstrom and longer >1.7 angstrom. The Raman modes of Bi4V2O11 could be assigned to vibration of these bonds and V-O-V linkages. We could correlate the difference in degree of anharmonicity of the phonon modes with temperature to differences in V-O bond strength. The local structure of vanadium-oxygen network in Bi4V1.8Cu0.2O10.7 was also obtained by similar studies. The effect of highly disordered anion sublattice in the doped compound is reflected in the broadening of the Raman modes.