Variable-temperature Raman spectroscopic and synchrotron X-ray diffraction studies were performed on BaTe2O6 (orthorhombic, space group: Cmcm), a mixed-valence tellurium compound with a layered structure, to understand structural stability and anharmonicity of phonons. The structural and vibrational studies indicate no phase transition in it over a wider range of temperature (20 to 853 K). The structure shows anisotropic expansion with coefficients of thermal expansion in the order alpha(b) >> alpha(a) > alpha, which was attributed to the anisotropy in bonding and structure of BaTe2O6. Temperature evolution of Raman modes of BaTe2O6 indicated a smooth decreasing trend in mode frequencies with increasing temperature, while the full width at half-maximum (fwhm) of all modes systematically increases due to a rise in phonon scattering processes. With the use of our earlier reported isothermal mode Gruneisen parameters, thermal properties such as thermal expansion coefficient and molar specific heat are calculated. The pure anharmonic (explicit) and quasiharmonic (implicit) contribution to the total anharmonicity is delineated and compared. The temperature dependence of phonon mode frequencies and their fwhm values are analyzed by anharmonicity models, and the dominating anharmonic phonon scattering mechanism is concluded in BaTe2O6. In addition to the lattice modes, several external modes of TeOn (n = 5, 6) are found to be strongly anharmonic. The ab initio electronic structure calculations indicated BaTe2O6 is a direct band gap semiconductor with gap energy of similar to 2.1 eV. Oxygen orbitals, namely, O-2p states in the valence band maximum and the sphybridized states in the conduction band minimum, are mainly involved in the electronic transitions. In addition a number of electronic transitions are predicted by the electronic structure calculations. Experimental photoluminescence results are adequately explained by the ab initio calculations. Further details of the structural and vibrational properties are explained in the manuscript.