II-VI widegap semiconductors such as ZnO are widely utilized in various electronic and optical devices. To widen the band gap of ZnO, we conducted a systematic investigation of solid solution thin films of Zn1-xMgxO, a group of ternary compounds of the Zn-Mg-O system. We prepared the thin films by radio frequency (RF) magnetron co-sputtering on fused silica substrates at room temperature. The thin films were a single phase of Zn1-xMgxO having the basic structure of ZnO at x less than or equal to 0.46 and the basic structure of MgO at x greater than or equal to 0.62, with segregation of the ZnO and MgxO phases at x = 0,58. The band gap of Zn1-xMgxO having the basic structure of ZnO increased from 3.24 eV at x = 0 (ZnO) to 4.20 eV at x = 0.46. Transmittances of Zn1-xMgxO thin films were nearly equivalent to those of ZnO. Zn1-xMgxO has a wider band gap than ZnO and can be expected to provide a useful window layer of solar cells that improves the overall efficiency by decreasing the absorption loss.