Polycrystalline BiFeO3 (BFO) and Bi0.90Gd0.10Fe1-xZrxO3 (x=0.0-0.10; BGFZ(x)) ceramics were synthesized by solid-state reaction method. Rietveld analysis of X-ray diffraction patterns showed that BFO and BGFZ(x=0.0) samples are stabilized in rhombohedral structure (space group R3c), whereas a small fraction of orthorhombic phase (space group Pn2(1)a) is observed for BGFZ(x=0.03-0.10) samples. Suppression and disappearance of some Raman modes indicated a structural phase transition with addition of Zr dopant at Fe site. Magnetic measurements exhibited weak ferromagnetic behavior of BGFZ(x) samples with increasing Zr+4 concentrations. The insertion of Gd+3 ions at Bi+3 sites and nonmagnetic Zr+4 ions at Fe+3 sites in Fe-O-Fe network suppressed the spin cycloid structure of BFO which in turn enhanced the magnetization of these ceramics. Electron spin resonance spectra revealed the breaking of spin cycloid of BFO due to the development of free spins with addition of Zr+4 dopants at Fe sites. UV-Visible diffuse reflectance spectra showed one d-d crystal field transition and two charge-transfer (C-T) transitions along with a sharp absorption of light in visible region for all samples. Almost frequency-independent dielectric constant and dielectric loss along with very low values of dielectric loss indicated greatly improved dielectric properties for BGFZ(x=0.03-0.10) samples.