Three novel rubidium borogermanates with three types of noncentrosymmetric structures, namely, RbGeB3O7, Rb2GeB4O9, and Rb4Ge3B6O17, have been synthesized by high-temperature solid-state reactions in platinum crucibles. The structure of RbGeB3O7 features a three-dimensional (3D) anionic framework composed of cyclic B3O7 groups corner-sharing GeO4 tetrahedra. The structure of Rb2GeB4O9 shows a 3D anionic framework based on B4O9 clusters connected by GeO4 tetrahedra via corner sharing. The structure of Rb4Ge3B6O17 is a novel 3D anionic framework composed of cyclic B3O8 groups, Ge2O7 dimers, and GeO4 tetrahedra that are interconnected via corner sharing. Second harmonic generation (SHG) measurements indicate that RbGeB3O7, Rb2GeB4O9, and Rb4Ge2B6O17 display moderate SHG responses that are approximately 1.3, 2.0, and 1.3 x KH2PO4 (KDP), respectively, which are slightly smaller than those from theoretical calculations (about 3.7, 2.8, and 2.4 x KDP, respectively).