Two-dimensional (2D) Ruddlesden-Popper perovskites with multiple quantum well structure have attracted extensive attention due to its superior ambient stability and rapid rise in efficiency for perovskite solar cells. For 2D Ruddlesden-Popper perovskites (BA(2)(MA)(n-1)PbnI(3n+1)), the value of n determines the thickness of perovskite layers within each quantum well. For perovskite with low n phases, the device performance is hindered by formation of severe charge transfer barrier, acting as trap centers. Thus, it is favorable to prepare high performance 2D Ruddlesden-Popper perovskite films with suppressed low n phases. Herein, in this work, the co-solvent engineering and thermal engineering strategy were investigated with optimized solvent ratios (DMF to DMSO) and substrate preheating temperature to suppress low n phases in 2D Ruddlesden-Popper perovskite films. The results showed that with optimized co-solvent ratio of 1:3, the low n phases are considerably suppressed with largely increased perovskite grain size. This was characterized by the steady-state photoluminescence measurement excited from the perovskite film back side, achieving 8.87% photoelectric conversion efficiency. By further optimizing the substrate preheating temperature, the formation of low n phases was further suppressed as convinced by the photoluminescence spectra. With the combined approaches, the solar cell device performance was synergistically boosted up to 11.6% with negligible hysteresis and superior stability. This work reveals that the co-solvent engineering and thermal engineering strategy is a valuable approach for control the low n phase in 2D Ruddlesden-Popper perovskites, which may bring broad interest for pursuing high performance 2D perovskite solar cells.