A mathematical model (numerical simulation) of gas-oil flow in a multi-cup downhole gas-oil separator is proposed. The model takes into account the Gibbs wave equation to obtain the pump plunger velocity and the population balance model (PBM) for describing bubble aggregation and breakage. The gas-oil flow field distribution and the bubble behavior in the multi-cup gas-oil separator were simulated using computational fluid dynamics (CFD). The numerical simulation results showed that the main role of gas-oil flow distribution in the process of oil separation from the separator cup wall is the viscous effect. Small bubbles in the center tube aggregate into large bubbles, and interaction of the bubbles along the flow and the fluid running down the cup wall forms a clockwise spiral, which accelerates separation of the gas and oil phases in the separator cup. During the upstroke, the oil volume being sucked out by the center tube through the small hole is larger than the oil volume in the separator cup, which causes entry of air into the center tube and failure of the separator cup.