For oil wells with high sand content, traditional oil well pumps easily get stuck. Increasing the gap between the plunger and the pump cylinder can effectively prevent the sticking, but the gap leakage also increases, resulting in a decrease in the volumetric efficiency of the pump. As shown in this paper, when a rectangular groove is formed in the plunger of the oil well pump, it can significantly reduce the leak rate. In order to reveal the flow resistance mechanism of the gap seal structure with rectangular groove and the reasonable design of the seal structure parameters, the flow characteristics in the gap seal structure with rectangular groove are studied by CFD technology. The results show that the gap seal structure with rectangular groove reduces the leakage amount by 87% compared with the prototype gap seal structure. The optimal number of rectangular grooves is 3-4. The sealing effect of the rectangular groove at the inlet of the pump is better than that at the pump outlet. The rectangular groove spacing has little effect on the leak rate.