In recent years, the exploration and development of shale oil and gas reservoirs has become a hot topic in various countries around the world. The mechanical properties of shale are an important basis for evaluating the mining effect. Previous studies have shown that shale swells with water, which greatly effects its mechanical properties. Based on the analysis of shale composition, a clay mineral swelling test and 3D compression test were carried out to analyze the influence of clay minerals on the mechanism of shale swelling and softening in water. The results show that hydration of clay minerals results in interlayer and intergranular swelling. Furthermore, an expansion force will be exerted on the rock skeleton. The clay minerals expansion will cause a decrease in the uniaxial compressive strength, decrease in the elastic modulus, and increase in the Poisson's ratio, both in normal and parallel directions to the shale plate bedding. The brittleness index calculated based on the elastic modulus and Poisson's ratio values also decreases. This work is significant for further engineering development of shale reservoirs.