The water networks with internal water mains have an advantage for easy operation and control. In this paper, based on the understanding of the physical insights of the water networks with internal water mains, a new rule-based design methodology is presented. In this methodology, firstly an original water network is obtained by using a basic mathematical programming model. Then the water sources and water sinks for each internal water main are selected over again. At last following the heuristic rules of water allocation from internal water mains and the heuristic rules for reducing water to internal water mains, the final water network is designed by adjusting and simplifying the original water network. Case studies show that this method is practical and effective for water network design. And to compare with other existing design methods, through using this method it is possible to obtain a simpler network and less freshwater consumption.