We demonstrate that capillary suspensions can be formed from hydrophilic calcium carbonate particles suspended in a polar continuous media and connected by capillary bridges formed of minute amounts of an immiscible secondary liquid phase. This was achieved in two different polar continuous phases, water and glycerol, and three different oils, oleic acid, isopropyl myristate, and peppermint oil as a secondary liquid phase. The capillary structuring of the suspension was made possible through local in situ hydrophobization of the calcium carbonate particles dispersed in the polar media by adding very small amounts of oleic acid to the secondary liquid phase. We observed a strong increase in the viscosity of the calcium carbonate suspension by several orders of magnitude upon addition of the secondary oil phase compared with the same suspension without secondary liquid phase or without oleic acid. The stability and the rheological properties of the obtained capillary structured materials were studied in relation to the physical properties of the system such as the particle size, interfacial tension between the primary and secondary liquid phases, as well as the particle contact angle at this liquid-liquid interface. We also determined the minimal concentrations of the secondary liquid phase at fixed particle concentration as well as the minimal particle concentration at fixed secondary phase concentration needed to form a capillary suspension. Capillary suspensions formed by this method can find application in structuring pharmaceutical and food formulations as well as a variety of home and personal care products.