A series of sulfonamide-conjugated poly-(phenylacetylene)s with various electron withdrawing and donating substituents were designed and synthesized in order to develop a novel colorimetric probe for anions in water. The UV-vis absorption measurements, which were performed in an organic solvent to provide fundamental insights into the anion detection properties of these polymers, clarified that the colorimetric response ability is highly enhanced by the incorporation of strong electron withdrawing groups (EWG). Sulfonamide-conjugated poly(phenylacetylene) with nitro groups, the strongest EWG, was therefore used for the detection of anions in an aqueous environment. In a solution containing up to 20% water, the polymer exhibited obvious color changes to anions including the biologically important carboxylates. On the basis of the NMR titration analysis, such a colorimetric response was confirmed to be based on the deprotonation event of the sulfonamide binding site, which was suggested to be essential for the detection of aqueous anions.