A new synthetic strategy was developed to prepare polyphosphazenes with second-order nonlinear optical chromophore in which sulfonyl groups are the acceptors. Thus, polyphosphazenes P1 and P2 that contain aniline or indole groups as side chains were obtained from a highly reactive macromolecular intermediate, poly(dichlorophosphazene), by nucleophilic substitution reaction. Then a post-azo coupling of p-ethylsulfonylbenzenediazonium fluoroborate or p-octylsulfonylbenzenediazonium fluoroborate toward the aniline or indole ring in P1 and P2 afforded the sulfonyl-based chromophore-functionalized polyphosphazenes P3, P4, and P5. The polymers exhibit good solubility in common organic solvents and are thermally stable. The maximum absorption appeared at about 440 nm in P3, while that of P4 was at about 393 nm, which were blue-shifted about 52 and 32 nm, respectively, compared to the corresponding chromophores with nitro acceptor and resulted in a wider transparency window. The poled films of P3 and P4 exhibits a resonant d(33) value of 27 and 18 pm/V, respectively, by second harmonic generation (SHG) measurements.