Polymers containing in the chain backbone carbazoles substituted with two acceptor groups as nonlinear optical chromophores have been synthesized from 3,6-diformyl-9-heptylcarbazole and bis(cyanoacetate) by the Knoevenagel polycondensation. A one-stage polycondensation was carried out in tetrahydrofuran (THF) solution to yield a carbazole main-chain polymer with low molecular weight in low yield. A two-stage polycondensation first carried out in THF and followed by a solid-state polycondensation yielded a carbazole main-chain polymer with high molecular weight in high yield. Thin films of polymers with good optical quality could be obtained from their chloroform solutions by spin-coating. After electric poling, the films exhibited reasonable second-order nonlinear optical responses and the large off-diagonal tensor components.