In this work, photoinduced orientation of three epoxy-based polymers bearing different type azo chromophores was studied. The epoxy-based azo polymers were synthesized through post-polymerization azo-coupling reactions based on an epoxy-based precursor polymer (BP-AN), which was synthesized by the step polymerization between bisphenol-A diglycidyl ether and aniline. The chromophore orientation and cooperative motion of the main-chain groups were studied by birefringence characterization, polarized infrared spectroscopy, and two-dimensional (2D)-IR correlation spectroscopy (COS). The results show that the orientation behavior of the epoxy-based azo polymers is closely related with the electron-withdrawing groups on the azo chromophores. The azo polymer BP-AZ-CN, which contains azo chromophores with cyano as the electron-withdrawing group, shows the fastest birefringence growth rate. The azo polymer BP-AZ-CA, containing carboxyl as the electron-withdrawing group, possesses high birefringence residual value and the highest saturated orientation degree in the series. Cooperative motion between azobenzene moieties and non-photochromic polymer backbone was revealed by polarized FTIR and 2D-IR COS. The photoinduced anisotropy is a result of the orientation of both azo chromophore and polymer main-chain. The understanding of the structure-property relationships can be used to develop materials with better performance for data storage and other applications.