Halogen bonding is arguably the least exploited among the many non-covalent interactions used in dictating molecular self-assembly. However, its directionality renders it unique compared to ubiquitous hydrogen bonding. Here, the role of this directionality in controlling the performance of light-responsive supramolecular polymers is highlighted. In particular, it is shown that light-induced surface patterning, a unique phenomenon occurring in azobenzene-containing polymers, is more efficient in halogen-bonded polymerazobenzene complexes than in the analogous hydrogen-bonded complexes. A systematic study is performed on a series of azo dyes containing different halogen or hydrogen bonding donor moieties, complexed to poly(4-vinylpyridine) backbone. Through single-atom substitution of the bond-donor, control of both the strength and the nature of the noncovalent interaction between the azobenzene units and the polymer backbone is achieved. Importantly, such substitution does not significantly alter the electronic properties of the azobenzene units, hence providing us with unique tools in studying the structureperformance relationships in the light-induced surface deformation process. The results represent the first demonstration of light-responsive halogen-bonded polymer systems and also highlight the remarkable potential of halogen bonding in fundamental studies of photoresponsive azobenzene-containing polymers.