The trans form of an azobenzene-containing guest can complex with a pillar[6]arene, while it cannot complex with pillar[S]arenes due to the different cavity sizes of the pillar[6]arene and the pillar[S]arenes. The spontaneous aggregation of its host guest complex with the pillar[6]arene can be reversibly photocontrolled by irradiation with UV and visible light, leading to a switch between irregular aggregates and vesicle-like aggregates. This new pillar[6]arene-based photoresponsive host guest recognition motif can work in organic solvents and is a good supplement to the existing widely used cyclodextrin/azobenzene recognition motif.