Microbial fermentation in the hindgut is likely an important contributor to energy availability in ruminants, except for the rumen. This study aimed to investigate commensal bacteria in the colon influenced by diverse dietary niches. Fifteen male sheep were randomly allotted into three feeding groups: non-pelleted low-grain (CON, n = 5), non-pelleted high-grain (HG, n = 5), and pelleted high-grain (HP, n = 5) diets. The HG and HP groups had higher fermentation parameters than the CON group, especially acetate concentration (CON = 46.91; HG = 61.66; HP = 77.99). The HG diet altered the composition of commensal bacteria in the colon in comparison to the CON group, including the increase of genera related to acetate production (e.g., Acetitomaculum spp.), butyrate production (e.g., Coprococcus spp. and Subdoligranulum spp.), and starch degradation (e.g., Prevotella spp., Roseburia spp., and Oscillibacter spp.). The colon functional compendium had co-alteration with taxonomic changes that indicated non-pelleted HG diet caused a detrimental colonic niche. The HP diet specifically promoted the abundance of Ruminococcus, Olsenella, and Alloprevotella genera to achieve the highest acetate concentration and decreased the starch-degrader Roseburia spp. and Oscillibacter spp. in contrast to the HG group. Our results provide a systematic view of the microbial fermentation, community, and functional guilds in colonic digesta and mucosa in regard to using an HP diet to maintain colonic niche homeostasis under the adverse influence of the HG diet. Key Points Non-pelleted and pelleted high-grain diets altered sheep colonic fermentation. Non-pelleted and pelleted high-grain diets resulted in diverse microbial composition. The pelleted method ameliorated microbial functions compared with the high-grain diet.