The association between microbial communities and plant growth in long-term fertilization system has not been fully studied. In the present study, impacts of long-term fertilization have been determined on the size and activity of soil microbial communities and wheat performance in a red soil (Ultisol) collected from Qiyang Experimental Station, China. For this, different microbial communities originating from long-term fertilized pig manure (M), mineral fertilizer (NPK), pig manure plus mineral fertilizer (MNPK), and no fertilizer (CK) were used as inocula for the Ultisol tested. Changes in total bacterial and fungal community composition and structures using Ion Torrent sequencing were determined. The results show that the biomass of wheat was significantly higher in both sterilized soil inoculated with NPK (SNPK) and sterilized soil inoculated with MNPK (SMNPK) treatments than in other treatments (P < 0.05). The activities of beta-1,4-N-acetylglucosaminidase (NAG) and cellobiohydrolase (CBH) were significantly correlated with wheat biomass. Among the microbial communities, the largest Ascomycota phylum in soils was negatively correlated with beta-1,4-glucosidase (beta G) (P < 0.05). The phylum Basidiomycota was negatively correlated with plant biomass (PB) and tillers per plant (TI) (P < 0.05). Nonmetric multidimensional scaling analysis shows that fungal community was strongly correlated with long-term fertilization strategy, while the bacterial community was strongly correlated with beta-1,4-N-acetylglucosaminidase activity. According to the Mantel test, the growth of wheat was affected by fungal community. Taken together, microbial composition and diversity in soils could be a good player in predicting soil fertility and consequently plant growth.