Boiling water extracts of 66 selected Chinese medicinal herbs were screened for their anticyanobaterial activity against Microcystis aeruginosa by the soft-agar overlayer (SAO) method. Results indicated that extracts from 16 materials could inhibit the growth of this bacterial species. Among these anticyanobacterial samples, eight extracts showed low minimum inhibitory concentrations (MIC), including four extracts with MICs between 1 and 6 mg/mL, and four extracts with MICs < 1 mg/mL which could be considered useful to prevent the outbreak of cyanobacteria before the appearance of cyanobacterial blooms. Further study showed that three extracts with MIC values < 1 mg/mL induced intensive chlorophyll-a lysis within 7 days at the MIC. The results suggested that highly efficient anticyanobacterial compounds must be involved in the inhibitory activities. The final results indicated these three extracts (from Malaphis chinensis, Cynips gallae-tinctoriae and Fructus mume) had the potential to be developed as algicides due to their remarkably anticyanobacterial activities.