The hot topic in cancer therapy, human epidermal growth factor receptor 2 protein (HER2), have been indicated that the overexpression of HER2 associated with ovarian, esophageal, gastric and breast cancers recently. To investigate the novel tyrosine kinase inhibitor of HER2 receptor, we employed the molecular simulation and structure-based drug design. Compounds from our laboratory database and two control kinase inhibitors, Tarceva and Iressa, were employed to dock into the HER2 ATP binding site. Three candidates with the highest dockscore were used for De Novo Evolution and 24 virtual compounds were generated. We found that the key residue, Phe731, would construct the pi-stacking interaction with both Tarceva and Iressa was conclusively regarded as the key residue for the inhibitory efficiency. The addition of 1,5-dimethyl-1H-imidazole group on compound A, produced the extra hydrophobic interaction, and the pi-stacking interaction with Phe731 was generated by the same aromatic group on both compound A and A,. Although compound 131 and C, could not produce the pi-stacking interaction with Phe731, both compounds which possessed the lower binding energy and higher dockscore than the two control compounds were believed to compete with the ATP molecule. Compound A,, 131 and C, were suggested the potent ATP-competitive inhibitors through the significant scores, binding affinity, hydrogen bond linkages and pi-stacking interaction, and these scaffolds were proposed for further drug design. (c) 2008 Taiwan Institute of Chemical Engineers. Published by Elsevier B.V. All rights reserved.