Polymer materials are widely used in controlled drug release, and the diffusion property of drug molecules in these materials is of great importance. In this work, the diffusion behavior of a model drug (aspirin) in different ratios of poly(lactic acid-co-ethylene glycol) (PLA-PEG) was investigated by molecular dynamics simulations. Two major factors, which influence the diffusion of aspirin in polymer matrix: the wriggling of the polymer chain and the free volume of the polymer matrix, are discussed. The wriggling of the polymer chain mainly controls the diffusion of aspirin molecules. Free volume becomes the secondary effect. For two different polymers having a similar degree of wriggling, the free volume controls the diffusion of the aspirin molecules. Comparing with the diffusion behavior of small gas molecules in polymer matrix, a different mechanism was proposed for the drug molecules. The drug molecules can only diffuse along with the wriggling of the polymer matrix.