The instability of a solution film cast on a solid support to form a membrane can lead to holes or other defects periodically on the membrane surface and these defects may reduce membrane performances. It was found that there are two reasons which can cause the film instability. One is very steep concentration gradient near the liquid-gas interface, another is the molecular interaction between the film and the support. Stability analysis for the film was carried out. An analytical expression for the growth coefficient of the surface disturbance was obtained. Membrane morphology related to the film instability was simulated with a computer. The results show that besides the molecular interactions and concentration gradients, the solution viscosity, surface tension, film thickness and support structure are important factors to the stability of the film, and the film instability can lead to convexities, concavities and holes on the membrane surface. An optimal membrane thickness was predicted, which is valuable for preparing ultrathin composite membranes.