The interaction of Ag with the Si(100)-2X1:H surface has been studied using a scanning tunneling microscope (STM). By comparing surfaces with various dangling bond defect densities, it is shown that such sites nucleate the growth of Ag islands. The critical Ag coverage above which coalescence of islands occurs, together with the mean height and width of islands are all determined by the initial dangling bond density on the Si(100)-2X1:H surface. We have also observed that the surface may be modified by scanning with a STM resulting in the transfer of Ag from the sample to the tip.