Failure caused by dendrite growth in high-energy-density, rechargeable batteries with lithium metal anodes has prevented their widespread use in applications ranging from consumer electronics to electric vehicles. Efforts to solve the lithium dendrite problem have focused on preventing the growth of protrusions from the anode surface. Synchrotron hard X-ray microtomography experiments on symmetric lithium-polymer-lithium cells cycled at 90 degrees C show that during the early stage of dendrite development, the bulk of the dendritic structure lies within the electrode, underneath the polymer/electrode interface. Furthermore, we observed crystalline impurities, present in the uncycled lithium anodes, at the base of the subsurface dendritic structures. The portion of the dendrite protruding into the electrolyte increases on cycling until it spans the electrolyte thickness, causing a short circuit. Contrary to conventional wisdom, it seems that preventing dendrite formation in polymer electrolytes depends on inhibiting the formation of subsurface structures in the lithium electrode.