In human bone, an amorphous mineral serves as a precursor to the formation of a highly substituted nanocrystalline apatite. However, the precise role of this amorphous mineral remains unknown. Here, we show by using transmission electron microscopy that 100-300nm amorphous calcium phosphate regions are present in the disordered phase of trabecular bone. Nanomechanical experiments on cylindrical samples, with diameters between 250nm and 3,000 nm, of the bone's ordered and disordered phases revealed a transition from plastic deformation to brittle failure and at least a factor-of-2 higher strength in the smaller samples. We postulate that this transition in failure mechanism is caused by the suppression of extrafibrillar shearing in the smaller samples, and that the emergent smaller-is-stronger size effect is related to the sample-size scaling of the distribution of flaws. Our findings should help in the understanding of the multi-scale nature of bone and provide insights into the biomineralization process.