Fading memory is a distinguishing characteristic of viscoelastic solids. Its assessment is often achieved by measuring the stress due to harmonic strain histories at different frequencies: from the experimental point of view, the storage and loss moduli are, hence, introduced. On the other side, the mathematical modeling of viscoelastic materials is usually based on the consideration of a kernel function whose decay rate is sufficiently fast. For several different solid materials, we have collated experimental evidence showing an high sensitivity to frequency variations of both the storage and loss moduli. By contrast, we prove that the commonly employed viscoelastic kernels (Prony series, continuous kernel, etc.) cannot reproduce this experimental behavior, as the resulting frequency sensitivity of the storage modulus is always zero when assessed at low frequency. This leads to identification problems of the material parameters which are strongly ill conditioned. However, we identify the specific kernel property which is responsible for this misbehavior: the long-term material memory must not decrease too fast. Some viscoelastic kernels, showing the correct memory's rate of decay, are introduced and their improved ability to match the experimental data analyzed.