We report a method for the structure-based calculation of the spectral density of the pigment-protein coupling in light-harvesting complexes that combines normal-mode analysis with the charge density coupling (CDC) and transition charge from electrostatic potential (TrEsp) methods for the computation of site energies and excitonic couplings, respectively. The method is applied to the Fenna-Matthews-Olson (FMO) protein in order to investigate the influence of the different parts of the spectral density as well as correlations among these contributions on the energy transfer dynamics and on the temperature-dependent decay of coherences. The fluctuations and correlations in excitonic couplings as well as the correlations between coupling and site energy fluctuations are found to be 1 order of magnitude smaller in amplitude than the site energy fluctuations. Despite considerable amplitudes of that part of the spectral density which contains correlations in site energy fluctuations, the effect of these correlations on the exciton population dynamics and dephasing of coherences is negligible. The inhomogeneous charge distribution of the protein, which causes variations in local pigment-protein coupling constants of the normal modes, is responsible for this effect. It is seen thereby that the same building principle that is used by nature to create an excitation energy funnel in the FMO protein also allows for efficient dissipation of the excitons' excess energy.