Theory of analytic first-order properties is formulated in a basis set independent way using the first-quantized many-body perturbation theory. This formulation allows the correlation effects to be described with explicitly correlated basis sets. The basis of Gaussian geminals is employed to calculate the second-and third-order correlation corrections to the lowest multipole moments of the H-2 and LiH molecules. The same formalism is also utilized to compute the intra-atomic correlation contribution to the first-order interaction energy for the helium dimer. The results compare favorably with the literature data obtained using the conventional, orbital basis approach.