Many population analysis methods are based on the precept that molecules should be built from fragments (typically atoms) that maximally resemble the isolated fragment. The resulting molecular building blocks are intuitive (because they maximally resemble well-understood systems) and transferable (because if two molecular fragments both resemble an isolated fragment, they necessarily resemble each other). Information theory is one way to measure the deviation between molecular fragments and their isolated counterparts, and it is a way that lends itself to interpretation. For example, one can analyze the relative importance of electron transfer and polarization of the fragments. We present key features, advantages, and disadvantages of the information-theoretic approach. We also codify existing information-theoretic partitioning methods in a way, that clarifies the enormous freedom one has within the information-theoretic ansatz.