We investigated experimentally and theoretically the interactions and assembly of rodlike colloids in a ferrofluid confined at solid/liquid interface by the gravity under external magnetic fields. We first derived analytical expressions for the interaction energy of a single rod with the external magnetic field and the interaction between two rods using classical electromagnetism. The theory well captured the experimentally observed alignment of a single rod along the field direction under an in-plane field and switching between the horizontal and the vertical configurations in an out-of-plane field due to the competition between the magnetic energy and the gravitational energy. The theory can also predict the symmetric position fluctuations of a free rod on a fixed one at 90 degrees and the gradual bias toward the end of the fixed rod as the angle was reduced to 0 degrees, favoring the tip-toe arrangement. Finally, we showed that this anisotropic interaction led to the formation of chain-like structures, whose growth kinetics followed a simple scaling behavior with time. This work provides a theoretical framework for understanding the behaviors of rodlike colloids in ferrofluids and highlights the importance of shape anisotropy in manipulating colloids and their self-assembly.