The rheology of fat crystal networks under linear shear deformations has been extensively studied due to its role in material functionality and sensory perceptions. In contrast, there has been limited focus on their viscoelastic response under large shear deformations imposed during processing and product use. We probed the nonlinear viscoelastic behavior of fats displaying mechanics akin to ductile and brittle solids using large amplitude oscillatory shear (LAOS). Using the FT-Chebyshev stress decomposition method, and local measures of nonlinear viscoelasticity, we obtained rheological properties relevant to bulk behavior. We found that ductile fats dissipate more viscous energy than brittle fats and show increased plastic deformation. Structural characterization revealed the presence of three hierarchy levels and layered microstructures in ductile fats in contrast to only two hierarchies and random microstructures in brittle fats. We suggest that these structural features account for increased hypothesize dissipation, which contributes to their ductile-like macroscopic behavior.