This work presents a hybrid auxiliary density functional theory (ADFT) study of the neutral and hexaanionic C-104 and C-106 fullerenes with the aim to determine their ground state structures. To this end, all C-104 and C-106 fullerene structures that obey the isolated pentagon rule (IPR) were optimized with the Perdew-Burke-Ernzerhof generalized gradient approximation followed by a single-point energy calculation with the PBEO hybrid functional. Our studies show that this composite approach yields relative energies of giant fullerenes that are accurate to around 1 kcal/mol. As a result, the ground states of C-104, C-104(6-), and C-106(6-) can be assigned to the isomers 234:C-s, 821:D-2, and 891:C-s, respectively. On the other hand, the energetically lowest lying IPR isomers of C-106, 331:C-s, 1194:C-2, 534:C-1 are separated by less than 1 kcal/mol which makes an unequivocal ground state assignment by hybrid DFT methods impossible. To guide future experiments, we also report the simulated IR and Raman spectra of the most stable neutral and hexaanionic C-104 and C-106 fullerenes.