Crystal structure prediction has been a grand challenge in material science owing to the large configurational space that one must explore. Evolutionary (genetic) algorithms coupled with first principles calculations are commonly used in crystal structure prediction to sample the ground and metastable states of materials based on configurational energies. However, crystal structure predictions at finite temperature (T), pressure (P), and composition (X) require a free-energy-based search that is often computationally expensive and tedious. Here, we introduce a new machine-learning workflow for structure prediction that is based on a concept inspired by the evolution of human tribes in primitive society. Our tribal genetic algorithm (GA) combines configurational sampling with evolutionary optimization to accurately predict entropically stabilized phases at finite (T, P, X), at a computational cost that is an order of magnitude smaller than that required for a free-energy-based search. In a departure from standard GA techniques, the populations of individuals are divided into multiple tribes based on a bond-order fingerprint, and genetic operations are modified to ensure that cluster configurations are sampled adequately to capture entropic contributions. Team competition introduced into the evolutionary process allows winning teams (representing a better set of individuals) to expand their sizes; this translates into a more expanded search of the phase space allowing us to explore solutions near possible global minimum. Each team explores a specific section of the structural phase space and avoids bias on solutions arising from the use of individual populations in a purely energy-based search. We demonstrate the efficacy of our approach by performing the structural prediction of a representative two-dimensional two-body system as well as Lennard-Jones clusters over a range of temperatures up to its melting point. Our approach outperforms the standard GA approaches and enables structural search under "real nonambient conditions" on both bulk systems and finite-sized clusters.