We have generalized the microscopic polymer reference interaction site model (PRISM) theory to study the structure and phase behavior of polymer-tethered spherical nanoparticles in a dense homopolymer melt. In the absence of a polymer matrix, fluids of such hybrid nanoparticles show strong concentration fluctuations indicative of aggregate formation and/or a tendency for microphase separation as the total packing fraction and/or nanoparticle attraction strength increase. In the presence of a polymer matrix, there is a competition between tether-mediated microphase separation and matrix-induced macrophase separation. For a single tether of eight segments on a nanoparticle twice the diameter of a segment, the apparent microphase spinodal curve exhibits both dilution-like and depletion-like features and a nonmonotonic dependence of the spinodal temperature on matrix chain length. As the particle size and tether length are both increased, such that the total space filling volume of the tether continues to equal the nanoparticle volume, the shape of the microphase spinodal curve remains unchanged, but the effect of matrix polymer chain length on the spinodal temperature diminishes. For a larger filler, the tendency for macrophase separation grows with increasing matrix polymer length. As the number of tethers is increased, the microphase spinodal curves become more dilution-like, and the effect of matrix degree of polymerization, particle size, and tether length on the apparent spinodal temperature diminishes.