In this paper, we consider the adaptive attitude control of spacecraft with time-varying inertial properties. We use a coordinate independent approach for the purpose of designing the control and estimation laws in terms of the rotation matrices representing the spacecraft body frame and reference tracking signals. This method helps to overcome the difficulties regarding the attitude representation on SO(3) such as ambiguities associated with quaternion representation and inherent singularities inside Euler parameters. We model the time variations in inertial parameters in two different ways, and design adaptive control schemes for each case. As the first uncertain dynamic model, we consider a setting of spacecraft with multiple moving appendages, and based on this model, design an adaptive control scheme with three different versions, where Frobenious norm is used in measuring the deviation of the estimated inertia tensors from their actual values. The proposed adaptive control scheme is later extended for the more direct model where the inertia tensor of the spacecraft has a nonlinear relation with the norm of the input moment. Further, we derive the allowable sets of initial conditions to ensure the convergence of the tracking error. Simulation results are provided to illustrate the effectiveness of our proposed approach.