We provide Lyapunov-like characterizations of boundedness and convergence of non-trivial solutions for a class of systems with unstable invariant sets. Examples of systems to which the results may apply include interconnections of stable subsystems with one-dimensional unstable dynamics or critically stable dynamics. Systems of this type arise in problems of nonlinear output regulation, parameter estimation, and adaptive control. In addition to providing boundedness and convergence criteria, the results allow us to derive domains of initial conditions corresponding to solutions leaving a given neighborhood of the origin at least once. In contrast to other works addressing convergence issues in unstable systems, our results require neither input-output characterizations for the stable part nor estimates of convergence rates. The results are illustrated with examples, including the analysis of phase synchronization of neural oscillators with heterogeneous coupling.