This paper investigates the near optimal control for a kind of linear stochastic control systems governed by the forward-backward stochastic differential equations, where both the drift and diffusion terms are allowed to depend on controls and the control domain is not assumed to be convex. In the previous work (Theorem 3.1) of the second and third authors, some problem of near optimal control with the control dependent diffusion is addressed and our current paper can be viewed as some direct response to it. The necessary condition of the near-optimality is established within the framework of optimality variational principle developed by Yong and obtained by the convergence technique to treat the optimal control of FBSDEs in unbounded control domains by Wu. Some new estimates are given here to handle the near optimality. In addition, an illustrating example is discussed as well.