The aim of this paper is to solve the Hamilton-Jacobi-Bellman (HJB) quasi-variational inequalities arising in regime switching utility maximization with optimal stopping. The HJB quasi-variational inequalities are penalized into the HJB equations and the convergence of the viscosity solution of the penalized HJB equations to that of the HJB variational inequalities is proved. The finite difference methods with iteration policy are used to solve the penalized HJB equations and the convergence is proved. The approach is implemented via numerical examples and the figures for the exercise boundaries and optimal strategies with sample paths are sketched.