In this paper, we present a new fourth-order upwinding embedded boundary method (UEBM) over Cartesian grids, originally proposed in the Journal of Computational Physics [190 (2003), pp. 159-183.] as a second-order method for treating material interfaces for Maxwell's equations. In addition to the idea of the UEBM to evolve solutions at interfaces, we utilize the ghost fluid method to construct finite difference approximation of spatial derivatives at Cartesian grid points near the material interfaces. As a result, Runge-Kutta type time discretization can be used for the semidiscretized system to yield an overall fourth-order method, in contrast to the original second-order UEBM based on a Lax-Wendroff type difference. The final scheme allows time step sizes independent of the interface locations. Numerical examples are given to demonstrate the fourth-order accuracy as well as the stability of the method. We tested the scheme for several wave problems with various material interface locations, including electromagnetic scattering of a plane wave incident on a planar boundary and a two-dimensional electromagnetic application with an interface parallel to the y-axis.