You are spending much too much time with the razor on your face which is part of the problem. A DE should never take more than 15 minutes and even that is quite a bit. Watch you blade angle and your pressure. Those are the keys.
Importance of blade angle and pressure =
Importance of completing your shave in 15 minutes or less =