It's your choice, but with sensitive skin you will likely find rinsing the alum after 1 or 2 minutes will give you the best results. If you leave it on, it may cause your face to be a bit on the "tight" side. I leave it on sometimes if I'm applying a moisturizer, but always rinse if I'm going with an aftershave.
There is no harm to the skin either way.
(as a side note, pores don't open or close)
Brian - I think it would be totally inappropriate for me to even contemplate what I am thinking about.