What is your brush and how long have you had it? When I first started wet shaving a few years ago I bought the only brush I knew to start with, a Burma Shave. That guy lost hairs for several months at 3-5 at a time. But then after a few years it stopped and is now as soft as a quality badger. I soak my brush in a sink of hot water along with my bowl/mug for my soap/cream. That heats up the bowl while the brush is soaking and I do this probably 2-3 minutes or just long enough to wet my face and soften my whiskers. I don't touch my bristles on any brush after soaking. I just give it a firm shake straight down to get the excess water out of the knot. Afterwards my routine is soaking the brush in water and using it just as if I was using it to build up a later. I find this is gentle and gives equal rinse to the entire brush. I shake the brush in a downward motion several time until no more water comes out and call my shaving routine done. I don't hang my brushes upside down either.
I do have a brush rotation. I treat my brushes like I do my pipes. One use per week and let it rest. Like any hobby, I don't ask or anticipate anyone to go out about buy 5-7 quality brushes. I started with one for a few years, bought a VDH boar, saved up and got a M&F Blonde Badger from Lee, etc. Brushes and razors are like fleas, once you get one on you, the multiply faster than you can understand.
I have not failed. I've just found 10,000 ways that won't work. - Thomas A. Edison