Welcome to B&B.
I'm a big fan of 40's vintage Gillette Super Speed razors. I was recently at the Quantico PX recently and they had blades for $1.39 a 10 pack. They also had cheap VDH brushes for around $3 and VDH soap as well.
So I guess I would start at your local military exchange and see what you can get for cheap.