Different labels have different interpretations of a given "size", so I'd suggest shopping around until you find one that you like the fit of. I'd also suggest that clothes that are a little too big can be altered to a better fit, so take advantage of that were needed.
I'd suggest staying away from wearing clothes that are on the baggy/long side as it'd emphasise that you are too small for those clothes and make you seem even smaller (if that makes sense ... you don't want to look like a kid trying on his Dad's too-big clothes.)
Most clothes that are made for the mass market in North America, when it comes to age-appropriate for mid-40s, will be expecting an out-of-shape (ie tubby) customer. so look for slim-fit clothes in age-appropriate labels. Also, sometimes more youth-oriented labels have a few items that will work for the older crowd, so don't discount them totally.
Be there or be square. Only I can do both!
"My lord, I will use them according to their desert."
"God's bodykins, man, much better: use every man
after his desert, and who should 'scape whipping?"
Veteran of the Great Irisch Moos Campaign of 2008-09