My wife and I have had disagreements about downtown ballparks before. She grew up in the Candlestick/Coliseum era, so taking a long drive to see a game was normal for her. I'm a big fan of the Toronto/Bronx/Boston/Chicago model of having a strong vibrant community right outside the gates. Living in Philly for a few years really reinforced how much more fun it is to be able to pre- and post-game with a huge crowd of fans instead of everyone dispersing into a hundred different directions as soon as the game ends.
Yes, there is nothing like the atmosphere you describe. I agree with most of the cities you mentioned. However, that never seemed to the the case in Yankee stadium since it was not in an area of the Bronx in which you want to hang out.