The downside of NYC (from my experience as one who is pretty much on the bottom of the totem pole anyway):
The biggest problem with NYC is that it's so saturated with talent that musicians end up taking less and less $$$ for gigs, to the point that many even play for free. Same for teaching private lessons, if you look on Craigslist for private guitar teachers, some are advertising their services at ridiculously low rates. That, coupled with the always increasing costs of living, can make things difficult. Venue owners probably feel those high costs too, so many might not even have the cash to compensate musicians at a decent rate. I'm lucky enough to have landed a decent day job so I don't need to worry so much, but many of the guys I play with are full-timers, and are struggling to make ends meet.
The upside: You can play with a lot of great players, and see world-class music on a nightly basis. Getting my ass kicked has made me grow a lot as a musician, more than I would have if I'd stayed in the rural area I'm from (even though I made a lot more money there). Most musicians I've met here are pretty cool too, and I haven't run into much of that old-school "cutting-heads" mentality (nor have had the cops called on me by a douchebag drummer). There is an audience for jazz here too, so the playing opportunities do exist.
Also, there's a 24-hour subway, which gives one plenty of opportunities to unintentionally pass out on the train and wake up to a sunrise in Coney Island :-p