55 Bar...still only $10, cheapest ticket in town for a world-class jazz bar. There's a 2 drink minimum but I don't think they keep track. Nir Felder plays there every thursday unless he's out of town touring, if you're into Ari and Gilad I highly recommend checking out his show. Also for $10-$15 there's the Cornelia St Cafe, which can have some great stuff. If you like experimental/Avant-garde music there's also The Stone in the East Village for $10, which is a performance hall with John Zorn as the artistic director.
Only the other thing on the cheap I know is Fat Cat, which has a $3 cover. The music isn't always great, but it's a cool place to hang either way. They've got couches, pool, ping-pong, shuffle-board, chess, and checkers. You can get great beer or $3 cans of PBR (3$ for a shit beer is actually really cheap for NYC)
Also check the schedules for all the big clubs, Village Vanguard, Jazz Standard, The Iridium, Smoke, Birdland, Blue Note...these places can get kind of pricey, but a lot of heavyweights play at these places, so if there's someone you really want to see then it's worth it. One of the cool things about this town is there's always someone I'd be interested in going to see on any given night.
Other recommendations: Buy a 7-day unlimited Metrocard (works for buses too). If you smoke cigarettes, bring some with you. With all the city taxes they run about $13.00 per pack. If you go to the Empire State Building, don't do the SkyRide, it's for little kids. Eat an American-sized burger at Jackson Hole, don't go to Shake Shack like all the other tourists do.