I also went to UW for undergrad. I loved living in Madison. There is also more to do than drinking, though it does have a great bar scene. There is a lot of live music, theatre, film festival in the summer. It's also close enough to go to Chicago on the weekends, or up north to go hiking or camping.
Cost of living is pretty good. I only payed 280 my senior year, and shared with 3 girls only about 5 blocks from the school. You can live downtown for under 600. You don't need a car, you get a bus pass, and everything downtown is walking distance.
The campus is gorgeous especially in the spring. The winters are harsh, but I think can be really fun.
There are homeless people, like in any city, especially in the downtown area. Politics in Madison are really interesting. Obviously mostly liberal, but a large socialist population keeps things even more interesting, and the conservative sect is loud and proud.
There are a number or great free events through the year, including speakers, concerts, symposiums etc.
I miss Madison greatly.