Though the OP needs to man up and end it, I don't think that he's being superficial to a point that its LOL worthy. The first thing I care about in a guy is looks. My best friend is a man who absolutely adores me, supports me, and is always there for me when I need him, but I would never consider dating him just because I'm not physically attracted to him AT ALL. If you have to force the attraction, there simply is none.
I'm sorry but if I was with a guy who used to hit the gym and was all buff and sexy when we started fucking, but then got too comfortable and became fat while I was still in shape, I would lose interest. If I'm working out trying to look good for you, I expect you to at least try to do the same for me. Reciprocity matters. Besides, I would want my man to still look at me and find me sexually desirable. If I know I'm out of shape but my man is hanging out with his boys in bars/clubs amongst gorgeous girls (admittedly this isn't the case of the OP), that's when the seeds of jealousy and insecurity start to set in.
Looks aren't everything, but I'm honest enough to admit that it matters a hell of a lot to me. In this regard, the only difference between me and the OP is that I would (and did) have the balls to admit I wasn't happy and leave.