I recently stayed for two nights at the Travelers Palm Inn on Catherine Street in Old Town, Key West. It was prefect for a quiet weekend getaway. It was close enough to walk or bike to pretty much every attraction on Key West, yet several blocks away from the loud, late night madness of Duvall Street. The property itself is beautiful, with well manicured tropical landscaping and a nicely maintained small pool. The rooms are well appointed and away from one another, there was no noise from neighbors or other guests. In fact, even though the place was full, you could hardly tell others were staying there. The proprietor, Michele, is very friendly and accessible via cell phone if needed. He provides good recommendations for both recreational activities and dining. You can rent bikes from him as well, which I would highly recommend doing.
I would say the only thing that may dissuade some people is that it's not quite the traditional "B and B" experience, if that's what you're looking for. No breakfast served and no cocktail hour, no socializing with other guests, so if that's what you're looking for, check out the myriad of places both on and off Duvall Street.
If you're looking for a quiet and well appointed room, competitively priced, close to everything with friendly staff, I think this is a great pick. I would stay here again.