If you would like some peace and quiet -- go to this island. Bring a bike, pack some groceries, plan to do some home cooking while you're there. They do a really nice (though I hear expensive) restaurant called Willows Inn. Even if you don't go there to eat out, you should consider driving by and parking your car there because there's a really convenient spot to go on to the beach (especially if you want to watch the INCREDIBLE sunset off of the island). When we were there in July they had a community chili cook-off -- it was something like 6 or 7 bucks to get in and THERE WAS SO MUCH CHILI AND FOOD!!!!
The people on the island are super friendly and laid back -- you really need to experience this. As a native of the Philadelphia area, it was a wonderful culture shock.