If you want to see the whole island, you should rent bikes. It's really the best way to see the island. It's about 7 miles long so there is no way you can walk the whole thing. Other options are to rent a cab for sightseeing, and then walk around Old Harbor for the day. A *must* see is Northeast Light, which is possibly within walking distance of Old Harbor, but ideally you should take a bike there.
Another *must* see is Crescent Beach (Fred Benson Town Beach). It's seriously the most beautiful beach in the whole state. I LOVE swimming there! The sand is like silk underfoot. There are chair and umbrella rentals, a bath house with showers, food, etc there.