We had a flight delay, and therefore an extra day in Vermont. We asked the employees at the front desk of our hotel for a non-touristy fun thing to do and they suggested Shelburne Farms. It was great! I wish we would have come here earlier in our trip, instead of wasting time at the Cabot Annex and eating non-delicious "legendary" cider donuts.
Definitely take the tour, and try to go on one of the earlier ones so you can actually see the cheese making in progress. Also, even if you don't have kids, go to the petting zoo. I got to hold a goat! And be sure to eat at their farm stand--it takes credit cards and everything we had was delicious (especially the cider donuts). You can eat in the Pipe Room nearby if it is cold/raining.
The store at the Visitor's Center is charming and is reasonably priced, so be sure to make a stop there. We only had 4 hours, but wished we would have gone first thing so that we could have done some hiking and explored the house more.
This review is the subjective opinion of a TripAdvisor member and not of TripAdvisor LLC.