But before you go, make sure to read up about him and his architectural ideas. Knowing about all the care, thought and attention he put into designing the house really makes it interesting. For example, the green shuttered rooms on the back of the house were added to give him some privacy in later years because of all the visitors that came to the mountain to try to see him. He didn't want to add them because he thought it destroyed some of the perfection of the octagonal shapes of the house.
He was a fascinating man and this was one of the most interesting historical homes I've every been to. The tour guide I had on my first visit was pretty decent, not anything spectacular but certainly knowledgeable and pleasant. The woman who gave the Plantation Community Tour was awesome! Definitely take the Plantation Community Tour. If you're into gardening there is a gardens tour, but if you're not into plants and trees, definitely skip that.
Plan to spend a while walking around taking in all the different areas and grounds. It is truly beautiful.