The garden is not just beautiful. It is also culturally interesting. The gardens were originally constructed by the fuedal lords in the area starting in the 1620's. It was the Matsudaira Family's villa for 228 years and opened to the public in 1875. It is a step back in time to wander this fuedal lord's strolling garden. Go to the tea house. Read your little guide pamphlet as you go. It will definitely add to your enjoyment.
As a side note, we had chestnut ice cream. Wow, that was good.
We are glad we made Takamatsu part of our visit to Shikoku. It is a good one day stop. The small city is somewhat off the tourist route, but very accessible. Besides the gardens, the city has a very long shopping arcade between the train station and gardens. Since the city is small, taxis are relatively inexpensive way around. We picked up a rental car the next day and made our first stop the nearby Shikoku-Mura folk open air museum which was a great surprise.