To me, the most amazing and unique thing about Hawaii is the vegetation- giant trees, beautiful exotic flowers, amazing views. This is where you go to see them up close.
We were the only people there when we went on a week day in September. It was hot and humid, but really amazing. I brought my 3 year old, 6 month old, and pregnant neice. The place is great for kids- no rules, just walking around on trails is what you do. Now, you do walk up a pretty steep hill and it was tough on our pregant compadre, but you can take as long as you like to do it. The view at the top was just about the best thing I've ever seen (and I've been to some amazing places on this planet).
Its just a great authentic place- not touristy. The gift shop is even nice. The drive out there is great too. The arboretum is near the hike to the waterfall (I forgot the name- we didnt do it w/ kids and pregnant mama). I think we were here about 1-2 hours total. This is definately a must-do.