The building has limited hours - so if you want to visit the Mansion itself plan ahead. The tour is worth the time.
But you can visit the grounds even when the Mansion is not open. The view over Portland is outstanding. When the weather is clear you can see all the way to Mt Rainier. The Parks dept keeps the place looking great.