I visited a few days ago, and right now (late April) the poppies are blooming. It's a lovely place, but I think calling it an arboretum would be a little more accurate than calling it a garden. It's like a little nature preserve, capturing a small redwood forest and the stream bed deep in the arroyo. Fringing this are the visitor paths and the few more curated gardens. The canyon still contains the remains of the irrigation systems built in the early 1800s. Stone-lined aqueducts and decaying wooden troughs can still be seen there. The system used to carry water all the way down the mountain to the Mission. I really liked the Manzanita grove, something that's pure Santa Barbara, and smells great. Part of the garden grounds are across the street, and lead up a hill to the conservation center, which features an amazing view, all the way to the ocean, as well as some nice places to sit down and enjoy the scenery. I took the hard path, but I noticed too late that an access road will take a golf cart straight up from the parking lot to the conservation center. A good option for handicapped guests.
Speaking of handicapped, even though the paths are well-maintained, given that it's a canyon there are some dramatic changes in elevation that might make navigating difficult if you are in a wheelchair or on crutches. Also, if you're susceptible to heat stroke (like I am) please pace yourself and be prepared. I failed to do this adequately and had to cut my visit short, because the heat was getting to me.
What I mean to say is that this is still a lovely place to visit, but this isn't just a casual, flat rose garden. It's an active, semi-wild landscape, and I would prep for it as you would a hike, rather than a walk in a city park. I'm always happy to pay the admission that a garden or zoo asks, because I know the money is going into the maintenance of the place, and I think that has value. Crowds are thinner after 2pm, but come earlier in the day if you want to beat the heat.