I've lived here a year, but still enjoy visiting this beautiful park whenever I'm uptown (that's what the "Bay Street" area is called). We'll have lunch at one of the restaurants on Bay St. and then stroll along the water or sit in one of the wooden (squeeky) swings. There is also a playground at the far end toward the bridge, which my 4y.o. son absolutely loves - it even has a public restroom (with a changing table for babies).
The thing is, you have to pay for parking via meters along Bay St. or use pay machines if you park in one of the lots (and put the receipt on your dashboard)... Sometimes the meters will be paid for by the local business on holiday weekends (not the lots, just the street meters), but that makes it harder to find a spot because people will be parked for longer... If you park in one of those lots, 15min = $0.25, and no change is given if you use cash, so have quarters ready or just swipe a card - yes, they take credit cards at the machines. The meters become free after 6p.m. because many of the stores close at 5p.m., so it's mainly for the restaurants and the park visitors... take advantage of this if you want to see a beautiful sunset while at the park - and flying a kite is great here because it's windy most of the time, just watch out for the higher trees.
I'd reccommend parking at the lot at the end of Charles Street, the one near the Marina, because it usually has the most spots available and you're not having to parallel park on the main street. Or you can park at the playground lot because it's near the only public restroom (unless you go into a restaurant, but that's frowned upon if not a patron). To get to the playground lot, go the very end of Bay Street (east), take a right, and then the very next right into the lot.