My wife and I did the Gaspe trip last September on our way to Prince Edward Island. It is a must see! Dramatic coastline and roads that hug the sea for most of the way. If you have the time you will love it. We live in southern Ontario so it took us some driving to get to Rimouski where we stayed at the Hotel Rimouski. It is a fairly large convention type hotel and a great spot right on the waterfront. The next night we stayed in Gaspe at the Hotel des Commandants after about a 350 Km drive. Another great find. They both have excellent restaurants too. It is about another 320 Km or so to the New Brunswick border. The area is VERY French, nevertheless the folks in the hospitality industry all speak some English, and you can get by at the gas stations with a credit card and very little French vocabulary.
Perce is a touristy little town. Lots of mom and pop type motels and beach type restaurants and souvenir shops, but very pretty. You will want to spend a couple of hours there at least.
As for the weather in May, the area is quite north, so it will probably still be quite cool with frosty nights, but the days will probably be comfortable if the sun is shining. I expect that you could run into a bit of fog in the area if the conditions were against you, but that is part of the charm of the area too.
I have done quite a few of the scenic drives around North America, and I would rate this one high on the list.
And contact the tourism folks in the Province of Quebec and they will send you a comprehensive information package about the area as well as a terrific map of Gaspe and guide book with lots of B&B���s if you like to go that route.
If you plan to visit PEI, you could take the Confederation bridge over, and the ferry back to the mainland. We stayed at a great B&B there that I would be happy to tell you about if it is on your itinerary. Just repost and I���ll watch for the next few days if you are interested.