Charles, the Sunday market in L'Isle-sur-la-Sorgue is an institution and should not be missed, IMO. But, you can get there real early, walk around, make your purchases, assuming you want to buy anything at all, and leave by 10:00-10:30.
Assuming you did not buy a rottiserie chicken or foie gras which need refrigeration [sp but datt will surely fix it :) ], you can then continue on to Gordes, do a walk about, and even catch another village or two of the Luberon, e.g. Rousillon [sp once again]. Lunch ... I cannot recommend one restaurant in Gordes because I think they may be too tourist trappy. So just grab a slice of pizza or a sandwich. The views onto the village of Gordes are striking to say the least. I think the best of all the villages of the Luberon.
BTW - there is no lavender in front of the Abbey for several years now. Someone said there is wheat but we just saw a bunch of weeds in June 2011. Still, lovely architecture and well worth a drive down a crazy one lane road.
Now, the fact that you did the market in L'Isle-sur-la-Sorgue on a Sunday should not prevent you from heading toward Bedoin on a Monday. The market is much smaller but very nice and the whole town of Bedoin is a charmer. You could certainly grab a bite to eat before heading out. After Bedoin, you could visit Vaison and some wine villages in the Dentelles.
Here you go - two days worth of outings!