A picnic lunch on the Banks of the Rhone just outside of Avignon's city walls can be delightful.  Head early to Les Halls (Avignon's large enclosed fresh produce marketplace) and stock up with a bottle of wine, some cheese, olives and bread, each of which can be purchased at adjoining stalls.  Don't forget a corkscrew.  The picnic will be far cheaper than a sit down restaurant and just as romantic!

Just over the Rhone from Avignon proper is Villeneuve les Avignon (sometimes seen as Villeneuve lez Avignon).  This town was established by the Popes and is a must see part of your trip.  If you're looking for a quieter tourist location this is it.  There is plenty to see if you want to take a day or two here.