Jory is a very good restaurant. If you stay at the Allison Inn, you ought to eat there. But it is good enough that you should eat there even if you are not staying there.
If it is warm, eat outside. Tables are spaced well, so that you will have privacy. You'll also have a pleasant view. Inside is nice as well -- you can eat at a table, and there are also seats at the kitchen bar.
Service was attentive, efficient, but not rushed. Food came at a nice pace, but one easily could linger. The menu is small, but you can also order off of the bar menu, which adds a few more options. Portions were nice -- not huge, but also not tiny. The food was fresh and local -- great combinations, great flavors.
The wine list is very nice. There are also lots of selections by the glass, and a few wine flights. If you want to drink a bottle you brought with you -- or bought earlier in the day -- they appear to make it easy to do so.
All in all, a lovely, yummy, romantic place. Not cheap, but not as expensive as you would think for excellent unfussy resort food, and definitely not an expense that you will regret.