I work in the hotel industry here in Victoria (not the Aerie :O) ) and I am happy you are coming here! Great place for a vacation I think. I would like to tell you what I "see" in your post... You say "my wife and I", "sailing", and "beaches". I translate that into you are looking for a romantic getaway. I would not think the Royal Scott is appropriate for you. The Aerie is. The Aerie is absolutely beautiful. I have been there on crystal clear summer nights with beautiful sunsets with eagles flying by as well as the quiet solitude of being the only two people in the dining room on a stormy winter night just listening to the piano and drinking wine. It is a magical place. If romantic is what you are after there is no choice in my mind. The Aerie is a little bit of a drive outside of Victoria though (30 mins) so if you have the time I would stay a night or two actually in downtown Victoria. Along the same romantic lines of the Aerie (but not quite the same caliber) is a bed and breakfast called "Abigails". It is walkable to everything downtown. A f-a-n-t-a-s-t-i-c restaurant downtown in called "Cafe Brio" and is a 10 min walk from Abigails. One of the three best restaurants I've every been two. Note that one of the other two was the Dining Room of the Aerie.
If you are a little on the adventurous side I would suggest things to do like a small kayak trip in the inner harbour. It is one of the most beautiful harbours in the world and being on the water reveals that for sure! Also the 30 minute Floatplane tour around the city is lots of fun especially if you haven't been on a floatplane before. Now obviously there is the world famous Butchart Gardens which is a must see. That is also about a 30 min drive outside of town. Downtown, a must see is our museum, the "Royal BC Museum". If you are interested in the natives here and how the lived, the pacific coast rainforest, old ways of Victorian living, the dioramas in this large museum are impressive.
Sorry for being so long, but your email struck a chord with me and I want everyone who comes here to have the experience they are looking for. If you are on any kind of a budget though, I would only go for dinner at the Aerie and not stay there... skip Cafe Brio for Il Terrazo... drop the floatplane for the kayak, etc. BUT if you want to do it "right" and have unforgettable memories... I would bite the bullet and do it. My motto is that you always remember how much you spent when you didn't enjoy it, if you enjoyed it you only remember the experience. My email is firstname.lastname@example.org if you have any specific questions you would like to ask about Victoria.