We just stayed at Fort Recovery for a second time. It was the peak of low-season, so the island and most places were quiet and many closed, but we had an excellent time. My wife, and two daughters (ages 3 and 5) stayed in the two bedroom suite with full kitchen and loved it. It's such a nice and mellow place and the staff is great. Rooms were cleaned every day and they made sure our villa was in great order when we returned in the evenings. We spent most of our days out and about, checking out all of the beaches, sites, restaurants, and the Baths on Virgin Gorda, but they have a pool, a decent little quiet beach, a spa, and a restaurant if you want to be mellow and stay in at the hotel.
I like the West End of Tortola, it's mellow, and close to Smuggler's Cove, Cane Garden Bay, and the West End port. You can hop over to Jost Van Dyke in 15 minutes on a Ferry. There are also some good places to eat near by like Fish n' Lime Inn (the old Jolly Rogers), Soapers Hole, and across the hill is Sebastians. We flew into St. Thomas and took the ferry to West End. Denzil Kline has a nice local jeep rental place close by, and will pick you up at the port upon your arrival.
Fort Recovery is a great place to stay, and I highly recommend it.