I was just reviewing the "best" restaurants in Annapolis on this siter and thinking how difficult it would be for an outsider to use these ratings to decide since the restaurants vary greatly from pizza places to crab houses, to delis to tourist traps to chains to excellent/ fine dining. Also some weird or unknown ones made it high on the list!!
As an Annapolitan and frequent eater, here are my suggestions for fine /good dining for couples/ small groups (older children): O'Leary's, Les Follies, Jalapenos, Yin Yankee, Joss, Tsunami, Metropolitan, Lemongrass, Indias, Wild Orchid, Piccola Roma., Kyma, Lewnes. You'll have a decent, perhaps excellent meal at any of these establishments. Call for reservations - definitely call on weekends..
For casual dining:: Boatyard, Carrolls Creek, Pauls, Rams Head, Squisitos (several locations).
Avoid restaurants near the water unless the view is all that counts.
Many of the other places on the ranked list are overpriced tourist traps.