Naval Academy tour should be first stop if you've never been there. Some people also like the Paca House tours.
Londtontowe down in Edgewater is cool if you're into that sort of thing.
Good people/boat watching can be done at Pusser's Landing, but I never eat there.
The best part of Annapolis is honestly just walking around and looking at the houses and enjoying the town. I've lived here 14 years and still find new stuff every time I go for a walk.
For seafood, do not go to Buddy's. That's where we send the tourists so we have the rest of the places to ourselves.
I like the cream of crab soup at Acme.
Stay away from most of the restaurants on Main Street and the City Dock….period. Over priced and indifferent service.
We like Galway Bay on Maryland Avenue. It's not "Maryland" seafood, but they have good seafood all the same.
The Point is a little bit of a drive (15 minutes) from downtown, up in Arnold, but you'll get more bang for your buck and it's right on the water, something that most Annapolis restaurants surprisingly can't offer. Best fish and chips I've ever had, even after 3 trips to England and Ireland. The crabs always look good there and they sell a bunch of 'em. Decent beer selection….not overwhelming, but some decent choices.