Williamsburg is a great place! Go during the right time of year and you won't be disappointed.
Go see the colonial governor's mansion and you will get to see guns. Lots of em:
If you can look at that and not get instant wood, then you belong on a knitting forum.
Beautiful flintlock muskets, pistols, and swords arranged beautifully along the walls. It will give you a sudden and intense interest in redecorating your house.
There are lots of things within driving distance of Williamsburg that are lots of fun. King's Dominion is a pretty nice place, and you also have Jamestown and Yorktown nearby, which are both great places to visit.
The MacArthur memorial is around there too.
Lots of good history stuff to keep you busy for several days.
Just watch out for the guys shooting off canons in colonial Williamsburg....
Virginia beach is also pretty close, which can be a real blast at certain times of the year. Try night fishing on the Virginia Beach pier when the blue are running.
There is a marine biology museum in Norfolk called Nauticus, and there are some Naval museums as well.
You could spend two weeks vacation in the area and still not see everything there is to see.