Alright....here's my $0.02:
I'm from Chesapeake as well. I am a huge aviation geek and I love photography.
I have done the drive up, tour and drive back. I did not really feel like I had enough time there and I was dead tired when I got home.
For the past few years Udvar-Hazy has had an open house where you could tour some of the conservation, storage, and restoration parts of the facility. So far this year, they haven't scheduled a date for it (I've asked a few times already).
When I have gone to the open house in the past, I do one of two things:
1) Drive up Friday after work and stay at a Hotel Friday and Saturday night. Plenty of time to do what you want and not feel rushed.
2) Drive up early Saturday and do the open house, and stay at a hotel Saturday night. You have plenty of time at the museum, and you have somewhere to crash because you're going to be tired if you drive up and spend all day walking around.
You could also drive up, get a hotel the night before, go to the museum in the morning and drive home whenever you're done.
A word of warning: the ONLY food on site is an in-house McDonalds. If you're going to spend all day there, I'd suggest keeping some food in the car. It also gives you a nice break to walk outside and get some air. Doubly so if you have a bored wife and/or kids.