The Richard Thompson WMA is pretty good-I actually grew up less than 2 miles from it. Sky Meadows, at The Plains, I believe, is another good day hike with beautiful views.
In the GW, I'd recommend Elizabeth's Furnace in Fort Valley, between Front Royal and Strasburg. Some great hiking, good prepared spots and lots of great spots on either side of an easily accessible creek. Actually, looking back at your criteria I think this area would probably suit you best.
Andy Guest State Park is also out near Front Royal. It's a pretty new park, opened only a couple of years ago, and the campsites are great. The park has several miles of frontage on the Shenandoah-not so much a park for hiking but great if you want to tube or canoe the river and then camp out.
If you're willing to go out a little further, there're several good places out past Sperryville. Check out the areas around Old Rag Mountain and Mary's Rock. These are in the Shenandoah so you'd have to get a back-country permit, tho.