My wife and I are thinking about retiring to the Prescott area in a few years. We'll be in Prescott this week to look at  neighborhoods and I thought I'd get some opinions on areas to look at.
We'd like a moderate sized house, 2,500-3,000 square feet, on a 1/4 acre or more and $1M or so max. Wife needs a pool (or at least access to one) and I'd like a large detached garage or shop. Wife doesn't want to live out in the sticks, so something within 5-10 miles of city limits.. Any suggestions on areas/neighborhoods?

Welcome!  This is a great area to live.  The other day I was in Chino Valley at a BBQ joint (Big Daddys) and there was a young man in there open carrying a custom 1911.  God bless Arizona.

You will have to be a bit out in the sticks to get what you want.  Suitable areas would be north of Prescott along Williamson Valley road.  Some areas northeast of Prescott Valley are suitable say up Coyote Springs road there are still a lot of 10 acre parcels and homes.  A little farther east up along Prescott Ridge road there are homes with a few acres and some multi acre parcels.  

Please remember once out of town limits in the county you will be well/septic, propane and limited police/fire ambulance services.  I live northeast of Prescott Valley in the county, on 12 wooded acres.  There is no fire dept service where I am and we rely on the YCSO.   They run 8-10 cars for 8000 sq miles.  The closest anything to me is 9 miles.

It's worth it.

I can tell you now, you probably won’t get a neighborhood with a detached garage AND access to a pool (unless you go use the YMCA, the college, or the Prescott Valley one at the park).

We used to live in the Hidden Valley Ranch subdivision, south of town off Senator Highway and Haisley Rd.  It borders on National Forest and has some really nice homes.  There's a community pool & tennis courts that didn't get a lot of use when we were there, plus an excellent RV storage area with water and dumping facilities.  Lots of wildlife in the area so bring your camera.

You are asking a lot if you are expecting to find a home already built with your specs.  What you are asking isn't impossible, but it will be close...you will have to settle on something.

You would be better off buying land, building a home you want, and adding a swim spa on the back deck.  Or joining the local YMCA.  

I've got property up Lynx Lake/Walker Rd that I will be building on some day.  It is out of the way, but also less than 20 minutes to CostCo and the town.

We're full.

I live in Prescott valley ( technically Dewey ) in the quailwood subdivision/ hoa.
We are happy here and it suits our needs. Won’t get you the land or big workshop though.
Trying to be close to town plus a big lot plus workshop and either a pool or access to one under a million is a tall order.
If only your wife and you, not sure why you would need 2500-3000 SF.?
Our house is 1800 and plenty for the wife and I with enough space for visitors gun safe reloading etc.
Community pool is real nice too.
The typical nice 1600-1800 SF house in one of the numerous subdivisions ( the usual small lots, hoa with dues but pluses like community center and pool) is now running about half a mil.
Also with water so scarce houses that might fit your criteria and budget may have issues with water.
When I lived back East my well was like 90 feet deep and was great. Here wells can run 700+ feet deep and what happens when they run dry? Dig several hundred feet more at 80 or more bucks per foot and there are issues with various contamination in the water ( coworker having challenges now with a high arsenic content in his well) some houses have water storage tanks and have water delivered which is expensive also.

Another HUGE consideration people often overlook is if you are near or past retirement age, close proximity and access to health care already is or will become important- in “smaller” communities like we have here certain specialties are limited or nonexistent and even primary care is hard to come by.
I see lots of posts by older retired people talking about getting that remote 50+ acre parcel, then have to remind them the nearest hospital is 2 hours by ground ambulance! If you don’t have chronic illnesses that need attention that is great, but sooner or later you will.

@ErikOR
Prescott is a great area to live & retire. Been here since 2006 and love it.
The cost of free standing homes here has reached levels that are greatly inflated and in my opinion unsustainable. Similar to the price run up in 2004-2008 we are in bubble territory and I expect to see a correction just like we experienced after the 2008 crash. Since you are looking to retire in a few years I suggest you either wait for the correction or buy land and hold off building until prices come down.

For $1M you could find what you are looking for minus the pool but there is fierce competition for homes so be prepared for listings to go fast. As for building new the builders here are booked solid and cost per square foot is pushing $300/sqft with many builders quoting open ended cost based on the price of materials when they actually start building (~1 year out or more from contract).

My advice is come out and see the various environments and neighborhoods and if Prescott feels right wait before you buy.
The Williamson Valley area has a lot going for it. I suggest driving from the start of Williamson Valley rd and go out 15 miles or so. The further you go the more rural it gets and the larger the property.
Best of luck

We're heading home now after visiting Prescott yesterday. We really like the area, climate, etc. It's slvery similar to Central Oregon's climate. which we really like.
I agree that the Williamson Valley is a very nice area with better prices than most other areas.
We figure if housing prices in the area don't level off or or drop somewhat, we' might just stay put in Oregon with the high income tax, outrageous property tax and shitty politics.