I'd have no qualms about using the water from one of the dehumidifiers, as long as it's treated as any other 'found' water source. Filtered, tabs, run thru the berkey, or whatever method.
We keep one in our 'intermediate' pantry; a 7x10' climate-controlled room. We don't get a quart a month from it now, in that room. But that's because it's already so dry in there, that there's simply not much moisture to remove. In an open area or a room that hasn't had a dehumidifier, the same typical unit will pull several gallons a day from the air. (So if we've got a generator and fuel, we've got water straight from the air.)
I also keep one in another storage room. Things I'm going to put back long-term (rice, ammo, whatever) get left in that room for several weeks to reduce their moisture content before being put into long-term storage.
Both units are kept at their lowest setting of 35% RH, which as I understand it, is roughly the same humidity level of the mojave desert.