I'm sure this is a much asked question, sorry, but.

I want a polymer lower for a 22lr upper, just because.

Would one of the smaller Harbor Freight drill presses be adequate to mill out the pocket?

I don't need a drill press for anything else and I surely don't need a milling machine, not to mention space.

Can it be done?

I'm sure it would be fine. I've never done a poly 80% lower, but I've done aluminum lowers.

Aren't poly 80% 3 times as much as an aluminum?

what you really need is a plunge router... I dont have a drill press and I do aluminum 80's.

It can, but if you can go the router way you won't need the $80 cross slide vise.

Well, I already have the vice.  I bought it years ago to hold a mag well insert.

But without gearing I'd be sliding the vice along the drill press table.  That sounds kind of iffy to me.  Obviously I have machine experience.

I'll have to look into the plunge router, that's not a method i've seen mentioned so far.

First of all you need a cross slide vice on any drill press you use. It essentially turns it into a mill.
I went to harbor freight to look at the drill presses just this week, for other reasons, not for milling lowers. The smallest one they have, $69 iirc, seemed really crappy. The deck, or table, whatever they call it seemed to be lacking on area to clamp stuff down. Also I'm not sure if you have enough room between the Chuck and the vertical tube (I don't know the technical name, but the thing that gos from the floor mount all the way up to the motor. It's not very deep. I have a pretty big drill press, probably in the upper mid range of what HF sells, and I even run into problems if I don't set it up right and my cross slide hits the rear post before I finish the cut. I just bought a Skill brand press for $119 somewhere else because I liked the deck better and looked much higher quality. It's my second press.

$19.09 per in bulk:

https://www.eparmory.com/BUY-7-GET-4-FREE_p_395.html

Check out YouTube.
Most of the jig companies reccomend a router

Yes, a polymer lower can be milled out on a drill press.  A X-Y vise would be the best choice, though I have seen others do it by hand with a standard drill press vise.  You have to have enough room to work with the jig and vise, so I would recommend at least a 10" drill press depending on the height of your vise.  I personally wouldn't recommend a router for polymer lowers if you can't get the RPM low enough.  Heat will cause the polymer to soften and smear instead of cut.  Even on a drill press, if I ran the RPM too high it just melted the polymer and I had to spend twice as much time filing and sanding to clean up the slag and waves.

The runout on those is ridiculous.  They are barely worthwhile for drilling holes.  Get a router and a jig.

This +1

I did an EA poly lower (zombie green) for my dedicated .22LR built (Spike's upper).  It was the first lower I did with my original Modulus Universal Jig (I upgraded to their HD jig after doing a couple billet aluminum lowers and passed the original on to a friend). I have a Makita router and it cut the poly lower like it was warm butter.

+1 here.  I had their cheap benchtop drill press.  Not only is there not enough vertical height the table wasn't square and the runout was stupid.