The M4 takes a AA battery, has an integrated mount, and multiple reticle choices. The M3 takes a different type of battery, the mount is extra, and is just a dot (there is a version with the same reticle choices as the M4). The downside of the M4 is that if you decide later that you want to add a 3x or 5x magnifier, you will have to remove your rear sight to have enough room to do so. The M3 with cantileaver mount gets the sight forward enough so that there is room for the magnifier to sit between the red dot and rear sight. If you are sure that you will not want to add a magnifier at some point, the M4 is probably a better choice. The M3 will leave the option open for a magnifier if you ever decide to add one. Just to make your decision a little harder, there is also a PA microdot. It is small and light and would also leave plenty of room for a magnifier.
It really sounds to me like you will want a magnifier at some point. If you are just shooting a single target at the range, magnified optics are more fun. In my opinion, your best bet would be to get the PA M3 multi reticle (if you want something other than a dot) on a cantileaver mount. Add a magnifier on a flip-to-side mount (I don't think that PA has a flip mount option yet) later as your budget allows. If you don't want the double optics thing cluttering up the top of your rifle, just get the Millet DMR or similar scope and have fun.
Why not call Marsh at Primary Arms and get his opinion? It will certainly be better than mine
.