I always wondered. Also, do military special forces use body armor? If so, what level are they?

Swat teams wear level IV and higher body armor.

Of the SWAT types I know, and I know a few, Level III and IV primarily.  Of course, Level IV armor is only Level IV where the trauma plates cover.

Swat operators follow the direction of Mall Ninjas

www.mallninja.com/

In my experience, civilian SWAT mostly IIIA vests with level III or IV hard plate inserts.

US Military special forces use a variety of body armor types, the most prevalent (and most often seen) currently being the Interceptor OTV system made by Point Blank.  This is a Kevlar KM2 IIIA vest with NIJ Level III ceramic SAPI hard plates inserted to increase protection to cover 5.56mm, 7.62x39, and 7.62mm NATO hits to mid torso/mid back.

Mall ninjas use level IV+++++ head-to-toe body armor along with 1/2" ballistic steel plating on their pursuit golf carts.