I have also worked for large companies and small companies. I will never go back to a large company. The key is finding a good small company - this means a good owner, someone you can get along with and is not a nazi about everything.
From my experience:
Large companies - better benefits, inept managers, budgeting concerns overpower personal decisions, less freedom
Small companies - benefits not as good, only people that can actually get the job done remain, much more personal freedom
For example, I'm currently at a small company. I came here three years ago and built my department into an efficient and profitable machine. Insurance through the company sucks, but we opted to use my wife's to alleviate that problem. I was able to hand pick my team, and train them to my preferences. I answer only to the owner of the company, and as long as I maintain a good profit margin, he's pretty hands-off. I come in when I want, I leave when I want. I browse ARFcom between reports. I am not subject to bullshit PC rules or other HR nonsense.
YMMV :D