It's a darn good field, with companies all over the place, so no need to stay in Cali. For example, I work for a small studio in Utah. NWManitou from this site works for another such company in the intermountain west. ReelFX in Dallas is a great studio.
Pay is pretty good, around here you'll start out at most places at around $40,000. The field is more resilient than some of you think. Our studio, for example, goes up and down on Hollywood contracts (always expect to be screwed by them), but industry pays very well for commercial work. It's plentiful and when the economy goes south they just want more of it! If you're a small studio you really have no trouble meeting payroll just with small, quick, local jobs.
Additionally, freelance work pays very well. Only negative is it's less consistent, but some of my coworkers made much more working freelance than they do now (they got married and wanted the benefits and stability of a company).
You really don't need to go to school to do this. If you're a genius and can figure out Maya on your own, you're in. My officemate here, for example, never did, and he's a freakishly talented guy. Getting a job here is all about your demo reel and, often, prior work experience. No one's too impressed by what school you went to... but, like anything else, that university training is invaluable in helping you get good demo reel work. The math side is much overrated, you need some basic programming skills, but I think I personally overtrained in that regard- I almost never use those skills. They have specialists for that stuff.
They're right though in that you'll end up getting specialized quick. Unless, of course you work for a small studio- I have a friend at ILM who does lighting for hair. That's it- just sets up lighting for any hair in a shot. I, on the other hand, only get to work on romantic comedies and other light FX films, but I get whole sequences to myself. You decide which way you want to go. Small studios want generalists who can handle anything and you'll get a lot of work on small movies. Big studios want experts who are the absolute best at one specific thing and that's all you'll ever do, but you get to work on the biggest films.
In short, I guess, it's definitely worth it. It's a fun field, whether you end up in video games (which are booming), or VFX. If you've got the interest, patience, and talent, go for it!
ETA: Oh, and make sure you learn one of the top tier packages. I recommend Maya. It's impressive and you can always go back down. http://www.alias.com has a free trial version.