Horses are expensive and require maintenance and care. If you are already into the animal raising lifestyle, you will notice less impact if you bring one home. If you don't raise animals they can have a steep learning curve. All that aside, they can be fun and if you like to ride, there is no substitute. A good trail horse or a good mule can be a large source of enjoyment and could serve as transportation in tough times.
I look at them like reloading. If you are trying to reload .223 on the cheap, thats gonna be tough because decent ammo right now is affordable and you probably can't do it cheaper. If you want custom loads though, you can roll your own for that purpose. And just to have the capability, its worth the investment.
Such is the way of horses and mules. They are not a money saving source of transportation but they can be used when gas is unavailable. Plus, as noted earlier, if you like animals and like to ride, they are fun.
Don't think you'll save any money owning one though.