Parts of the Out Islands of the Exumas have become a Nature preserve and I've heard there is a bit of camping and sea kayaking going on there. I lived/cruised there on a sailboat for a year back in the mid-80's. Can't say I saw a lot of camping going on though. Internet search may give you more current info but much of the Bahamas will charge big for that "getaway" feeling. Have you considered the East coast of the Yucatan in Mexico ? Been there also ... Fly to Cancun, rent a VW beetle (cheap/lockable) and drive South, hitting the interesting little dive "resorts". Some are quite rustic and have a real laid back feel with good cheap food places there or nearby. The people are friendly, English is seldom a problem on that coast, and they'll usually negotiate your accomodation rates when you stay for more than a couple days. The cenotes (inland open-top caves) are a unique experience, also. On the other hand...if you don't dive ??? Scuba is great but simple snorkeling is just fine most places. Many places will teach/certify SCUBA, also . Enjoy. Stay safe