I quit hunting with a rifle about 20 years ago after just one too many incidents of being shot at. The straw that broke the camels back for me was when I got so pissed at the idiots that shoot at sound or movement (after having another round sent in my general direction) that I found myself shooting back at them. I fired several rounds in their direction (high into the trees above their heads) and shouted "Scary, isn't it?". After that day I've never rifle hunted again. I bought a hunting bow and learned to use it and since then have only hunted during special bow only seasons or in areas where firearms aren't allowed during hunting season. I've just found that bow hunters (as well as most pistol and BP hunters) are a more dedicated breed, better equipped and trained and a lot less likely to shoot at sounds or movement.
In the 15 years I hunted with a rifle regularly I can't count the number of times I saw Rangers and Police with some hunter pulled over for doing something stupid like bagging the wrong game, elk during deer season or vice versa. Or some poor farmers cow or horse. Or some other hunter. Or for using a a weapon not allowed, like a rifle during pistol season, or a centerfire rifle during BP season. Not a single season passed without running into at least one idiot with all his gear confiscated and on his way to court.
Unless you get to an area too remote for the idiots to get there, or it's a special season or something like that, hunting just got too dangerous for me with all the idiots running around with guns who don't have a freaking clue what they're doing.