Okay, for your size, you don't need to worry about touching the ground flat-footed or anything so that's good.
Personally, I'd get the KLR650 only if I wanted something to take on long road trips. That's where the huge gas tank and mini fairing make it the winner.
But for something more dirt-oriented, I'd go with the Honda XR650L. A little more modern suspension and frame than the KLR, and it definitely feels more nimble in the dirt. Things I'd add would be more aggressive tires (yet still DOT-legal) and a bigger tank because stock, it goes on reserve in about 70 miles. I think its brakes are superior to the KLR's as well.
But if you wanted something in between the XR-L's dirt-worthiness and the KLR's long-distance street orientation, then I'd suggest Suzuki's DR650. To me, it feels a little more comfortable on the street than the Honda, but it also is better in the dirt than the Kawasaki. Again, I'd opt for a larger aftermarket gas tank and better tires, unless you think you'll spend 90 percent of your time on pavement.
You might find used KTM 620s or 640s. Those are definitely more dirt-oriented and feel a tad rough on long road rides. They don't have quite the finished feel of the Japanese machines, though the newer KTMs come very close. Of course, if you're talking big twins, the 950 Adventure is awesome and clearly superior to the pre-2005 BMW GSs in the dirt. I've put a quite a few miles on the 950s with stock tires and found them to be pretty darn good in the dirt. Little jumps, no problem. Only thing is the high asking price currently.
For something completely different, of course, there are the ATKs and CCMs with their pretty bullet-proof Rotax motors. It's just that finding a good dealer close by for parts and service might be a long shot. Good luck and good riding.