Just an update, we had the chance to try several places suggested by in this thread.
Famous Louise's- very good, I had the catfish and split the calabash shrimp with one of my kids.
Nicks in Banner Elk- very good, even though both my wife and I ordered the sirloin steak and hers looked like a completely different cut than mine much thicker and more marbled like a ribeye. They were ordered medium and hers came out perfect while mine came out more medium well (mine was noticeably thinner) but the flavor was excellent. Good chili and sides also.
Foggy Rock (tried the Blowing Rock Ale House first but when we walked in they told it there was an hour and a half wait, called the foggy rock from the parking lot and they could seat us immediately) honestly prob the best food I ate while we were up there. I ordered the packer burger (pulled pork, bacon, and BBQ sauce topping a nice thick hamburger patty. My wife ordered their "French dip grilled cheese" (roast beef, sautéed onions and provolone cheese on sourdough) and my sister ordered their southern croissant melt (applewood bacon and homemade peach marmalade on a croissant with melted cheddar cheese on top) and we each tried a bite of the other. All three sandwiches were great but the croissant was the best. I wanted to order a couple of those just to take home lol.
We went to family billiards the night we ended up at nicks but they were having a big tournament and it was a little crowded for us.
Wasn't suggested on here but was close to where we were staying and we wanted pizza and wings one night so we tried the Banner Elk Cafe. I can't recommend them, the pizza was ok but not worth the cost, and the service was lacking. Was probably the only place we ate at there that I wouldn't feel comfortable sending someone.
Thanks for the suggestions guys and we certainly have a few "go to" restaurants in the area now for when we return. Wanted to try the Stone Walls but my BIL was a little "intimidated" by the outside I think. It looks a lot nicer/fancier than he was comfortable with. That may be a dinner date for the wife and I at a later point.