Police officers open carry every day. If mere open carry were brandishing or threatening, then every encounter a police officer has with a citizen would be considered a detainment. If a police officer asked you where the 7-11 is, you would consider yourself seized under the Terry rules. That would open a HUGE can of worms for the police and severely limit what the State can use as evidence.
EDIT to add: A holstered gun is not brandishing. Brandishing is:
...to wave or swing (something, such as a weapon) in a threatening or excited manner.
So unless you're a hula dancer on the side, your holstered gun cannot be brandished.
That said, there IS a way they might charge you for 'brandishing' even though your gun is properly holstered, and it has to do with something the OP mentions. Say you're carrying a concealed pistol and you go to WalMart to return a crap TV you foolishly purchased there. The clerk at the returns window refuses to give you a refund citing the cola dripping from the cooling vents. You brush your shirt back exposing your firearm and suggest he rethink his decision. THAT might be considered brandishing. It apparently was a problem here in WA when concealed carry first got started- people showing their guns to make a point.
Inadvertent exposure however is not brandishing.