Symbols are weird sometimes. Take the swastika, which is a symbol of Indian origin, for example. We know that some people who use it are part of a particular group, but not all the people who use it are. Humans tend to see patterns and want to generalize to shortcut coming to conclusions about things, when really, more info is usually needed about a person than just "they have a certain patch or symbol on their arm, so they must belong to this group!" What if they saw the patch in a gas station and just thought it looked cool and were unaware of its implications? Context is key.