Their stupid confusion need not make you ill.
People who project their feelings/assessments onto others are idiots and the world is full of them. Fortunately, there are mature people who do not think this way and have the experience or wisdom to understand what is going on, or if they do not understand what is going on they refrain from hasty judgement.