He is invading your personal space. Back up and if he keeps on leaning to close tell him you are uncomfortable. I have actually said, get out of my face. The person said it hurt their feelings, but continued to do so, so now we are not even friends. It can be a form of trying to control you.
