If this guy and his friends hate you, then try to stay away from him. I know that you may like him and stuff, but the chances of him changing his mind is very unlikely. That's why you just have to accept that he isn't the guy for you and move on. It's easier said than done, but it's better than chasing after a guy who you can't have. Going after someone who isn't into you is a waste of time and energy.
Plus why do you even like the guy anyway? From the sounds of it, he seems to be a jerk. A descent human being would never call someone a pervert just for talking or flirting with him. If he didn't like you but at least was an okay person, he'd probably make an excuse for leaving or just flirt back. But he flat out insulted you when you tried, thus making him immature and a moron. That is another reason why he should just stay out of your life and you from his. You deserve so much better than him.