i dont know him so i cant judge but are you sure they're not the ones sticking around? Last year my best friend who i kind of liked used to hang out with girls all the time, turns out they were the ones sticking to his side. He was too shy to tell them he didn't want to hang out so he just rolled on with it.
He might also be the type of guy who doesn't date but hangs out with girls instead?
Depends on the person, really.