It's nice that you are considering your boyfriend's feelings. But don't you think that you deserve to be treated the same way with your boyfriend considering your feelings?
I hate it when guys start talking about their exes and stuff like that because it kind of makes you feel like they're still into them or that they're comparing you to them. I'm guessing you feel somewhere along those lines too. That's why you should tell him. Don't worry about hurting his feelings. It's not fair that your feelings get thrown down just because of him. Your feelings still matter too. And when you tell him, be sure to be specific into why you feel that way and why you want him to stop.
If your boyfriend truly cares about you, he'd apologize and at least put in effort to try to stop or something like that. If he thinks that you're just being paranoid or whatever, then he's a turd! Because you are not being paranoid! He's the one who's being ignorant to your feelings and no one in your position should ever have their feelings disrespected. Plus being paranoid would be by getting overly jealous and stalking him in fear of him cheating on you with his exes. You're probably not doing that (I hope), so don't worry if you think that it'll come off as paranoid. And, like I said, if he thinks that you are coming off that way, he's the one with the problem.