Why did heexpect me not to care? My ex cheated on me so I left him, and after he said "I always made you promise never to leave me because I knew that you would know one day"

It depends on what relationship you had.
If you were only dating ... It is not "cheating" to date multiple people at the same time.  In fact, in most of the world it is common to have multiple "suitors" at the same time.

Why?  Because the whole point of dating is to find out what type of person you want to spend the rest of your life in marriage to.  And if you limit yourself to dating one person at a time, that would take too long.  >> Americans who think you only date one person at a time, and so limit their experience, are no doubt part of the reason our divorce rate is so high.

I see that you were engaged.....he cheated, you left, why are you sweating this? He made you promise you'd never leave? Well, sounds to me like he gave you enough reason to leave.

Apparently he knew that there were some things that he preferred not to restrict himself from doing even though he was intending to enter a relationship (marriage) that precluded them.

So he was probably trying to get an advanced permission slip from you for when he cheated.

And now he can blame you for the failure of his relationship with you.

This guy is a master manipulator.

Avoid him, it's unlikely he will ever change unless he seeks and obtains professional help.

Which only suggests that he sabotaged your marriage just to see how far he could push his limits .. Or he is simply trying to deflect blame on why your marriage failed. A classic manipulation technique.  There is always a certain level of asumptions when making a promise like 'I will never leave you' .. The assumption in this case, being that he wouldn't turn out to be a lying, cheating manipulative SOB.