It's a bit fishy, especially since they got defensive about it. Personally, I keep all my exes' phone numbers and if people ask me why I kept their numbers, I'd just tell them the truth; that I became friends with them and we like to talk every once in a while or they happen to be in the same club/activity as I am and we need to talk sometimes (even though we really don't want to). I'm very open about why I do what I do because I've got nothing to hide. But the fact that your partner is closed off about this can be something you should watch out for. I'm not saying that your partner is still madly in love with their ex; I'm just saying that your partner does harbor some feelings for them. It's not always something to freak out about because it can go away sometimes. Although other times, those feelings won't go away throughout your entire relationship because you are their rebound.
You need to somehow be able to distinguish which one your partner most likely is and go on deciding from then. If you feel like your partner is very much ready to move on completely sometime in the very near future, then go ahead and wait for them but still be cautious about where your partner stands. If you feel like they can't get over their ex so easily anytime soon, then it's best to just leave them because they obviously still have some feelings to sort out and you don't want to waste your time being in a relationship where your partner doesn't know what they want or what they want isn't you. Don't be someone else's rebound when you don't want to be. Everyone deserves better than to be used like that. Trust me, I've been down that road and it did not end pretty.
Anyway, I wish you luck on this! May you find your path soon :)
