It is good that your boyfriend is taking an active interest in his baby. He may well be harbouring feelings for his ex so tread carefully and do not start a family until you are completely sure of his feelings towards you. If he takes the baby out and the mother stays at home I feel you have less to worry about. You might put feelers out as to whether you could be involved in taking the baby out to give the mother a break. Bringing up a baby is very hard work.
Just continue to trust him, that he will be faithful to you.
It's unfair that babies and sex go together. Dealing with men or woman who have had prior children or marriages can be a bitter pill to swallow. If you are to uncomfortably with his prior commitments, I'd suggest finding another mate with no children or prior marriages, which will make your life less complicated. If you feel he truly loves you and you love him you'll find away to make it work, but I doubt his schedule will change. Everyday is good for the baby mama and the baby. Suggest becoming a part of this schedule which might not fly to well. If he is just trying to be a good father and he and his x aren't still in love. There's no reason if your the new girlfriend that you can't be a part of all this togetherness unless the x doesn't and will not tolerate your presence. This would be a sign of something still going on in that picture. Life's to short to settle for less. Good luck!
Sorry I'm going through the same thing but mine is worse; the baby's Mom is pregnant again. I think if you can get out the situation run and put yourself first because that situation can get very messy, especially if it's been 2 years and another baby pops up and you're in love and want to fight for him lol.