It's hard to say from a distance. There are several possibilities.
*He may enjoy solitude or just his family at times.
*He may not be able to handle much spontaneity, at least not from others.
*He may be selfish.
*He may not be that into your friendship.
*He may be a spy, and unsolicited phone calls could blow his cover. Nah..