You can argue that it is. Some people may say that it's not, but most people will. And if you ask me, I think that that is definitely rape.
It's quite simple: You were physically coerced to have sex with him. Although he didn't come on to you violently like in many unfortunate rape cases, he did force you and that should still count.
But then again, in the end you did have sex with him. If you decide to press charges, lawyers may say that that would be a sign of consent in the end. Even though that's just messed up, you should be aware of that as a possibility. So be careful about this.
I'm sorry that you had to experience something like this. What that guy did was wrong. You should never feel like you have to have sex with someone and people should respect that.