I am a lady and there is a coworker of mine whom I like a lot...If I ask him out for a coffee after work would that be sexual harrasment?

Not at all but youre entering dangerous grounds of an office romance which is very dangerous and may be against company Policies too so please be careful here.  99% of all office romances never work out and cause a lot of problems in the work place after breaking up.  Good luck