No it is not bad to want peace in your own home .. If anywhere you deserve to have your own space without the threat of drama or tension is your own home . Emphasis on your home. A home is a retreat , a safe haven, A place to find peace . However, this is also a home that belongs to other people who have the right to choose who they want for visitors . This is something you should talk to about with your roommates to find a common ground where you can both respect each other's space and needs . If they are so inclined to have her come to visit that is their prerogative .. You will have to find some way to deal with it .
Is it bad that I don't want my bf's ex hanging around the house which is a venueish space. She feels like I'm taking her friends away who live here but I just want to come home to a safe space n not have a 4 year relationship staring me in the face?
Sounds like you are scared he will go back to her which means you don't think he can move on from his past, and you are insecure about yourself and your ability to be a good enough girlfriend that he would want to keep.
It's not about you or your fears, its about trust. You should not be mistrustful about a relationship because you will create what you fear. Hold your head up and worry about you and him not him and her. She can be there or not and what you fear would happen will still happen. Why not just let her continue on and think, actually I can observe things better seeing her.
Then just make a relationship with him instead of living in fear of it 90% of the time. You will have a lot more enjoyment in that relationship trust me.