It doesn't have to be bed time for them for them to be out of your room. A "no kids in the adult" bedroom" rule is reasonable as long as it's consistent for all kids in the house.

My skids are 5 and 8.  They're usually in bed at 8:30 on school nights, but on weekends they often stay up much later- 10 would not be unheard of.  BUT, they're not in my bedroom - either we're all in the common area doing something together or they're in their rooms.

Kick the kids out, lock the door, put in some ear plugs, and who cares what time they go to bed, let your SO deal with that.

I was not engaged at all in parenting SS, but I would have had zero trouble telling him it was time to get out of my room, and enforcing it, if I needed rest.

Why are they in your room in the first place?

Get up out of bed and go lock yourself in one of their rooms to "hang out in". When they start complaining that it's "their room"... point out that they can do it to you, so why is it not okay for you to just go in their room and act like it's yours?"

I have found that they don't always clue in on how their behavior or attitude is disrespectful until the shoe is on the other foot. 
don't let them make the rules, make sure they know it's you and DH that does that

Get up out of bed and go lock yourself in one of their rooms to "hang out in". When they start complaining that it's "their room"... point out that they can do it to you, so why is it not okay for you to just go in their room and act like it's yours?"

I have found that they don't always clue in on how their behavior or attitude is disrespectful until the shoe is on the other foot. 
don't let them make the rules, make sure they know it's you and DH that does that