Hugs to you. Set your boundaries, and if you have a spare bedroom in your house, start it now. One of you move out and let the kids see that mom and dad are not ok, and they still love them.
I will tell you that your "separate flat" idea is a great idea. In fact, I would state is the best one I have heard yet.
As for the concern about female stuff, trust me on this he will benefit more from it they you. Why? If he brings a woman over and sees accommodations for women, his bonus points should go up.
Is it cheating? Well, Marriage is what two people define it as, and yes laws and lawyers can mark that as a actionable view. There are great lawyers here on Medium that you can maybe reach out too to protect yourself and your rights.
If you muzzle your desires, your sexuality, and you, you actually do more harm to the kids then faking it until you make it. Of course please talk to a therapist about this as they are far more versed at it.
As for shattering his world? Fuck him, neglect is abuse. As for the children, they can learn how to end things properly and with consent and respect. You can show it and if your partner doesn't, then taking the high road makes you the better parent.
Be well, hugs (consensual to you), and remember that the sexiest thing about a woman to a man that is worth his weight in gold in her bed is "confidence" and "liberation".
Not the cup size of her. breasts or her weight or how many lines or tiger stripes for being a fighter woman on her belly.
That shit is vapid ... be real, be weird, be you.
Now Be Well!