Stop being the other woman! Don’t give him his cake and let him eat it too. Leave him and see how he reacts. Either way your problem will be fixed.