Do I get out now or try to make this work?

I would definitely confront him about it since it seems like he was doing that for almost 2 months when you first were together. But at the same time he might have felt those lingering feelings for her since he was his wife you know? And maybe he had cold feet starting a new relationship so he regressed back to her. But at least now he isn't talking to her I'm assuming and I'm also assuming that he cares a lot about you if you are moving in and whatnot. I would just tell him what happened about you finding the messages and asking him if he felt anything for his ex. But I'm just going to say that he was scared to start a new relationship but it seems like now things between you two are good if you are moving in together. I wish you the best.

Yes, agreed. Confront him. A large part of a relationship working is trust, and he obviously has abused that. Tell him how it makes you feel, and in the end, it's ultimately up to you whether you decide to stay or leave him. Hope I helped! Message if you need. xoxo - Darling Alice