I see a few issues. It doesn't seem as though you have any court orders at this point (or did the court issue temporary custody orders?). That is both good and bad. *Technically*, without a court order, you *could*, in theory, pick the kids up and then keep them. That may, or may not, be an issue down the line. If Mom chooses to retaliate in kind, your kids are like ping-pong balls. Which is never good.
You didn't mention how old the kids are, but I get the sense they are younger. If the only "proof" you have of Mom's transgressions are one child's word, that may not get you far. For one, depending on age, children's sense of time can be wonky (for lack of a better word). Also, courts really dislike putting kids in the middle, especially when it comes to "tattling" on a parent. Additionally, courts are loathe to control how either parent spends their parenting time. They will instill some boundaries (i.e. out of state/country travel requiring some type of notice/sharing of information, relocation, and so on) but as long as the kids are adequately sheltered, fed, clothed? They tend to err on the side of assuming a parent puts their child(ren)'s welfare as a priority.
Additionally, this whole C-19 thing is relatively uncharted territory on all fronts, including co-parenting. The court is unlikely to see "Mom's b/f is coming to dinner, according to kid, so the kids might be exposed" as an emergent situation, unlike "Mom's b/f is a convicted pedophile, he's coming over and babysitting the kids - who are the ages/genders of those he offended against - and I have admissible evidence of it all."
Honestly, all you can really do is make a decision that you are comfortable with - both for yourself, the kids, and your Mom. Maybe Mom would be amenable to regular Facetime sessions? Or would you be able to spend your parenting time more local to the ex (perhaps at a hotel - I haven't looked at CO's stay-at-home/essential business orders... camping?) - to lessen your Mom's exposure? What about going to Mom's area on each of your w/e days (when the weather's nice) and spending the days at a park? You can run around, go biking, have a picnic.... Just trying to think of some options.
On a slightly different note... This would be a good time to start bringing a longer-term plan vis-a-vis your living arrangements into play. If you have any hopes of a larger share of parenting time, you need to be in the same community. It is much more difficult to change orders than have them from the get-go. Courts want to see "I now have a home at 123 XYZ Rd, which is x.z miles from hum-a-nuh-hum-a-nuh ES/MS/HS that the children attend" not "yeah, I plan to move into the school district sometime in the next 3/6/12 months." It also makes it easier for the court to put specific relocation orders into play - the kids remain residing within hum-a-nuh-hum-a-nuh SD, absent mutual agreement/court order. Honestly - if you're not living in the SD now? There is little reason for the court to restrict Mom moving w/the kids. This is a serious discussion you should have with your lawyer. I'm also going to PM you a link to a father's support board. They have had similar discussions re C-19, parenting time, etc. They are great at helping other Dads formulate strategies, etc.
p.s. Also ditto the suggestion of a communication platform (was it clever red?) - lots of courts use Our Family Wizard. But there are others. Yes, there is a fee involved. But it keeps things more civil and the court can access the communications if needed.
