I'd say be quite honest about it. That you enjoy the time, you enjoy the sex. That you can both maintain a healthy, loving family life and still find time for a special friend. That their mothers know and support you.
Once upon a time in our history - and not that long ago in the scheme of things - men were able to openly enjoy the kind of lives you have. It's healthy.
Certainly more healthy than having to sneak around as so many men are forced to do, like I did with my first wife by "going to the gym"
How would you have answered?