The most important variable is how much time you spend with your spouse. I've known several otherwise marvelous, decent people who've succumbed to temptation when they've been apart from their spouses, around someone from the opposite sex. Does that mean that every situation in which you're alone with a member of the opposite sex will result in infidelity? Of course not and it's an idiotic interpretation of why some people do what Pence does in these situations. Unfortunately, most of us lack the ability to see into the future and identify which people will present us with those feelings in the right moment of weakness.
What Pence and people of his belief system understand and others do not understand is that human beings are fallen creatures and as such hopelessly flawed and susceptible to indiscretions and transgressions when presented with the opportunity. With regards to this issue, the staggering divorce statistics attributable in many/most cases to infidelity, the millions frequenting Ashley Madison and all sorts of other measures reinforce this recognition of the base fragility and weakness of human beings.
Pence will also be happier than the folks who get exercised over his custom. As a minister who'd counseled many couples told someone in my family who had been unfaithful: "I know you think you're going to find someone better...you won't." It's true. Those folks usually end up less contented than they were in the supposedly "unhappy marriage" they had before and often end up alone as they grow older, which we know results in most being less happy and often depressed. Pence's rule is a wise method to maintaining his marriage and I suspect that's what annoys some of the people critical of him on this count.
I won't even attempt to unwind how his practice demonstrates "disrespect for women."