Millions of dollars in unreported income, utilized sophisticated offshore dummy companies to help mask it. That warrants a prosecution. I've seen tax prosecutions for much less. And, while living in opulence technically is only marginally relevant, it provides terrible optics.
I have a civil case right now where an individual was running a telemarketing scam. He made millions, used a host of dummy companies to launder it, and spent like a drunken sailor. The killer, though, was that, AFTER he got sued by the government, he decided (i) to take his family to the British Open, and (ii) to buy a luxury suite to watch Dallas Cowboys games. That did not provide good optics.
Golfernot's comment is right -- tax cheats don't hang out with solid citizens. They hang out with other tax cheats. And, Gates isn't getting set free -- he's hoping for leniency, but there's nothing guarantied, and if he lied, the government will come down on him hard. Gates made the smart move, because the documentary evidence was overwhelming. Manafort should have tried to cut a similar plea deal (although I don't know if the government was entertaining palatable plea deals).
Take the politics out of it, and you have an arrogant tax cheat who should have been prosecuted. Gates is a rat, and nobody likes rats, but that's the way the criminal justice system works.